-- Logs begin at Sat 2022-09-10 20:01:15 UTC, end at Sat 2022-09-10 22:12:19 UTC. -- Sep 10 20:30:12.292003 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Sep 10 20:30:12.300739 ncdv-master-1 devstack@keystone.service[93513]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 10 20:30:12 2022] *** Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: nodename: ncdv-master-1 Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: machine: x86_64 Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: clock source: unix Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: pcre jit disabled Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: detected number of CPU cores: 4 Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: current working directory: / Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: detected binary path: /usr/bin/uwsgi-core Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: your processes number limit is 47680 Sep 10 20:30:12.302838 ncdv-master-1 devstack@keystone.service[93513]: your memory page size is 4096 bytes Sep 10 20:30:12.303456 ncdv-master-1 devstack@keystone.service[93513]: detected max file descriptor number: 2048 Sep 10 20:30:12.303456 ncdv-master-1 devstack@keystone.service[93513]: lock engine: pthread robust mutexes Sep 10 20:30:12.303456 ncdv-master-1 devstack@keystone.service[93513]: thunder lock: enabled Sep 10 20:30:12.303456 ncdv-master-1 devstack@keystone.service[93513]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 10 20:30:12.303456 ncdv-master-1 devstack@keystone.service[93513]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 10 20:30:12.347937 ncdv-master-1 devstack@keystone.service[93513]: Python main interpreter initialized at 0x5630e69c09c0 Sep 10 20:30:12.347937 ncdv-master-1 devstack@keystone.service[93513]: python threads support enabled Sep 10 20:30:12.347937 ncdv-master-1 devstack@keystone.service[93513]: your server socket listen backlog is limited to 100 connections Sep 10 20:30:12.347937 ncdv-master-1 devstack@keystone.service[93513]: your mercy for graceful operations on workers is 90 seconds Sep 10 20:30:12.348302 ncdv-master-1 devstack@keystone.service[93513]: mapped 403077 bytes (393 KB) for 2 cores Sep 10 20:30:12.348850 ncdv-master-1 devstack@keystone.service[93513]: *** Operational MODE: preforking *** Sep 10 20:30:12.348850 ncdv-master-1 devstack@keystone.service[93513]: *** uWSGI is running in multiple interpreter mode *** Sep 10 20:30:12.348850 ncdv-master-1 devstack@keystone.service[93513]: spawned uWSGI master process (pid: 93513) Sep 10 20:30:12.348866 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Sep 10 20:30:12.353323 ncdv-master-1 devstack@keystone.service[93513]: spawned uWSGI worker 1 (pid: 93527, cores: 1) Sep 10 20:30:12.353323 ncdv-master-1 devstack@keystone.service[93513]: spawned uWSGI worker 2 (pid: 93528, cores: 1) Sep 10 20:30:12.353323 ncdv-master-1 devstack@keystone.service[93513]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 10 20:30:16.993331 ncdv-master-1 devstack@keystone.service[93528]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:30:16.993331 ncdv-master-1 devstack@keystone.service[93528]: from cryptography.utils import int_from_bytes Sep 10 20:30:16.993952 ncdv-master-1 devstack@keystone.service[93528]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:30:16.993952 ncdv-master-1 devstack@keystone.service[93528]: from cryptography.utils import int_from_bytes Sep 10 20:30:16.998694 ncdv-master-1 devstack@keystone.service[93527]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:30:16.998694 ncdv-master-1 devstack@keystone.service[93527]: from cryptography.utils import int_from_bytes Sep 10 20:30:16.999145 ncdv-master-1 devstack@keystone.service[93527]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 20:30:16.999145 ncdv-master-1 devstack@keystone.service[93527]: from cryptography.utils import int_from_bytes Sep 10 20:30:17.043443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 20:30:17.043634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 20:30:17.043822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] command line args: [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 20:30:17.044088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 20:30:17.044264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ================================================================================ {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 20:30:17.044512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] admin_token = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.044752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] config_dir = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.044935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.045114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] config_source = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.045293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] debug = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.045494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.046047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.046047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.046278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.046522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.046615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 20:30:17.046754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] list_limit = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.046838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 20:30:17.046900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] log_config_append = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.046969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] command line args: [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 20:30:17.047118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.047255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 20:30:17.047384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] log_dir = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.047517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ================================================================================ {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 20:30:17.047652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] log_file = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.047652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.047915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] admin_token = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] config_dir = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] config_source = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] debug = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.048941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] list_limit = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.049945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] log_config_append = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] log_dir = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] log_file = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.050968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] publish_errors = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.051995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] standard_threads = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.052984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] use_journal = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] use_json = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.053854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] use_stderr = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.054071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] use_syslog = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.054254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.054450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.054596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] publish_errors = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.054735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.054939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.055019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.055242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.055402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.055481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.055552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.055631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.055709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.055812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.055903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.056043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] standard_threads = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.056124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.056268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.056346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.056514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.056597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.056726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.056811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.056881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] use_journal = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.057020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.057097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] use_json = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.057163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] use_stderr = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.057227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.057344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] use_syslog = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.057454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.057574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 20:30:17.057662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.057858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.057970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.058909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.059859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.060931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.061964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.062930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.063926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.064948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.065921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.066968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.067891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.068948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.069954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.070979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.071927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.072994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.073941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.074974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.075908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.076964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.077979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.078920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] auth.external = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.079998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] auth.password = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] auth.token = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.080987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.081914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] credential.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.082851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] auth.external = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.083956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] auth.password = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] auth.token = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.084961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.085635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.085721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.085955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.086039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.086218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.086456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.086535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.086705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.086791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.086937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.087303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.087424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] credential.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.087529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.087665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.087781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.087908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.088925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.089930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.090891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.091945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.092950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.093957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.094874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.095944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.096937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.097960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.098063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.098167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.098274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.098409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.098511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.098609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.098705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.099760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.099760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.099760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.099971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.100767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.101577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.101732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.101813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.102951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.103971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.104930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.105855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.106943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.107972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.108893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.109996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.110939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.111945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.112869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.113983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] role.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] role.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.114985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.115903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.116066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.116153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.116296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.116381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.116589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.116667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.116836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.117926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] role.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] role.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.118935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.119984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.120933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.121894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.122943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.123972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.124910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.125851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.126938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.connection = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.127970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.128886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.129985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.130919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.131005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.connection = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.131152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.131231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.131381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.131469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.131543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93528) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 20:30:17.131642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.131881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.132166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.132613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.132878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.133118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.133365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.133601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.133873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.134112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.134358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.134591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.134835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.135061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.135299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.135529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 20:30:17.135698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 20:30:17.162036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.162248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.162484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.166278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.166517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.166718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.171271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.175283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.201439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.201650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.203924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.204154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.254218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.254400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.303499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.303730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.305112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.305514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.372252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.372527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.372711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.372859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.373005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.373146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.373285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.373434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.373575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.373725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.373725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.374014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.397237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.397512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.398297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.398438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.398567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.398715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.398849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.398979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.399108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.399237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.399365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.399365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.399629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.399795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.410178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.410881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.423284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.423513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.423776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.423975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93528) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.424911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.425157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.425280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 20:30:17.441332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.441482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.441621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:30:17.441758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:30:17.441910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.442038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.442161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:30:17.442286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 20:30:17.442420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.442545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.442669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.442792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.442916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.443048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.443183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.443308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.453202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.453479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.453607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.453743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.453900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.454026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.454152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.454328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.461991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.462577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.465123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.465407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.465528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.465688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.465771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.465994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.466080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.466275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.466372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.466535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.466618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.466803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.466885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.467945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.468128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.492421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.493115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.493909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.494195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.494606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.494888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.498865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.499149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.499413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.499497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.499696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.500088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.505883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.506137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.506500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.506782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.510805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.511097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.511396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.511680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.511763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.511957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.512040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.512248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.512340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.512522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.512648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.512806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.512924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.513206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.513474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.513739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.528456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.528732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.529677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.529983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.532599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.533837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.535828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.536122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.536935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.537345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.541360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.542363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.544503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.545599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.547718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.548015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.548299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.548591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.548709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.548870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.549005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.549159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.549287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.549682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.549998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.550280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.565492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.565781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.566075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.566339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.566880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.567171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.567437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.567699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.573277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.574836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.574934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.576372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.576488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.576646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.578055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.578332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.579262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.579560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.579858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.580138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.580525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.581064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.581348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.581650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.581959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.582237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.592011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.592297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.592578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.592860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.593258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.593540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.593844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.594107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.594394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.594674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.594955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.595236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.595515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.595801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.609794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.612076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.612710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.615112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.615637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.617801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.619020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.619340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.619626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.620018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.620466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.621949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.622243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.622535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.626405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.627866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.629916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.630759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.631377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.632071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.632361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.632671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.634633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.634926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.635210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.635487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.640585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.640880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.641161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.643460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.643753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.644033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.648165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93528) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.648418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.648695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.648967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.649236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.649527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.649845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.650135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.650412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.650678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.650947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.651205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.651296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 20:30:17.651475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.651599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.651755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93528) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.651837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.652063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.652347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.652619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.652888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.653237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.653415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.653695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.654002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.654364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.654645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.654905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 20:30:17.686007 ncdv-master-1 devstack@keystone.service[93528]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5630e69c09c0 pid: 93528 (default app) Sep 10 20:30:17.689233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ebe9-f97e-43dd-ab65-47b2bd5d27cc None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:17.689337 ncdv-master-1 devstack@keystone.service[93527]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5630e69c09c0 pid: 93527 (default app) Sep 10 20:30:17.689519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ebe9-f97e-43dd-ab65-47b2bd5d27cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:17.689796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ebe9-f97e-43dd-ab65-47b2bd5d27cc None None] PATH_INFO: `/v3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:17.694055 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Sat Sep 10 20:30:17 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:24.986210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea411731-e225-4c13-a0e3-c290d3f6b85f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:24.986603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea411731-e225-4c13-a0e3-c290d3f6b85f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:24.986769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea411731-e225-4c13-a0e3-c290d3f6b85f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:24.991003 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:24.997117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:24.997648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:24.997648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:25.074951 ncdv-master-1 devstack@keystone.service[93528]: INFO dbcounter [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] Registered counter for database keystone Sep 10 20:30:25.094140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG oslo_db.sqlalchemy.engines [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93528) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 20:30:25.100304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writer thread running {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 20:30:25.278522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG passlib.handlers.bcrypt [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93528) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 20:30:25.278833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG passlib.handlers.bcrypt [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93528) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 20:30:25.337374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:25.338986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:25.470008 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 10 20:30:25.812202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea411731-e225-4c13-a0e3-c290d3f6b85f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:25.889815 ncdv-master-1 devstack@keystone.service[93527]: INFO dbcounter [None req-ea411731-e225-4c13-a0e3-c290d3f6b85f None None] Registered counter for database keystone Sep 10 20:30:25.911556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG oslo_db.sqlalchemy.engines [None req-ea411731-e225-4c13-a0e3-c290d3f6b85f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93527) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 20:30:25.918015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writer thread running {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 20:30:26.037076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea411731-e225-4c13-a0e3-c290d3f6b85f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:26.041784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce745ce9-bdda-464a-b3f3-df5806b0b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:26.042439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745ce9-bdda-464a-b3f3-df5806b0b992 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:26.042720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745ce9-bdda-464a-b3f3-df5806b0b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:26.042989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745ce9-bdda-464a-b3f3-df5806b0b992 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:26.051222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce745ce9-bdda-464a-b3f3-df5806b0b992 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:26.104541 ncdv-master-1 devstack@keystone.service[93527]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce745ce9-bdda-464a-b3f3-df5806b0b992 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 20:30:26.104824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce745ce9-bdda-464a-b3f3-df5806b0b992 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:26.114537 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Sat Sep 10 20:30:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:30:26.120774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:26.137886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532be74-c677-42b2-a9c4-7fd91a1d4dca None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:26.142259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad98a7-a756-4ce3-b9a8-89d0e8bd69ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:26.143337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad98a7-a756-4ce3-b9a8-89d0e8bd69ff admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:26.143613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad98a7-a756-4ce3-b9a8-89d0e8bd69ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:26.143885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad98a7-a756-4ce3-b9a8-89d0e8bd69ff admin admin] PATH_INFO: `/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:26.145312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad98a7-a756-4ce3-b9a8-89d0e8bd69ff admin admin] RBAC: Authorizing `identity:get_project(project_id=ff6cfd9ba910450d96d6133cd5225f2f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:26.196943 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.common.rbac_enforcer.enforcer [None req-acad98a7-a756-4ce3-b9a8-89d0e8bd69ff admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 20:30:26.197289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad98a7-a756-4ce3-b9a8-89d0e8bd69ff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:26.201805 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Sat Sep 10 20:30:26 2022] GET /identity/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:28.604226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec95d8-d38a-4741-a569-a248bb824df0 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:28.604757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec95d8-d38a-4741-a569-a248bb824df0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:28.604757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec95d8-d38a-4741-a569-a248bb824df0 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:28.608803 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:28.614395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857ebd6-14aa-4251-ab2a-a65bceaaa604 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:28.614790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857ebd6-14aa-4251-ab2a-a65bceaaa604 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:28.614932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857ebd6-14aa-4251-ab2a-a65bceaaa604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:28.636495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c857ebd6-14aa-4251-ab2a-a65bceaaa604 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:28.638086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c857ebd6-14aa-4251-ab2a-a65bceaaa604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:28.645959 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 10 20:30:28.973341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec95d8-d38a-4741-a569-a248bb824df0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:28.986625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec95d8-d38a-4741-a569-a248bb824df0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:28.991237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb7404-c8af-4d7c-8d05-f78be0fa2d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:28.991845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7404-c8af-4d7c-8d05-f78be0fa2d2f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:28.992242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7404-c8af-4d7c-8d05-f78be0fa2d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:28.992526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7404-c8af-4d7c-8d05-f78be0fa2d2f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:29.000814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb7404-c8af-4d7c-8d05-f78be0fa2d2f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:29.002755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb7404-c8af-4d7c-8d05-f78be0fa2d2f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:29.012516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Sat Sep 10 20:30:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 20:30:29.018267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857ebd6-14aa-4251-ab2a-a65bceaaa604 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:29.028104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857ebd6-14aa-4251-ab2a-a65bceaaa604 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:29.032798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02038b00-7ea0-42c2-836a-43cd24d28691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:29.033426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038b00-7ea0-42c2-836a-43cd24d28691 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:29.033711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038b00-7ea0-42c2-836a-43cd24d28691 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:29.034036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038b00-7ea0-42c2-836a-43cd24d28691 admin admin] PATH_INFO: `/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:29.035373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02038b00-7ea0-42c2-836a-43cd24d28691 admin admin] RBAC: Authorizing `identity:get_user(user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:29.037219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02038b00-7ea0-42c2-836a-43cd24d28691 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:29.038861 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Sat Sep 10 20:30:29 2022] GET /identity/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:31.563690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4c0b4-603b-4684-bada-48e92c9d0606 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:31.564207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4c0b4-603b-4684-bada-48e92c9d0606 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:31.564288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4c0b4-603b-4684-bada-48e92c9d0606 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:31.565558 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:31.571420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4eac8-218e-416f-b6ae-af0ff4614a73 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:31.571870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4eac8-218e-416f-b6ae-af0ff4614a73 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:31.571964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4eac8-218e-416f-b6ae-af0ff4614a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:31.593162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e74f-8688-4173-9692-9ec66e9a6a64 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:31.593454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e74f-8688-4173-9692-9ec66e9a6a64 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:31.593547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-86c4eac8-218e-416f-b6ae-af0ff4614a73 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:31.593737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e74f-8688-4173-9692-9ec66e9a6a64 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:31.594943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-86c4eac8-218e-416f-b6ae-af0ff4614a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:31.595079 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:31.600146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:31.601046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:31.601416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:31.647859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG passlib.handlers.bcrypt [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93527) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 20:30:31.648528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG passlib.handlers.bcrypt [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93527) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 20:30:31.699451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:31.701338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:31.794152 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:31.806677 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:30:32.128904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4eac8-218e-416f-b6ae-af0ff4614a73 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.132817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.155146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:32.245689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4eac8-218e-416f-b6ae-af0ff4614a73 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.246644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02a41b-3547-48e2-8d5c-f008298b193f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.248964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:32.282642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.283240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.283524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.284894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.287570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.288806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.289133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.289410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.297263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:32.307209 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:32.308166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:32.308991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:32.314323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.374040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:32.375667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:32.403463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:32.423190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62456a84-02bf-4c87-9756-baf4b57cd4cb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.462842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.463463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.463748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.464285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.464695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:32.465000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:32.466429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:32.479935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:32.485433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.493285 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:30:32.499220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.502074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:32.606764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19845c7-a39e-4511-ab0a-6d3c648ce05f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.624897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc99b-74d1-45f2-bd5b-39c54da306eb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.645169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.645989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.646349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.646695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] PATH_INFO: `/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.648308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] RBAC: Authorizing `identity:get_user(user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:32.649815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:32.651560 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:32.657592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.675139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:32.770449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.771241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.771522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.771976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.775630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffce7b-a4f6-4170-b7b0-0524ede64eab None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.792972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:32.794608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:30:32.814421 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:32.815334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:30:32.816770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:30:32.816770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:30:32.816770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:30:32.816770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:30:32.817009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.817294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.817573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.817904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.819140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:32.820634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:32.821664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.822337 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:32.827949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:32.886772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:32.919692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:32.941486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023edbf-5b04-46c6-820e-ef69bfde8e0e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.942387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca47f4-2e95-4af6-8a57-fc4e8a6e7f8a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:32.983245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c84b4d-6674-4edd-97a2-765bb87d2755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.983413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b75f3-e2f8-468f-b752-3e0d6ef361ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:32.983867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84b4d-6674-4edd-97a2-765bb87d2755 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.984019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b75f3-e2f8-468f-b752-3e0d6ef361ed None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:32.984149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84b4d-6674-4edd-97a2-765bb87d2755 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.984471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b75f3-e2f8-468f-b752-3e0d6ef361ed None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:32.984608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84b4d-6674-4edd-97a2-765bb87d2755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.984852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b75f3-e2f8-468f-b752-3e0d6ef361ed None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:32.985493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84b4d-6674-4edd-97a2-765bb87d2755 None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.domain.id=default, user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:32.985583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b75f3-e2f8-468f-b752-3e0d6ef361ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:32.985806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84b4d-6674-4edd-97a2-765bb87d2755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:32.986130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b75f3-e2f8-468f-b752-3e0d6ef361ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:32.987303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84b4d-6674-4edd-97a2-765bb87d2755 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:32.987573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b75f3-e2f8-468f-b752-3e0d6ef361ed None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:32.995852 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/role_assignments?user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6&scope.domain.id=default&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 233 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:33.007667 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:36.591145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25e2e-b5d0-4f6c-9712-c48ccb9abf15 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:36.592173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25e2e-b5d0-4f6c-9712-c48ccb9abf15 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:36.592683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25e2e-b5d0-4f6c-9712-c48ccb9abf15 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:36.594405 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:36.599656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b604b3-ebbd-4a38-bad9-5782e0b0d153 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:36.600566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b604b3-ebbd-4a38-bad9-5782e0b0d153 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:36.601047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b604b3-ebbd-4a38-bad9-5782e0b0d153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:36.627089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-98b604b3-ebbd-4a38-bad9-5782e0b0d153 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:36.628617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-98b604b3-ebbd-4a38-bad9-5782e0b0d153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:36.706409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:36.893347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044984f-9bd3-4792-8d84-e053330ebf9a None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:36.894010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044984f-9bd3-4792-8d84-e053330ebf9a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:36.894517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044984f-9bd3-4792-8d84-e053330ebf9a None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:36.896309 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 13/24] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:36.902302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c01ea9-9f73-4c2f-ba2d-ad63c22c9841 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:36.902846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c01ea9-9f73-4c2f-ba2d-ad63c22c9841 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:36.903320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c01ea9-9f73-4c2f-ba2d-ad63c22c9841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:36.927069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-00c01ea9-9f73-4c2f-ba2d-ad63c22c9841 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:36.928691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00c01ea9-9f73-4c2f-ba2d-ad63c22c9841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:36.984027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:37.019415 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 12/25] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:37.052848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8fe5d-c0c0-46e4-80e9-910752e5eaad None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:37.053133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8fe5d-c0c0-46e4-80e9-910752e5eaad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:37.054256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8fe5d-c0c0-46e4-80e9-910752e5eaad None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:37.056558 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 14/26] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:37.067992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a9862-898e-4938-819c-5ac95f70f690 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:37.068488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a9862-898e-4938-819c-5ac95f70f690 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:37.068905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a9862-898e-4938-819c-5ac95f70f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:37.095788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-906a9862-898e-4938-819c-5ac95f70f690 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:37.097217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-906a9862-898e-4938-819c-5ac95f70f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:37.196898 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 13/27] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:37.259003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8fe5d-c0c0-46e4-80e9-910752e5eaad None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:37.381898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b9f5-c130-4f2d-820a-5e85bbcf2885 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:37.381898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b9f5-c130-4f2d-820a-5e85bbcf2885 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:37.381898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b9f5-c130-4f2d-820a-5e85bbcf2885 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:37.382954 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 14/28] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:30:37.391208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644d3f-7cc8-4d6a-8302-6dc35bd52dfb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:37.391723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644d3f-7cc8-4d6a-8302-6dc35bd52dfb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:37.392197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644d3f-7cc8-4d6a-8302-6dc35bd52dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:37.453886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-70644d3f-7cc8-4d6a-8302-6dc35bd52dfb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:37.455403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-70644d3f-7cc8-4d6a-8302-6dc35bd52dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:37.510699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8fe5d-c0c0-46e4-80e9-910752e5eaad None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:37.595798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 15/29] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:30:37.598210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70644d3f-7cc8-4d6a-8302-6dc35bd52dfb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:37.611150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:37.611812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:37.612062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:37.612334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:37.635954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:37.635954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:37.646669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:37.671950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:30:37.676314 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:37.677174 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:30:37.678004 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:30:37.678004 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:30:37.678004 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:37.678989 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:37.685930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:37.822644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70644d3f-7cc8-4d6a-8302-6dc35bd52dfb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:37.847067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd35ba-dc98-4aa0-95d4-ee1768b42ad5 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:37.886655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:37.887266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:37.887551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:37.887826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:37.888474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:37.888776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:37.890299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:37.903157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:37.903819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:37.904103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:37.904377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:37.911196 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 16/31] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:37.912366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:37.931483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:37.945339 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:37.946315 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:37.947065 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:30:37.952048 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 16/32] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:37.953247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:38.044313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:38.074004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:38.080658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f577088-4611-4f22-a865-62be43402a2b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:38.103147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd09f-6656-4ca3-9dd9-58393bbc0ec8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:38.119083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:38.119999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.120460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.120894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.121975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:38.123574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:38.142124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6823-9e63-45b3-b245-d5be22ba2f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:38.142736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6823-9e63-45b3-b245-d5be22ba2f44 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.143018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6823-9e63-45b3-b245-d5be22ba2f44 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.143289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6823-9e63-45b3-b245-d5be22ba2f44 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.143959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d6823-9e63-45b3-b245-d5be22ba2f44 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:38.145449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d6823-9e63-45b3-b245-d5be22ba2f44 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:38.270773 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 17/33] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:30:37 2022] POST /identity/v3/projects => generated 301 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:30:38.271983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:38.282087 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 17/34] 10.222.0.33 () {60 vars in 1131 bytes} [Sat Sep 10 20:30:37 2022] POST /identity/v3/roles => generated 219 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:30:38.282087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42170b3-9daf-4a12-ab88-ea3672d4ca8c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.282087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42170b3-9daf-4a12-ab88-ea3672d4ca8c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.282087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42170b3-9daf-4a12-ab88-ea3672d4ca8c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.282087 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 18/35] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:38.282087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42170b3-9daf-4a12-ab88-ea3672d4ca8c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:38.469007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42170b3-9daf-4a12-ab88-ea3672d4ca8c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:38.491860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:38.527985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818a82f-fbc1-432a-8bc7-cd6359af893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:38.528625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818a82f-fbc1-432a-8bc7-cd6359af893a None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.528909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818a82f-fbc1-432a-8bc7-cd6359af893a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.529187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818a82f-fbc1-432a-8bc7-cd6359af893a None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.529911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818a82f-fbc1-432a-8bc7-cd6359af893a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:38.535312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1992b-aa6f-45ae-a735-0d9e5f767a15 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:38.537183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818a82f-fbc1-432a-8bc7-cd6359af893a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:38.565698 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 19/36] 10.222.0.33 () {60 vars in 1131 bytes} [Sat Sep 10 20:30:38 2022] POST /identity/v3/roles => generated 225 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:30:38.567857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718ac05-23b8-4220-838e-e45424218a3e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.568147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718ac05-23b8-4220-838e-e45424218a3e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.568423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718ac05-23b8-4220-838e-e45424218a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.599378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:38.599596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3718ac05-23b8-4220-838e-e45424218a3e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:38.601301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.601750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3718ac05-23b8-4220-838e-e45424218a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:38.602234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.602663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.603668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:38.604131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:38.605958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:38.633757 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 18/37] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:38.634977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:38.653720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:38.717745 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 20/38] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:38.791418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efb13c-aafa-4d3f-9b0c-c2198bc32491 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:38.833588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:38.834270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.834686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.834964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] PATH_INFO: `/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.836727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] RBAC: Authorizing `identity:get_user(user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:38.838604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:38.840084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dca807-4c52-4584-9e42-294468beeb9e None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.840272 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 19/39] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:38 2022] GET /identity/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:38.840435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dca807-4c52-4584-9e42-294468beeb9e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.840720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dca807-4c52-4584-9e42-294468beeb9e None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.842688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 21/40] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:30:38.845575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:38.849933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d0ad4-bfaa-45e4-aa8c-85cce602a0cc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:38.850505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d0ad4-bfaa-45e4-aa8c-85cce602a0cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:38.851956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d0ad4-bfaa-45e4-aa8c-85cce602a0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:38.875187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-444d0ad4-bfaa-45e4-aa8c-85cce602a0cc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:38.876660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-444d0ad4-bfaa-45e4-aa8c-85cce602a0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:38.942831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:38.956384 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 22/41] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:38.969031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9891-6473-4f16-aa7e-2136dd084b9b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:39.007980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:39.008668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:39.008956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:39.009229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:39.010450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:39.011959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:39.013774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 20/42] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:39.019163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d0ad4-bfaa-45e4-aa8c-85cce602a0cc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:39.055745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:39.063339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:39.125085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d0ad4-bfaa-45e4-aa8c-85cce602a0cc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:39.164490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:39.165150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:39.165431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:39.165713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] PATH_INFO: `/v3/domains/default/users/cd34ffbc794a4c04a8a5c4f195b6d3b6/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:39.167703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:39.169141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:39.187029 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 23/43] 10.222.0.33 () {58 vars in 1361 bytes} [Sat Sep 10 20:30:39 2022] PUT /identity/v3/domains/default/users/cd34ffbc794a4c04a8a5c4f195b6d3b6/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:30:39.192508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582592-78eb-4d9d-9227-12b1dcef3c03 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:39.233787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:39.234448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:39.234736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:39.235014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:39.235694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:39.237104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:39.254930 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 21/44] 10.222.0.33 () {60 vars in 1131 bytes} [Sat Sep 10 20:30:39 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:30:39.286261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:39.401580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:39.409742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363f291-d31e-40ac-a09d-1d984f55bfab None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:39.450371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d89f0-0ff2-45bd-a361-1223aeab766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:39.451064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d89f0-0ff2-45bd-a361-1223aeab766b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:39.451352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d89f0-0ff2-45bd-a361-1223aeab766b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:39.451626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d89f0-0ff2-45bd-a361-1223aeab766b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:39.452790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d89f0-0ff2-45bd-a361-1223aeab766b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:39.454321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d89f0-0ff2-45bd-a361-1223aeab766b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:39.456119 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 24/45] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:39.462672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:39.472521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:39.573239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196ca8c-7865-416c-acae-1e3e1882048a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:39.606954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80977a9-98bf-4956-a5b0-657d7de5e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:39.607564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80977a9-98bf-4956-a5b0-657d7de5e801 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:39.607849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80977a9-98bf-4956-a5b0-657d7de5e801 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:39.608126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80977a9-98bf-4956-a5b0-657d7de5e801 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:39.608774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80977a9-98bf-4956-a5b0-657d7de5e801 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:39.610266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80977a9-98bf-4956-a5b0-657d7de5e801 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:39.636753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 22/46] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:30:39 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:30:41.628193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb22dd8-4bbb-4393-bda5-43e030a5410c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:41.628759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb22dd8-4bbb-4393-bda5-43e030a5410c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:41.628759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb22dd8-4bbb-4393-bda5-43e030a5410c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:41.630147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 25/47] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:41.636862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a7125-1550-47c3-ae55-56a537940c4e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:41.637382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a7125-1550-47c3-ae55-56a537940c4e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:41.637508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a7125-1550-47c3-ae55-56a537940c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:41.656655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-165a7125-1550-47c3-ae55-56a537940c4e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:41.658181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-165a7125-1550-47c3-ae55-56a537940c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:41.710423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:41.725481 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 23/48] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:42.042030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeaf77-a255-4285-a837-a8b53cc9781d None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.042314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeaf77-a255-4285-a837-a8b53cc9781d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.042596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeaf77-a255-4285-a837-a8b53cc9781d None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.043925 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 26/49] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:42.049183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b202d-a69c-4857-9ef7-5521d16fbcea None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.049492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b202d-a69c-4857-9ef7-5521d16fbcea None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.049747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b202d-a69c-4857-9ef7-5521d16fbcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.052471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeaf77-a255-4285-a837-a8b53cc9781d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:42.067583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a02b202d-a69c-4857-9ef7-5521d16fbcea None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:42.068765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a02b202d-a69c-4857-9ef7-5521d16fbcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:42.139168 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 24/50] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:42.155530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeaf77-a255-4285-a837-a8b53cc9781d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:42.188347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:42.188975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.189264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.189544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.200533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:42.205461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:42.206544 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:42.207300 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 27/51] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:30:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:30:42.213362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b202d-a69c-4857-9ef7-5521d16fbcea None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:42.308811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b202d-a69c-4857-9ef7-5521d16fbcea None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:42.325770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:42.343043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:42.343659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.343947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.344227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.344890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:42.345199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:42.346617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:42.355143 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 25/52] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:42.362090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:42.391167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:42.463615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:42.465485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ca5c-cd5a-42c3-a8df-796673a643f7 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:42.499804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:42.500440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.500726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.501006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] PATH_INFO: `/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.502192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] RBAC: Authorizing `identity:get_user(user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:42.503655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:42.505236 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 28/53] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:42 2022] GET /identity/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:42.510704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:42.566715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e161d6-f887-456b-8c2d-e606771d5a41 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:42.601482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:42.602170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.602454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.602732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.603363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:42.603861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:42.605310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:42.606904 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 26/54] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:42.612359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:42.612975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b71b44-3e12-44f2-985c-d85da88744ee None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:42.654577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:42.655367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.656464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.656758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.658262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:42.659724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:42.661296 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 29/55] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:42.666795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:42.667228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:42.719090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985260c9-7f23-48a4-b640-e7926c4282e1 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:42.752393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d9d0b-a61d-4ecc-84f9-7df52960e14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:42.752998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d9d0b-a61d-4ecc-84f9-7df52960e14b None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.753280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d9d0b-a61d-4ecc-84f9-7df52960e14b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.753556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d9d0b-a61d-4ecc-84f9-7df52960e14b None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.754221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d9d0b-a61d-4ecc-84f9-7df52960e14b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:42.755661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d9d0b-a61d-4ecc-84f9-7df52960e14b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:42.767718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eaf92-fc3a-419b-b453-2385a1b2afec None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:42.782990 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 27/56] 10.222.0.33 () {60 vars in 1140 bytes} [Sat Sep 10 20:30:42 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:30:42.811376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80fd6e-97ed-40fb-a792-07de27219c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:42.812006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80fd6e-97ed-40fb-a792-07de27219c45 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:42.812292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80fd6e-97ed-40fb-a792-07de27219c45 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:42.812568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80fd6e-97ed-40fb-a792-07de27219c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:42.813389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80fd6e-97ed-40fb-a792-07de27219c45 None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.domain.id=default, user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:42.813707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80fd6e-97ed-40fb-a792-07de27219c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:42.815275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80fd6e-97ed-40fb-a792-07de27219c45 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:42.822210 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 30/57] 10.222.0.33 () {56 vars in 1322 bytes} [Sat Sep 10 20:30:42 2022] GET /identity/v3/role_assignments?user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6&scope.domain.id=default&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:45.216917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40c4a-e987-4dc0-af16-e724e217e1fe None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:45.217459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40c4a-e987-4dc0-af16-e724e217e1fe None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:45.217459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40c4a-e987-4dc0-af16-e724e217e1fe None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:45.218828 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 28/58] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:45.223915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89657df-2840-46f7-a605-a488fb1ce935 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:45.224301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89657df-2840-46f7-a605-a488fb1ce935 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:45.224544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89657df-2840-46f7-a605-a488fb1ce935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:45.242391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f89657df-2840-46f7-a605-a488fb1ce935 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:45.243594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f89657df-2840-46f7-a605-a488fb1ce935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:45.260817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:45.309480 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 31/59] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:45.639827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40c4a-e987-4dc0-af16-e724e217e1fe None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:45.735189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:45.739601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40c4a-e987-4dc0-af16-e724e217e1fe None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:45.774484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7334850-41d4-4d8e-941d-402f7c182808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:45.775095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7334850-41d4-4d8e-941d-402f7c182808 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:45.775377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7334850-41d4-4d8e-941d-402f7c182808 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:45.775654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7334850-41d4-4d8e-941d-402f7c182808 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:45.776746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7334850-41d4-4d8e-941d-402f7c182808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:45.778214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7334850-41d4-4d8e-941d-402f7c182808 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:45.779728 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 29/60] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:45.785103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89657df-2840-46f7-a605-a488fb1ce935 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:45.884484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89657df-2840-46f7-a605-a488fb1ce935 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:45.917074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea560f-18b7-45b4-868f-78188f1d690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:45.917608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea560f-18b7-45b4-868f-78188f1d690c None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:45.917921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea560f-18b7-45b4-868f-78188f1d690c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:45.918197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea560f-18b7-45b4-868f-78188f1d690c None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:45.918845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea560f-18b7-45b4-868f-78188f1d690c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:45.920278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea560f-18b7-45b4-868f-78188f1d690c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:45.946890 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 32/61] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:30:45 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:30:49.357986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a2901-a0a5-4fea-afd1-2562459a6f0e None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:49.358497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a2901-a0a5-4fea-afd1-2562459a6f0e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:49.358570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a2901-a0a5-4fea-afd1-2562459a6f0e None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:49.359981 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 30/62] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:49.367978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bdffe-30c0-43cf-b507-bab0d60bf986 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:49.368455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bdffe-30c0-43cf-b507-bab0d60bf986 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:49.368526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bdffe-30c0-43cf-b507-bab0d60bf986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:49.394827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2e1bdffe-30c0-43cf-b507-bab0d60bf986 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:49.396138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2e1bdffe-30c0-43cf-b507-bab0d60bf986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:49.399717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:49.487479 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 33/63] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:49.887899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a01d-0f0d-4d1e-91de-734984cd375a None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:49.888235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a01d-0f0d-4d1e-91de-734984cd375a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:49.888464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a01d-0f0d-4d1e-91de-734984cd375a None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:49.889858 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 31/64] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:49.895793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd45370-a064-43a1-ad08-00ac0b5d7a76 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:49.896095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd45370-a064-43a1-ad08-00ac0b5d7a76 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:49.896357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd45370-a064-43a1-ad08-00ac0b5d7a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:49.922821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8dd45370-a064-43a1-ad08-00ac0b5d7a76 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:49.924015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8dd45370-a064-43a1-ad08-00ac0b5d7a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:49.987841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79854a-c50a-41d8-9c38-369df4169444 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:49.988132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79854a-c50a-41d8-9c38-369df4169444 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:49.988411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79854a-c50a-41d8-9c38-369df4169444 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:49.990009 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 32/65] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:49.995002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdfb0c-f98f-45d8-b4de-f73aa81ce697 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:49.995536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdfb0c-f98f-45d8-b4de-f73aa81ce697 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:49.996017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdfb0c-f98f-45d8-b4de-f73aa81ce697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.023508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d3fdfb0c-f98f-45d8-b4de-f73aa81ce697 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:50.025752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3fdfb0c-f98f-45d8-b4de-f73aa81ce697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:50.032545 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 34/66] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:50.038079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd45370-a064-43a1-ad08-00ac0b5d7a76 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.102882 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 33/67] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:30:50.133011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843cf1e-048c-4e14-938d-b7bc949f6d2f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.133850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843cf1e-048c-4e14-938d-b7bc949f6d2f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.134402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843cf1e-048c-4e14-938d-b7bc949f6d2f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.136458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 34/68] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:30:50.141517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfef95f-42bc-4230-8d31-d919de148fb5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.142036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfef95f-42bc-4230-8d31-d919de148fb5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.142502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfef95f-42bc-4230-8d31-d919de148fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.161536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6bfef95f-42bc-4230-8d31-d919de148fb5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:50.163451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6bfef95f-42bc-4230-8d31-d919de148fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:50.182164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:50.232960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd45370-a064-43a1-ad08-00ac0b5d7a76 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:50.287781 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 35/69] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:30:50.300615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:50.301554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.302032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.302444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.316060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:50.321561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:50.322638 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:50.323601 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:50.331030 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 35/70] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:50.333942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b170ec7-e471-4b28-a3e2-d1671a29194a None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.334893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b170ec7-e471-4b28-a3e2-d1671a29194a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.335303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b170ec7-e471-4b28-a3e2-d1671a29194a None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.337044 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 36/71] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:50.338502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.342803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5dd80-c240-4dd4-92f7-682f4579c431 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.343311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5dd80-c240-4dd4-92f7-682f4579c431 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.343789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5dd80-c240-4dd4-92f7-682f4579c431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.367477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bcf5dd80-c240-4dd4-92f7-682f4579c431 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:50.369445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bcf5dd80-c240-4dd4-92f7-682f4579c431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:50.430292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:50.460007 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 37/72] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:30:50.466781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5dd80-c240-4dd4-92f7-682f4579c431 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.517128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decba43-4b2c-4a60-b030-584139da0992 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:50.564063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:50.564900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.565619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.570034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:50.570547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.571367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:50.571822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:50.573861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:50.586211 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 36/73] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:50.589971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5dd80-c240-4dd4-92f7-682f4579c431 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:50.591251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.632972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:50.633783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.634169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.634532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.649427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:50.655268 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:50.656138 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:50.656891 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:30:50.661483 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 38/74] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:50.662744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.683936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:50.709305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661545c0-c571-4fa1-afed-9a98bd55b9e6 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:50.744901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:50.745533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.745847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.746126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.756373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:50.761697 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:50.762671 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:50.763468 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 37/75] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:50.764084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.785818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de484a-b250-4c1a-849f-0153671c18b0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:50.816808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:50.825024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:50.825652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.825976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.826258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.841958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:50.843422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:50.845060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 39/76] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:50.845856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.866028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016288cf-2d4b-4eb0-a254-5880caeb94b3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:50.900258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:50.900865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.901150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.901427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.902128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:50.902435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:50.903837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:50.911699 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 38/77] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:50.913163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:50.921767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:50.946169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78c88d-f663-4b94-b114-aa3bb4c2326d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:50.982581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:50.983212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:50.983494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:50.983769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:50.994217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:50.999002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:50.999971 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:30:51.000720 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 40/78] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:51.001349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.021308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c0add-5fb3-4a7a-b78c-2928ff9740b9 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.035564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.056299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.056904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.057199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.057477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.058174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:51.058484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.059879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.067567 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 39/79] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:30:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:51.068353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.103692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50932b4-0c1e-442f-9fa3-9bd7b057d874 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.137736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.138404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.138686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.138966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.140083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.141536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.143105 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 41/80] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:51.143872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.149059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.172522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e412293-82f7-4e21-82cc-d7dd0c82fd63 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.206725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.207256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.207547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.207828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.246498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb8368-b96f-4930-91e7-f2be0c6b79f4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.253619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.276922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.278440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.279170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.279800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.280086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.280361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] PATH_INFO: `/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.281483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] RBAC: Authorizing `identity:get_user(user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.282930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.284499 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 42/81] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:51.285252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.354819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.379948 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 40/82] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:30:51.385500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.391115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24f334-c175-482c-9f6e-903664677f50 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.424923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.425529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.425841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.426130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.426796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:51.427104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.429048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.437115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 43/83] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:51.438750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.462150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.485278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9be519-cdad-4969-a2c3-35e1d53b1191 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.519853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.520480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.520753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.521024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.523048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.524425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.526026 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 41/84] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:51.526026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.547134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc5cc-58ba-4729-977f-8d786f0e0b83 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.581792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.582507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.582814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.583077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.583773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.585529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.596265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.612287 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 44/85] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:30:51 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:30:51.613092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.637144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfb3a6-a43b-4b2e-b873-b8a32fc64a94 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.679920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.680555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.680839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.681110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.698137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.728755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d6907-5187-433e-8f7a-c195523dbd80 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.758242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.759765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.769194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.769867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.770152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.770429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.771594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.773495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.780492 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 45/86] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:51.780492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.849760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.876319 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 42/87] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:30:51.877465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:51.898904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e7185-572d-40fb-8238-76570acf1911 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:51.931249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:51.940859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:51.941771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:51.942091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:51.942366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:51.945143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:51.946829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:51.949067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 46/88] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:51.949967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:52.001222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e6e8b-044d-4a5c-9803-f0f44ab27e20 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:52.045674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:52.046334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:52.046623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:52.046904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:52.076055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76070b84-6027-4a57-96e6-a8fb7f7a2c46 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:52.112602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:52.113248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:52.113538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:52.113909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:52.114647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=8116bc11609448a7a4b58a423a52d68d) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:52.115017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:52.116794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:52.125702 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 47/89] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/role_assignments?user.id=8116bc11609448a7a4b58a423a52d68d&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:52.130830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:52.132361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:52.134623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:52.136307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:52.223512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:52.284226 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 43/90] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:30:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:30:52.285524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:52.305857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6d575-48ac-4048-99f9-3b450ddc8bb0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:52.340457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:52.341121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:52.341406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:52.341686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:52.343001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:52.345053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:52.347081 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 48/91] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:52 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:52.347899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:52.459939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:52.500479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a535f-64b0-4e19-9427-2619555fbffc None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:52.529523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb47447-2b10-4383-90a7-82d2fcb2d3d1 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:52.571604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:52.572940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:52.573424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:52.573923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:52.609283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:52.610581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:52.612520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:52.613053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:52.614263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:52.617066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:52.620217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:52.626622 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 49/92] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:52 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:52.627743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:52.700737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:52.702732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:52.776611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2b595-5109-4f9e-84fc-2d7d9868ca2c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:52.805019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:52.815124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:52.815815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:52.816170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:52.816509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:52.817309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:52.817695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:52.819766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:52.826173 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 50/93] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:30:52 2022] GET /identity/v3/role_assignments?user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:52.826988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:52.903430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 44/94] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:30:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:30:52.909557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:52.946768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d933cf9-1a8e-4502-96df-bdcc3ecadc4d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:52.984919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:52.985548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:52.985863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:52.986207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:52.987864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] RBAC: Adding query filter params (role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=8116bc11609448a7a4b58a423a52d68d) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:52.987864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:52.988904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:52.995377 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 51/95] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:30:52 2022] GET /identity/v3/role_assignments?user.id=8116bc11609448a7a4b58a423a52d68d&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:53.012392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:53.101609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f7ccf-3dfb-4b44-ae43-400dec189682 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:53.149070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa7512-4228-4964-9380-31ff55cea28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:53.149730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa7512-4228-4964-9380-31ff55cea28a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:53.150075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa7512-4228-4964-9380-31ff55cea28a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:53.150332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa7512-4228-4964-9380-31ff55cea28a None admin] PATH_INFO: `/v3/projects/840a7d21509a4db5ae8a92a957703730` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:53.151525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa7512-4228-4964-9380-31ff55cea28a None admin] RBAC: Authorizing `identity:get_project(project_id=840a7d21509a4db5ae8a92a957703730)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:53.153028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa7512-4228-4964-9380-31ff55cea28a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:53.154754 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 45/96] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:52 2022] GET /identity/v3/projects/840a7d21509a4db5ae8a92a957703730 => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:53.160826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:53.241183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:53.288014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25df4c-886c-44a3-8aef-ba76b8e5845c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:53.323453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2ebf1-9d12-4c92-9de5-5bea7dc2ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:53.324075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ebf1-9d12-4c92-9de5-5bea7dc2ca6d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:53.324361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ebf1-9d12-4c92-9de5-5bea7dc2ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:53.324641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ebf1-9d12-4c92-9de5-5bea7dc2ca6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:53.325374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2ebf1-9d12-4c92-9de5-5bea7dc2ca6d None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=840a7d21509a4db5ae8a92a957703730, user.id=8116bc11609448a7a4b58a423a52d68d) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:53.325688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2ebf1-9d12-4c92-9de5-5bea7dc2ca6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:53.327181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2ebf1-9d12-4c92-9de5-5bea7dc2ca6d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:53.332968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 52/97] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:30:53 2022] GET /identity/v3/role_assignments?user.id=8116bc11609448a7a4b58a423a52d68d&scope.project.id=840a7d21509a4db5ae8a92a957703730&role.id=9d599f0800724bd4b78e84df540784c8 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:54.925502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bf9c2-9bdc-41c1-b186-3c0b001b97ce None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:54.926048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bf9c2-9bdc-41c1-b186-3c0b001b97ce None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:54.926048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bf9c2-9bdc-41c1-b186-3c0b001b97ce None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:54.927644 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 46/98] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:54.933117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69526c62-5dd2-4456-8b8c-9636e1077c2a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:54.933531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69526c62-5dd2-4456-8b8c-9636e1077c2a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:54.933712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69526c62-5dd2-4456-8b8c-9636e1077c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:54.958828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-69526c62-5dd2-4456-8b8c-9636e1077c2a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:54.960364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-69526c62-5dd2-4456-8b8c-9636e1077c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:55.055144 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 53/99] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:30:55.180752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05b01-03a8-4b51-b128-17e4b0886636 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:55.180936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05b01-03a8-4b51-b128-17e4b0886636 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:55.181224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05b01-03a8-4b51-b128-17e4b0886636 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:55.182604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 47/100] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:55.187841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15497ab3-bb21-4547-a6dc-d1eb379bf389 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:55.188137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15497ab3-bb21-4547-a6dc-d1eb379bf389 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:55.188420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15497ab3-bb21-4547-a6dc-d1eb379bf389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:55.210563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-15497ab3-bb21-4547-a6dc-d1eb379bf389 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:55.211758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-15497ab3-bb21-4547-a6dc-d1eb379bf389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:55.269244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:55.303509 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 54/101] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:55.604652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d05b01-03a8-4b51-b128-17e4b0886636 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:55.721802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d05b01-03a8-4b51-b128-17e4b0886636 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:55.737198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:55.767236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:55.767236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:55.767539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:55.767774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:55.780330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:55.785665 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:55.786750 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:55.788089 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 48/102] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:55.793953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15497ab3-bb21-4547-a6dc-d1eb379bf389 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:55.919239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15497ab3-bb21-4547-a6dc-d1eb379bf389 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:55.981395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:55.982061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:55.982345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:55.982617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:55.983291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] RBAC: Adding query filter params (name=reader) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:55.983596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:55.985064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:55.995331 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 55/103] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:30:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:56.001037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:56.020344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:56.092850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:56.137847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37bd7-6a28-4bfa-b362-5c02c2e4aa74 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:56.196509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:56.197492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.198202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.198690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] PATH_INFO: `/v3/users/33f7b6846839476eb4a00d41a07519f6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.210556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dbe9a-d08e-4122-9a2d-f90407039bbd None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:56.215988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] RBAC: Authorizing `identity:get_user(user_id=33f7b6846839476eb4a00d41a07519f6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:56.217638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:56.224806 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 49/104] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:55 2022] GET /identity/v3/users/33f7b6846839476eb4a00d41a07519f6 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:56.227284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:56.264401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:56.280044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:56.280701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.281094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.281374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.286745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:56.288909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:56.290984 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 56/105] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:30:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:30:56.298011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:56.406023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f521b46-4f13-4b9e-b68f-8eaf6b886b80 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:56.433528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269727a0-9522-4e24-9328-3affbf5b7794 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:56.446106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:56.454941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69333569-0b9f-43ca-9532-02d77f07a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:56.455609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333569-0b9f-43ca-9532-02d77f07a32f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.456414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333569-0b9f-43ca-9532-02d77f07a32f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.456883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333569-0b9f-43ca-9532-02d77f07a32f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.483399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f853431-b1b4-4619-be05-749db89b5d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:56.484337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853431-b1b4-4619-be05-749db89b5d7e None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.484799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853431-b1b4-4619-be05-749db89b5d7e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.485248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853431-b1b4-4619-be05-749db89b5d7e None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.486168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f853431-b1b4-4619-be05-749db89b5d7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:56.487851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f853431-b1b4-4619-be05-749db89b5d7e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:56.517771 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 57/106] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:30:56 2022] POST /identity/v3/projects => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:30:56.519693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06090c-31ae-4611-b5f7-901852897b92 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.520480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06090c-31ae-4611-b5f7-901852897b92 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.521136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06090c-31ae-4611-b5f7-901852897b92 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.523220 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 58/107] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:56.524980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb073c-082a-48eb-82d1-7f9232d14a78 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.525666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb073c-082a-48eb-82d1-7f9232d14a78 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.526181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb073c-082a-48eb-82d1-7f9232d14a78 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.527908 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 59/108] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:56.530334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a539e8-e7b9-4da4-96fc-c0b74d3bc733 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.531217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a539e8-e7b9-4da4-96fc-c0b74d3bc733 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.531818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a539e8-e7b9-4da4-96fc-c0b74d3bc733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.557137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69333569-0b9f-43ca-9532-02d77f07a32f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:56.559018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69333569-0b9f-43ca-9532-02d77f07a32f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:56.559702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b4a539e8-e7b9-4da4-96fc-c0b74d3bc733 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:56.561097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b4a539e8-e7b9-4da4-96fc-c0b74d3bc733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:56.595385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:56.641170 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 60/109] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:56.643113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3861c-511e-4257-920e-810944854125 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.643636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3861c-511e-4257-920e-810944854125 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.644116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3861c-511e-4257-920e-810944854125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.665736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-abb3861c-511e-4257-920e-810944854125 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:56.667789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-abb3861c-511e-4257-920e-810944854125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:56.693988 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 50/110] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:30:56.693988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f4ea5-7a63-4ed6-8004-ddc924560a11 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.693988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f4ea5-7a63-4ed6-8004-ddc924560a11 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.693988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f4ea5-7a63-4ed6-8004-ddc924560a11 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.698743 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 51/111] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:56.704032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed625d30-40ff-4d10-9885-3d4fc3782b60 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.704350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed625d30-40ff-4d10-9885-3d4fc3782b60 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.704602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed625d30-40ff-4d10-9885-3d4fc3782b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:56.748690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ed625d30-40ff-4d10-9885-3d4fc3782b60 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:56.750704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed625d30-40ff-4d10-9885-3d4fc3782b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:56.808888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 61/112] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:56.810413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3861c-511e-4257-920e-810944854125 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:56.854194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:56.857575 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 52/113] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:30:56.940009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3861c-511e-4257-920e-810944854125 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:56.996283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:56.996904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:56.997188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:56.997465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.002418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:57.003870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:57.005659 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 62/114] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:56 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:57.012836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed625d30-40ff-4d10-9885-3d4fc3782b60 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.079353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:57.131799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed625d30-40ff-4d10-9885-3d4fc3782b60 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:57.162067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.183886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74cc90-79f5-49f6-8ee4-a7306e51e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:57.184514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74cc90-79f5-49f6-8ee4-a7306e51e1bf None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.184795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74cc90-79f5-49f6-8ee4-a7306e51e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.185148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74cc90-79f5-49f6-8ee4-a7306e51e1bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.185912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74cc90-79f5-49f6-8ee4-a7306e51e1bf None admin] RBAC: Adding query filter params (role.id=887708bc8dee457481db8046c4f52ea8, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=33f7b6846839476eb4a00d41a07519f6) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:57.186225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74cc90-79f5-49f6-8ee4-a7306e51e1bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:57.187936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74cc90-79f5-49f6-8ee4-a7306e51e1bf None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:57.195772 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 53/115] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/role_assignments?user.id=33f7b6846839476eb4a00d41a07519f6&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=887708bc8dee457481db8046c4f52ea8 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:57.229615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ee0bd-18d8-4608-a1f1-8cff130b5756 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.229954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ee0bd-18d8-4608-a1f1-8cff130b5756 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.230218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ee0bd-18d8-4608-a1f1-8cff130b5756 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.231617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 54/116] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:30:57.237312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e012df-fddb-4835-b340-b256acf32a20 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.237638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e012df-fddb-4835-b340-b256acf32a20 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.237907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e012df-fddb-4835-b340-b256acf32a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.265603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-98e012df-fddb-4835-b340-b256acf32a20 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:57.266773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98e012df-fddb-4835-b340-b256acf32a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:57.356871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:57.375286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49d63f-8165-470f-9ebb-34cd590d72da None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:57.395662 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 55/117] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:30:57.396892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e012df-fddb-4835-b340-b256acf32a20 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.436597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:57.437762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.439500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.440181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.466657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:57.478383 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:57.479318 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:57.480087 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:30:57.482165 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 63/118] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:30:57.483251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.508925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:57.598356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e012df-fddb-4835-b340-b256acf32a20 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:57.619116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7863-05b7-4bef-8f9f-8fdb2e307b8b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:57.664596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:57.665621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.666127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.666624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.678544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:57.683889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:57.684524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.684806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:57.685027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:57.685890 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:30:57.686690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:57.687273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.689241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 56/119] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:57.690356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.706976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:57.713374 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:57.714294 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:30:57.715116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:30:57.726831 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 64/120] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:57.727700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.743981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:57.805418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916520df-027d-45c6-a6d0-2af9e6fe2f2a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:57.821361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:57.841251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6e5bf-304c-4a8b-ab41-518a0895b55e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:57.845414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:57.846312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.846798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.847303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.848192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] RBAC: Adding query filter params (name=member) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:57.848768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:57.850731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:57.860030 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 57/121] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:57.861109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.877432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:57.878076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:57.878370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:57.878635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:57.879295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:57.879598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:57.881039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:57.890205 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 65/122] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:57.891001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:57.978416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b6eb8-0203-4df6-95fd-175331ad525e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.004829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.016408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.016990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.017261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.017537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.029218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:30:58.034810 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:30:58.035704 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:30:58.036517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 58/123] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:30:58.037752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.051686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30582139-54fe-4a13-8b71-0f04bbf383d7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.088474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.098310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.100384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.101046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.101804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.102723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:58.103566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.106058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.118980 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 66/124] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:30:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.120193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.176734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40389c-633b-449d-8f0a-96d136c00135 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.219742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.220462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.220814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.221283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.222673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.226335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.228074 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 59/125] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.228909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.265301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8db5e3-0d29-4196-be2c-3b9e5cc8ad62 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.306073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.313570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.315064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.315536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.316259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] PATH_INFO: `/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.317973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] RBAC: Authorizing `identity:get_user(user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.320825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.323081 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 67/126] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.324721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.363531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de4ec5-2611-466a-a792-2eeb6dd7748c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.366984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.405793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.406799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.407285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.407816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.408686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] RBAC: Adding query filter params (name=member) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:30:58.409219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.411074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.420660 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 60/127] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.421780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.441293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f131c-151b-402a-b782-9d18eab37e7a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.478140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.478865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.479153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.479432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.480572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.482250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.483791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 68/128] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.484374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.540259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe494a0-a4a4-4939-a5d2-8c407219e5ad None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.569480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.579545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.580498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.581008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.582027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.583983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.585772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.588364 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 61/129] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.589511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.611633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594295b-3f4b-42ff-be39-9132eaed6d26 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.630155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.653312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.653980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.654278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.654985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.656402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.657896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.659659 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 69/130] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.660415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.731765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d0ec2-4b78-44e5-a92d-b4f0db5e718a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.777688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.778372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.778653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.778927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.780191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.781619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.783217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 62/131] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.784005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.789853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff97bb-f646-4f8b-954b-8d0dcb689c27 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.837246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.838867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.839362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.840245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.841675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.843903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.846234 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 70/132] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:58.847137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.851741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.900907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9433d14-7d06-4d17-a294-ab75b6bccfd4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:58.902507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:58.938740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:58.939509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:58.939859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:58.940203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/8116bc11609448a7a4b58a423a52d68d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:58.942259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] RBAC: Authorizing `identity:create_grant(project_id=62f33ff370254aa8a6e2b693a30a9b3c, user_id=8116bc11609448a7a4b58a423a52d68d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:58.944576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:58.961338 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 63/133] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:30:58 2022] PUT /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/8116bc11609448a7a4b58a423a52d68d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:30:58.962211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:58.980509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d989d66-0f43-488f-a87f-3731616b0166 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:59.082713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:59.083419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:59.083682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:59.084472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/cd34ffbc794a4c04a8a5c4f195b6d3b6/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:59.088134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] RBAC: Authorizing `identity:create_grant(project_id=62f33ff370254aa8a6e2b693a30a9b3c, user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:59.089610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:59.117473 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 71/134] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:30:58 2022] PUT /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/cd34ffbc794a4c04a8a5c4f195b6d3b6/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:30:59.118609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:59.138979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d892-814f-417f-b210-b4c39ed13de1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:59.184480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:59.185106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:59.185343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:59.185631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] PATH_INFO: `/v3/projects/840a7d21509a4db5ae8a92a957703730` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:59.187796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] RBAC: Authorizing `identity:get_project(project_id=840a7d21509a4db5ae8a92a957703730)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:59.189641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:59.191705 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 64/135] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:30:58 2022] GET /identity/v3/projects/840a7d21509a4db5ae8a92a957703730 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:30:59.198441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:30:59.249527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:59.260157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:30:59.349300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4f2cc-45be-4c85-8e68-02a7796cf3ce None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:59.387744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789a3cf-c18d-427d-bea6-26f06228586e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:30:59.427242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670e020-c288-4131-bfa3-f4987708836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:59.428068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670e020-c288-4131-bfa3-f4987708836d None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:59.428574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670e020-c288-4131-bfa3-f4987708836d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:59.429065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670e020-c288-4131-bfa3-f4987708836d None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/8116bc11609448a7a4b58a423a52d68d/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:59.436097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670e020-c288-4131-bfa3-f4987708836d None admin] RBAC: Authorizing `identity:create_grant(project_id=62f33ff370254aa8a6e2b693a30a9b3c, user_id=8116bc11609448a7a4b58a423a52d68d, role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:59.436602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf5372-db5b-4dca-b0c1-def102f98073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:30:59.437279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5372-db5b-4dca-b0c1-def102f98073 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:59.437568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5372-db5b-4dca-b0c1-def102f98073 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:59.437902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5372-db5b-4dca-b0c1-def102f98073 None admin] PATH_INFO: `/v3/projects/840a7d21509a4db5ae8a92a957703730/users/8116bc11609448a7a4b58a423a52d68d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:59.438795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670e020-c288-4131-bfa3-f4987708836d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:59.439855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf5372-db5b-4dca-b0c1-def102f98073 None admin] RBAC: Authorizing `identity:create_grant(project_id=840a7d21509a4db5ae8a92a957703730, user_id=8116bc11609448a7a4b58a423a52d68d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:30:59.441487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf5372-db5b-4dca-b0c1-def102f98073 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:30:59.468656 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 72/136] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:30:59 2022] PUT /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/8116bc11609448a7a4b58a423a52d68d/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:30:59.472113 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 65/137] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:30:59 2022] PUT /identity/v3/projects/840a7d21509a4db5ae8a92a957703730/users/8116bc11609448a7a4b58a423a52d68d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:30:59.700931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb534-2d18-4098-8dbd-1e2145d12fb4 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:59.701268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb534-2d18-4098-8dbd-1e2145d12fb4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:59.701452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb534-2d18-4098-8dbd-1e2145d12fb4 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:59.702930 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 73/138] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:30:59.721043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7bf0-783e-4367-a779-72064b0052c3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:30:59.721043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7bf0-783e-4367-a779-72064b0052c3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:30:59.721043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c7bf0-783e-4367-a779-72064b0052c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:30:59.747638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8b7c7bf0-783e-4367-a779-72064b0052c3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:30:59.748870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8b7c7bf0-783e-4367-a779-72064b0052c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:30:59.827753 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 66/139] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:00.294572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb534-2d18-4098-8dbd-1e2145d12fb4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:00.358992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ec24-912b-43e0-8784-890fa90df6c4 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:00.359532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ec24-912b-43e0-8784-890fa90df6c4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:00.359532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ec24-912b-43e0-8784-890fa90df6c4 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:00.361121 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 67/140] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:00.367727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf7e2-a886-420a-9243-2bc8eb18ba72 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:00.368008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf7e2-a886-420a-9243-2bc8eb18ba72 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:00.368285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf7e2-a886-420a-9243-2bc8eb18ba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:00.387010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b5bdf7e2-a886-420a-9243-2bc8eb18ba72 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:00.388212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b5bdf7e2-a886-420a-9243-2bc8eb18ba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:00.413655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:00.429175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb534-2d18-4098-8dbd-1e2145d12fb4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:00.481332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:00.500429 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 68/141] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:00.501732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:00.502405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:00.502690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:00.502965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:00.504137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:00.505666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:00.507338 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 74/142] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:00.512845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdf7e2-a886-420a-9243-2bc8eb18ba72 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:00.636666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdf7e2-a886-420a-9243-2bc8eb18ba72 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:00.673312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:00.673963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:00.674334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:00.674618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:00.675285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:00.676805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:00.701757 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 69/143] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:00 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:31:01.137476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:01.252889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d813f8-86b1-4bc8-bfc6-1cc9fbfbfaf1 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:01.288839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:01.289490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:01.289773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:01.290167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:01.301766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:01.308905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:01.309736 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:01.311304 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 75/144] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:01.316031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:01.326193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:01.454762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e010fbf-31c9-46ce-940a-6b88d4849108 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:01.499967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:01.500593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:01.500882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:01.501161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:01.501895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:01.502212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:01.503749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:01.514643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 70/145] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:01.521465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:01.590655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:01.641063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08c761-1bb6-4dc8-bec6-47213f041f66 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:01.676512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:01.677159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:01.677442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:01.677818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] PATH_INFO: `/v3/users/33f7b6846839476eb4a00d41a07519f6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:01.679100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] RBAC: Authorizing `identity:get_user(user_id=33f7b6846839476eb4a00d41a07519f6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:01.680700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:01.682324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 76/146] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:01 2022] GET /identity/v3/users/33f7b6846839476eb4a00d41a07519f6 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:01.688134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:01.760324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:01.799712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaeda8-fec5-46e6-95a0-57dc6621c345 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:01.839764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f12eac-049e-4f6f-a1de-c3658f970e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:01.840410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f12eac-049e-4f6f-a1de-c3658f970e07 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:01.840704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f12eac-049e-4f6f-a1de-c3658f970e07 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:01.841001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f12eac-049e-4f6f-a1de-c3658f970e07 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:01.842268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f12eac-049e-4f6f-a1de-c3658f970e07 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:01.843790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f12eac-049e-4f6f-a1de-c3658f970e07 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:01.845521 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 71/147] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:01 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:01.851553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:01.958463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1cd30-e443-4a9a-bb43-4ff024013e3b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:01.982209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:01.995674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d5a5a-d928-4cd5-932a-069a6cf9d19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:01.996354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5a5a-d928-4cd5-932a-069a6cf9d19a None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:01.996642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5a5a-d928-4cd5-932a-069a6cf9d19a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:01.996929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5a5a-d928-4cd5-932a-069a6cf9d19a None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/33f7b6846839476eb4a00d41a07519f6/roles/887708bc8dee457481db8046c4f52ea8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:01.998891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d5a5a-d928-4cd5-932a-069a6cf9d19a None admin] RBAC: Authorizing `identity:create_grant(project_id=62f33ff370254aa8a6e2b693a30a9b3c, user_id=33f7b6846839476eb4a00d41a07519f6, role_id=887708bc8dee457481db8046c4f52ea8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:02.000433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d5a5a-d928-4cd5-932a-069a6cf9d19a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:02.028135 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 77/148] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:31:01 2022] PUT /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/users/33f7b6846839476eb4a00d41a07519f6/roles/887708bc8dee457481db8046c4f52ea8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:03.020912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64284fd9-c585-4964-8a6e-e02b10d7070d None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.021937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64284fd9-c585-4964-8a6e-e02b10d7070d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.022359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64284fd9-c585-4964-8a6e-e02b10d7070d None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.023885 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 72/149] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:03.030131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d5fd2-0d79-429d-904b-83af4b9e5c50 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.031018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d5fd2-0d79-429d-904b-83af4b9e5c50 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.031527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d5fd2-0d79-429d-904b-83af4b9e5c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.057491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7a5d5fd2-0d79-429d-904b-83af4b9e5c50 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:03.059039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a5d5fd2-0d79-429d-904b-83af4b9e5c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:03.150273 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 78/150] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:03.208735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d38e8-53be-475e-959f-6d454e3805ee None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.209473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d38e8-53be-475e-959f-6d454e3805ee None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.209980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d38e8-53be-475e-959f-6d454e3805ee None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.211509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 73/151] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:03.226093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23721326-f306-4d04-aa68-5dc317a1dc61 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.226346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23721326-f306-4d04-aa68-5dc317a1dc61 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.226587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23721326-f306-4d04-aa68-5dc317a1dc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.255059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-23721326-f306-4d04-aa68-5dc317a1dc61 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:03.256359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-23721326-f306-4d04-aa68-5dc317a1dc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:03.345122 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 79/152] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:03.472974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707075-90ec-4ad1-8de8-1a79a09d6d40 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.473599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707075-90ec-4ad1-8de8-1a79a09d6d40 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.474114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707075-90ec-4ad1-8de8-1a79a09d6d40 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.475905 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 74/153] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:03.481426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cde42-8642-452b-b417-2584c9ae9a1b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.481685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cde42-8642-452b-b417-2584c9ae9a1b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.481956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cde42-8642-452b-b417-2584c9ae9a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.501900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-982cde42-8642-452b-b417-2584c9ae9a1b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:03.503403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-982cde42-8642-452b-b417-2584c9ae9a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:03.548804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:03.593494 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 80/154] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:03.794609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39289227-2f25-48ca-bd34-49dd213605f6 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.794938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39289227-2f25-48ca-bd34-49dd213605f6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.795131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39289227-2f25-48ca-bd34-49dd213605f6 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.796522 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 75/155] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:03.815719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de3054-a902-4966-865f-35b5d35726ae None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:03.816026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de3054-a902-4966-865f-35b5d35726ae None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:03.816281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de3054-a902-4966-865f-35b5d35726ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:03.838568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b0de3054-a902-4966-865f-35b5d35726ae None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:03.839821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b0de3054-a902-4966-865f-35b5d35726ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:03.893236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39289227-2f25-48ca-bd34-49dd213605f6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:03.928671 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 81/156] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:04.024982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de3054-a902-4966-865f-35b5d35726ae None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:04.069905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39289227-2f25-48ca-bd34-49dd213605f6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:04.119462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:04.120164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:04.120497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:04.120778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:04.123516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:04.137086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:04.142138 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:04.142992 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:04.143771 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 76/157] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:31:04.149966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:04.196739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de3054-a902-4966-865f-35b5d35726ae None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:04.199887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:04.267499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:04.268374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:04.268790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:04.269219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:04.292280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:04.302368 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:04.304570 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:04.305440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:04.308240 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 82/158] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:31:04.314791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:04.357250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0dcdc-3cb5-49ab-a10a-7a5247608f79 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:04.426733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:04.427604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:04.427891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:04.428132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:04.429472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] RBAC: Adding query filter params (name=member) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:04.429665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:04.435421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:04.452318 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 77/159] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:04.453192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:04.478281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc2dd4-9119-4e36-8f11-2d88b4887439 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:04.571369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:04.574592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:04.575243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:04.575526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:04.575847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:04.576533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:04.576836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:04.578414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:04.589316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 83/160] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:04.590267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:04.640534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:04.650656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab85c0-45ae-42b9-a90f-48507f4aa22a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:04.706672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:04.707369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:04.707653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:04.707981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:04.723214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:04.734178 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:04.735128 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:04.735906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 78/161] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:04.736535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:04.852998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99628eb-e81a-41c7-8b61-76f9903089ed None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:04.909868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077766f1-03e4-4673-9f56-29bb76f49e5f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:04.931213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:04.931862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:04.932141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:04.932413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:04.936722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:04.938222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:04.939860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 84/162] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:04 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:04.940707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:04.952454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:05.006661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:05.007413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:05.007703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:05.007987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:05.009298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:05.010977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:05.012643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 79/163] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:04 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:05.013235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:05.089139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:05.110517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebee2-83a0-41ae-8e57-d308dc4d7e3c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:05.165790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:05.166399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:05.166677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:05.166948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:05.187730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:05.193122 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:05.194967 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:05.195753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 85/164] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:05.196410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:05.229862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc47171-3104-4813-ab36-757b73e0e959 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:05.310987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:05.311960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:05.312443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:05.312955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:05.313844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:05.314367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:05.315954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:05.332994 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 80/165] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:05.334466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:05.363088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925e221-dd08-4b28-ba45-df347470a06f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:05.417944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:05.417944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:05.417944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:05.417944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:05.435338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:05.451480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:05.528922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:05.537953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:05.549928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd90cbd-1323-4fc4-8dc2-86665bf9f123 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:05.639507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:05.640157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:05.640441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:05.640719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:05.745673 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 86/166] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:05.747580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:05.784086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:05.786173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:05.824466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:05.891819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:05.921647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc36c15-2a61-466a-93c2-4417b4986a8b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:05.982139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f749a7-9743-4e2c-8d7d-6e956203c450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:05.982763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f749a7-9743-4e2c-8d7d-6e956203c450 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:05.983047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f749a7-9743-4e2c-8d7d-6e956203c450 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:05.983323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f749a7-9743-4e2c-8d7d-6e956203c450 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:05.983991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f749a7-9743-4e2c-8d7d-6e956203c450 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:05.984298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f749a7-9743-4e2c-8d7d-6e956203c450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:05.985673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f749a7-9743-4e2c-8d7d-6e956203c450 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:05.999540 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 81/167] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:06.000321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.005071 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 87/168] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:06.007169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923275f-efc3-4668-a12b-e5e3f5758381 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.007452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923275f-efc3-4668-a12b-e5e3f5758381 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.007728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923275f-efc3-4668-a12b-e5e3f5758381 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.008940 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 88/169] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:06.009635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923275f-efc3-4668-a12b-e5e3f5758381 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.165858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a2b7-eddf-473d-b5b6-3b8fdaec048d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:06.237563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:06.238233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.238488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.238768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] PATH_INFO: `/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.240579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] RBAC: Authorizing `identity:get_user(user_id=cd34ffbc794a4c04a8a5c4f195b6d3b6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:06.242075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:06.243964 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 82/170] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:05 2022] GET /identity/v3/users/cd34ffbc794a4c04a8a5c4f195b6d3b6 => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:06.245041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.247354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923275f-efc3-4668-a12b-e5e3f5758381 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:06.269609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:06.274500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:06.319980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:06.320624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.320910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.321190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.325157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:06.327606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:06.329285 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 89/171] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:06 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:06.330096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.394701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b55091-12fe-4c65-b735-ecf627073747 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:06.432961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3d102-82ff-4702-896a-b45b65f7f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:06.433927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3d102-82ff-4702-896a-b45b65f7f59b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.434431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3d102-82ff-4702-896a-b45b65f7f59b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.434844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3d102-82ff-4702-896a-b45b65f7f59b None admin] PATH_INFO: `/v3/projects/840a7d21509a4db5ae8a92a957703730` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.436142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3d102-82ff-4702-896a-b45b65f7f59b None admin] RBAC: Authorizing `identity:get_project(project_id=840a7d21509a4db5ae8a92a957703730)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:06.438056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3d102-82ff-4702-896a-b45b65f7f59b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:06.439960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 83/172] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:06 2022] GET /identity/v3/projects/840a7d21509a4db5ae8a92a957703730 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:06.446402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efd70e-66a2-4183-a273-3e4bd51159aa None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.446960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efd70e-66a2-4183-a273-3e4bd51159aa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.447398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efd70e-66a2-4183-a273-3e4bd51159aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.452776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca750ecf-8153-490a-8cb4-11a1cadaa20d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:06.469173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-24efd70e-66a2-4183-a273-3e4bd51159aa None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:06.470820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24efd70e-66a2-4183-a273-3e4bd51159aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:06.498419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:06.499620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.499914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.500193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] PATH_INFO: `/v3/users/8116bc11609448a7a4b58a423a52d68d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.501630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] RBAC: Authorizing `identity:get_user(user_id=8116bc11609448a7a4b58a423a52d68d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:06.503510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:06.505062 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 90/173] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:06 2022] GET /identity/v3/users/8116bc11609448a7a4b58a423a52d68d => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:06.509968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.557534 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 84/174] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:06.558783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efd70e-66a2-4183-a273-3e4bd51159aa None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.575659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:06.580175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:06.638007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c4c1a-cba8-4fe7-b779-2b0dff031ad1 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:06.670166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efd70e-66a2-4183-a273-3e4bd51159aa None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:06.674219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:06.674826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.675106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.675379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.705809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c3336-d8fa-4ea3-a23c-9cd3543ea572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:06.706446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3336-d8fa-4ea3-a23c-9cd3543ea572 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.706727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3336-d8fa-4ea3-a23c-9cd3543ea572 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.706998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3336-d8fa-4ea3-a23c-9cd3543ea572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.707711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3336-d8fa-4ea3-a23c-9cd3543ea572 None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=8116bc11609448a7a4b58a423a52d68d) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:06.708018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3336-d8fa-4ea3-a23c-9cd3543ea572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:06.709611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3336-d8fa-4ea3-a23c-9cd3543ea572 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:06.715733 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 85/175] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:06 2022] GET /identity/v3/role_assignments?user.id=8116bc11609448a7a4b58a423a52d68d&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:06.716886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d87b07-e3a5-45c6-9558-448c7f1ca68a None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.717169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d87b07-e3a5-45c6-9558-448c7f1ca68a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.717496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d87b07-e3a5-45c6-9558-448c7f1ca68a None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.719062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 86/176] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:06.719615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d87b07-e3a5-45c6-9558-448c7f1ca68a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.762921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:06.764571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:06.830528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:06.865576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:06.899673 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 91/177] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:06.900739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:06.901329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d87b07-e3a5-45c6-9558-448c7f1ca68a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:06.938926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dceb5-2540-44e8-addf-106b0435d60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:06.939600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dceb5-2540-44e8-addf-106b0435d60c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.939882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dceb5-2540-44e8-addf-106b0435d60c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.940155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dceb5-2540-44e8-addf-106b0435d60c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.940887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dceb5-2540-44e8-addf-106b0435d60c None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=840a7d21509a4db5ae8a92a957703730, user.id=8116bc11609448a7a4b58a423a52d68d) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:06.941193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dceb5-2540-44e8-addf-106b0435d60c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:06.942708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dceb5-2540-44e8-addf-106b0435d60c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:06.951309 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 87/178] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:06 2022] GET /identity/v3/role_assignments?user.id=8116bc11609448a7a4b58a423a52d68d&scope.project.id=840a7d21509a4db5ae8a92a957703730&role.id=9d599f0800724bd4b78e84df540784c8 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:06.951309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0be49-12ba-4133-b021-16616a8177ef None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.951569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0be49-12ba-4133-b021-16616a8177ef None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.951850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0be49-12ba-4133-b021-16616a8177ef None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.953116 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 88/179] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:06.954015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a964ff-d244-4180-a3f5-d7729b41b2e2 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.954291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a964ff-d244-4180-a3f5-d7729b41b2e2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.954585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a964ff-d244-4180-a3f5-d7729b41b2e2 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.955953 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 89/180] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:06.964746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39acb3-f09f-45d3-a7fc-3dcb3862b6c2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:06.965133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39acb3-f09f-45d3-a7fc-3dcb3862b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:06.965408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39acb3-f09f-45d3-a7fc-3dcb3862b6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:06.993556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5f39acb3-f09f-45d3-a7fc-3dcb3862b6c2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:06.993556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f39acb3-f09f-45d3-a7fc-3dcb3862b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:07.040241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f81bfb-e3c6-48bd-a687-5b75c32988bf None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.080674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:07.081212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.081495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.081770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.085916 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 90/181] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:07.089865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39acb3-f09f-45d3-a7fc-3dcb3862b6c2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.103592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:07.171320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:07.173501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:07.203003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:07.243487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39acb3-f09f-45d3-a7fc-3dcb3862b6c2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.289187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0e73d-dc57-4b17-8a67-ed677fe5fe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:07.290759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e73d-dc57-4b17-8a67-ed677fe5fe7e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.290759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e73d-dc57-4b17-8a67-ed677fe5fe7e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.290759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e73d-dc57-4b17-8a67-ed677fe5fe7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.301244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0e73d-dc57-4b17-8a67-ed677fe5fe7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5bc0e73d-dc57-4b17-8a67-ed677fe5fe7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:07.306818 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:07.307710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:07.313476 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:07.314143 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 92/182] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:07.314217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 91/183] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:07.314282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.314347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a4a2e-b601-4675-ad56-9fa307344097 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.314347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a4a2e-b601-4675-ad56-9fa307344097 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.314347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a4a2e-b601-4675-ad56-9fa307344097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.331298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e97a4a2e-b601-4675-ad56-9fa307344097 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:07.332420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e97a4a2e-b601-4675-ad56-9fa307344097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:07.364009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:07.409664 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 92/184] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:07.410971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a24b0-1127-4c30-92ac-fc96bc69ee09 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.411262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a24b0-1127-4c30-92ac-fc96bc69ee09 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.411536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a24b0-1127-4c30-92ac-fc96bc69ee09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.426430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209576b-8250-496a-ab6e-5a896faf5932 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.431348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7e9a24b0-1127-4c30-92ac-fc96bc69ee09 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:07.432638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7e9a24b0-1127-4c30-92ac-fc96bc69ee09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:07.464095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:07.464739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.464997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.465272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.466418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:07.467847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:07.469441 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 93/185] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:07.470343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.503913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:07.522439 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 93/186] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:07.522439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a24b0-1127-4c30-92ac-fc96bc69ee09 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.595668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761fa03-e529-4279-9d60-50f6a9b1004e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.619166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:07.643155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:07.644675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.645140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.645857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.647211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:07.647911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a24b0-1127-4c30-92ac-fc96bc69ee09 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.649122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:07.652124 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 94/187] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:07.653066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.687388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037a4ee-072c-4515-925a-839040949366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:07.688261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a4ee-072c-4515-925a-839040949366 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.688605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a4ee-072c-4515-925a-839040949366 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.688992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a4ee-072c-4515-925a-839040949366 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.690019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a4ee-072c-4515-925a-839040949366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:07.690392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a4ee-072c-4515-925a-839040949366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:07.692571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a4ee-072c-4515-925a-839040949366 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:07.703907 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 94/188] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:07.705415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037a4ee-072c-4515-925a-839040949366 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.766263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:07.778059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e8fa-82ee-466c-a795-089a5c95222a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.811695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037a4ee-072c-4515-925a-839040949366 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.813634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:07.814303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.814584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.814857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.825252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:07.830662 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:07.831548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:07.832362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 95/189] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:07.833016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.850868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:07.851713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:07.852202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:07.852735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:07.853468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:07.853781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:07.855243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:07.861564 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 95/190] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/role_assignments?user.id=cd34ffbc794a4c04a8a5c4f195b6d3b6&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:07.862371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:07.891597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:07.958587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9748-dd86-4305-807f-42aa97a1777b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:07.988274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5bf0b-f8a7-46d6-b547-158073670371 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.002524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.003141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.003427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.003743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.004530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] RBAC: Adding query filter params (role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=8116bc11609448a7a4b58a423a52d68d) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:08.004843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.006737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.011595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.014833 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 96/191] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/role_assignments?user.id=8116bc11609448a7a4b58a423a52d68d&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.015887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.041578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.042552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.043799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.044321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] PATH_INFO: `/v3/users/a941dfca3ad641da96f7bb754534f8bb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.066464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=a941dfca3ad641da96f7bb754534f8bb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.069720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.076824 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 96/192] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:07 2022] GET /identity/v3/users/a941dfca3ad641da96f7bb754534f8bb => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.076824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.170671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd0639-7007-4907-8ae8-8436bd493f97 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.189057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.222594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.223247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.223619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.223811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.236294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:08.242301 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:08.243182 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:08.243996 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 97/193] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:08.244666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.247985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46460b68-55dd-4af3-88d6-84cb93e7f6c2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.293537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.294188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.294466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.294746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.295411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:08.295719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.297162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.308012 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 97/194] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.310745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.357447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.362263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792216-bb15-4e49-84dc-3fea354ce440 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.396681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.397304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.397589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.397887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.398994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.400454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.402039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 98/195] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.402794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.418639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773248b3-0f69-4b61-a06c-5b3940806342 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.441702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.452341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.453044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.453330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.453605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.507206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d5ea2-0b5a-46c8-9596-794ca15e8bd7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.520370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.522434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.540923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.541623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.542015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.542365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.543091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:08.543399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.544938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.553762 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 99/196] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.554580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.560991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.621900 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 98/197] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:08.622977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.659033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.663224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680208d-0a22-4406-9a30-2d7dd1eb9195 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.696901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.697463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.697765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.698050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] PATH_INFO: `/v3/users/33f7b6846839476eb4a00d41a07519f6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.699177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] RBAC: Authorizing `identity:get_user(user_id=33f7b6846839476eb4a00d41a07519f6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.700615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.702211 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 100/198] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/users/33f7b6846839476eb4a00d41a07519f6 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.702961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.721689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf19965-f735-4f79-9949-f927a1585c88 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.757719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.758355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.758640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.758920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.759568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.761015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.771115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.783434 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 99/199] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:08 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:31:08.784240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.808767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc9a65-45eb-4ffb-9b28-676583fd5044 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.853050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.853896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.854210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.854574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] PATH_INFO: `/v3/users/a941dfca3ad641da96f7bb754534f8bb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.855888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=a941dfca3ad641da96f7bb754534f8bb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.857666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.864100 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 101/200] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/users/a941dfca3ad641da96f7bb754534f8bb => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.864100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.900311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:08.904909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ca3c0-91f1-4f9c-ac3a-54c407f19086 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:08.945959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:08.946615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:08.947006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:08.947234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:08.948393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:08.950535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:08.952883 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 100/201] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:08.953858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:08.982295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e07a5-0ced-4fb5-beee-6310a9052ee0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:09.028142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:09.028765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:09.029050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:09.029862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:09.034739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:09.079944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d6e9e-e8c6-458a-8a0e-bf2de4ae356a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:09.117729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:09.119421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:09.125680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:09.125680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:09.125992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:09.126324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:09.157646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:09.214908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:09.216689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:09.248055 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 102/202] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:09.249100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:09.283398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:09.327023 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 101/203] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:09.328145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:09.367758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e9d84-fe81-4d90-a90f-d076843d269a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:09.404756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:09.405366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:09.405649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:09.405954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:09.406795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=a941dfca3ad641da96f7bb754534f8bb) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:09.407110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:09.408693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:09.410598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:09.416128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 103/204] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:09 2022] GET /identity/v3/role_assignments?user.id=a941dfca3ad641da96f7bb754534f8bb&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:09.417043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:09.441448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f4831-faa8-4119-bb3d-b8ab2887c016 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:09.481159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:09.482083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:09.482510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:09.482926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:09.484353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:09.486265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:09.488203 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 102/205] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:09 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:09.494985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:09.554984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:09.565931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed1492-55df-4a30-b6be-9386f28c3591 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:09.639767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:09.640676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:09.641108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:09.641528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:09.642934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:09.646768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:09.648860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 104/206] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:09 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:09.654734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:09.691209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d3bc3-d637-4a36-a93c-dccd924cca18 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:09.760038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46a5fe-2f34-4d48-8a00-fefaa1aea52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:09.760579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46a5fe-2f34-4d48-8a00-fefaa1aea52b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:09.760858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46a5fe-2f34-4d48-8a00-fefaa1aea52b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:09.761129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46a5fe-2f34-4d48-8a00-fefaa1aea52b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:09.762594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46a5fe-2f34-4d48-8a00-fefaa1aea52b None admin] RBAC: Adding query filter params (role.id=887708bc8dee457481db8046c4f52ea8, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c, user.id=33f7b6846839476eb4a00d41a07519f6) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:09.762917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46a5fe-2f34-4d48-8a00-fefaa1aea52b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:09.764374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46a5fe-2f34-4d48-8a00-fefaa1aea52b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:09.769469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:09.776171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 103/207] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:09 2022] GET /identity/v3/role_assignments?user.id=33f7b6846839476eb4a00d41a07519f6&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=887708bc8dee457481db8046c4f52ea8 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:09.833914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefed44-e698-43e8-988b-615f85b2e461 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:09.888456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a189fc-5620-4e65-b853-0eaef97de4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:09.889087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a189fc-5620-4e65-b853-0eaef97de4e4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:09.889370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a189fc-5620-4e65-b853-0eaef97de4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:09.889645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a189fc-5620-4e65-b853-0eaef97de4e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:09.890485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a189fc-5620-4e65-b853-0eaef97de4e4 None admin] RBAC: Adding query filter params (role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=a941dfca3ad641da96f7bb754534f8bb) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:09.890798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a189fc-5620-4e65-b853-0eaef97de4e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:09.892326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a189fc-5620-4e65-b853-0eaef97de4e4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:09.901436 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 105/208] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:09 2022] GET /identity/v3/role_assignments?user.id=a941dfca3ad641da96f7bb754534f8bb&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:11.575574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953521c-5955-4833-8cd8-745a29ccaae8 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:11.576088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953521c-5955-4833-8cd8-745a29ccaae8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:11.576162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953521c-5955-4833-8cd8-745a29ccaae8 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:11.577721 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 104/209] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:11.582502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a84b7-574c-40e2-86dd-967705f4070e None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:11.582913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a84b7-574c-40e2-86dd-967705f4070e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:11.583111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a84b7-574c-40e2-86dd-967705f4070e None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:11.584390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd51843-3f58-46cf-a0bb-bf2ad68ac842 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:11.584880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd51843-3f58-46cf-a0bb-bf2ad68ac842 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:11.585230 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 106/210] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:11.585555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd51843-3f58-46cf-a0bb-bf2ad68ac842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:11.590433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d894-cce9-4517-8fe2-8f1b32b83ec6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:11.590718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d894-cce9-4517-8fe2-8f1b32b83ec6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:11.590997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d894-cce9-4517-8fe2-8f1b32b83ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:11.606656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1dd51843-3f58-46cf-a0bb-bf2ad68ac842 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:11.608160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1dd51843-3f58-46cf-a0bb-bf2ad68ac842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:11.610166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e779d894-cce9-4517-8fe2-8f1b32b83ec6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:11.612389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e779d894-cce9-4517-8fe2-8f1b32b83ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:11.685576 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 105/211] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:11.694035 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 107/212] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:12.029179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d894-cce9-4517-8fe2-8f1b32b83ec6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.030336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd51843-3f58-46cf-a0bb-bf2ad68ac842 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.057957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:12.106095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:12.148775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd51843-3f58-46cf-a0bb-bf2ad68ac842 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:12.151691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d894-cce9-4517-8fe2-8f1b32b83ec6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:12.186970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb36b2-df56-4d65-9ee0-8916263f0ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:12.187638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb36b2-df56-4d65-9ee0-8916263f0ce4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.187881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb36b2-df56-4d65-9ee0-8916263f0ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.188156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb36b2-df56-4d65-9ee0-8916263f0ce4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.189172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8309396-b9d9-4241-89fd-f1b57f122471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:12.189812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8309396-b9d9-4241-89fd-f1b57f122471 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.190129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8309396-b9d9-4241-89fd-f1b57f122471 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.190401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8309396-b9d9-4241-89fd-f1b57f122471 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.191510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8309396-b9d9-4241-89fd-f1b57f122471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:12.193774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8309396-b9d9-4241-89fd-f1b57f122471 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:12.195390 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 106/213] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:12.196361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b4c9a-e623-40b6-896b-b56adccaf41d None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.196650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b4c9a-e623-40b6-896b-b56adccaf41d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.196928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b4c9a-e623-40b6-896b-b56adccaf41d None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.202325 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 107/214] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:12.202325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b4c9a-e623-40b6-896b-b56adccaf41d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.206392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb36b2-df56-4d65-9ee0-8916263f0ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-5edb36b2-df56-4d65-9ee0-8916263f0ce4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:12.212299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:12.213435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:12.215156 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 108/215] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:12.217566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c64f-5d60-4f7a-9b5b-f09c801fcf03 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.218174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c64f-5d60-4f7a-9b5b-f09c801fcf03 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.218674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c64f-5d60-4f7a-9b5b-f09c801fcf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.237861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6555c64f-5d60-4f7a-9b5b-f09c801fcf03 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:12.238935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6555c64f-5d60-4f7a-9b5b-f09c801fcf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:12.301657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:12.313832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b4c9a-e623-40b6-896b-b56adccaf41d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:12.321908 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 109/216] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:12.321908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c64f-5d60-4f7a-9b5b-f09c801fcf03 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.354066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7332d51-2630-4fd7-9f1d-8b544efbdfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:12.354882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7332d51-2630-4fd7-9f1d-8b544efbdfd5 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.355175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7332d51-2630-4fd7-9f1d-8b544efbdfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.355444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7332d51-2630-4fd7-9f1d-8b544efbdfd5 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.356121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7332d51-2630-4fd7-9f1d-8b544efbdfd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:12.357661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7332d51-2630-4fd7-9f1d-8b544efbdfd5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:12.385422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:12.386000 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 108/217] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:12 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:31:12.452378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c64f-5d60-4f7a-9b5b-f09c801fcf03 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:12.456001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aeb7a-acae-46ab-9bf1-8bd7c6d796d5 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.456287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aeb7a-acae-46ab-9bf1-8bd7c6d796d5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.456565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aeb7a-acae-46ab-9bf1-8bd7c6d796d5 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.458344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 109/218] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:31:12.464483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1614a6-053f-475a-bb9a-4f80a6ba6599 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.464764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1614a6-053f-475a-bb9a-4f80a6ba6599 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.465042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1614a6-053f-475a-bb9a-4f80a6ba6599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.486504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1f1614a6-053f-475a-bb9a-4f80a6ba6599 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:12.487720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1f1614a6-053f-475a-bb9a-4f80a6ba6599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:12.502284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:12.502819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.503102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.503374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.504051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:12.504376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:12.505873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:12.515968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 110/219] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:12.522245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.592349 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 110/220] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:12.649050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9da00-976a-4354-bb4a-f62621692e26 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:12.666920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:12.688121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:12.688760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.689044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.689324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] PATH_INFO: `/v3/users/26793ca6aaf042b88a917ab50f8942c0` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.703644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1614a6-053f-475a-bb9a-4f80a6ba6599 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.706855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] RBAC: Authorizing `identity:get_user(user_id=26793ca6aaf042b88a917ab50f8942c0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:12.708326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:12.710482 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 111/221] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:12 2022] GET /identity/v3/users/26793ca6aaf042b88a917ab50f8942c0 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:12.716375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.810672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:12.846569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1614a6-053f-475a-bb9a-4f80a6ba6599 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:12.854132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a29c7-1c55-4d5f-a858-20146b13ae88 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:12.889497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:12.890178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.890459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.890737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.897786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:12.898620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:12.899210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:12.899497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:12.901708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:12.907717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:12.908645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:12.909432 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 111/222] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:31:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:12.915633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:12.948680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:13.001889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.003358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.043340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5bdd9-ce19-4a8d-8012-c9640c87845c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.079568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.080400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.080822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.081239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.082092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:13.082557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.084630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.090888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:13.094934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 112/223] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:13.095902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.117612 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 112/224] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:31:13.118898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.207811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:13.218556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8cfed-04cf-4911-b939-c0214f7e2957 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.247464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37685f-d460-485e-a509-979fd81f5051 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.264485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.265374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.265658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.266358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.281797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:13.284597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.285234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.285520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.285804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] PATH_INFO: `/v3/users/a941dfca3ad641da96f7bb754534f8bb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.287133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] RBAC: Authorizing `identity:get_user(user_id=a941dfca3ad641da96f7bb754534f8bb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:13.288041 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:13.288877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:13.290088 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:13.290926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.290995 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 113/225] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:13.290995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.291129 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 113/226] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/users/a941dfca3ad641da96f7bb754534f8bb => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:13.295698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.371949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:13.410421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230e01-3ce3-4cd0-a0b9-8ab5c4a3a249 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.435323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d86a-43aa-4dc0-8bca-76e15717f7d6 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.460118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.461616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.462076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.462493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.464979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.467424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.469641 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 114/227] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:13.470584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.477398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.478566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.480012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.480437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.481792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.483409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.485850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:13.486203 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 114/228] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:13.492797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.606816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937a10-2a3e-4c44-b33e-2654df44c619 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.607441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223c6b8-c213-4c77-9094-2ca851eb9c2a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.636451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:13.644565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.645258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.645692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.646149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.646824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.646958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:13.647275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.647973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.648424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.648867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c/users/a941dfca3ad641da96f7bb754534f8bb/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.649053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.651236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3066e494914bca9d3624f63d34656c, user_id=a941dfca3ad641da96f7bb754534f8bb, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.652957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.658048 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 115/229] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:13.658965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.669927 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 115/230] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:31:13 2022] PUT /identity/v3/projects/fc3066e494914bca9d3624f63d34656c/users/a941dfca3ad641da96f7bb754534f8bb/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:13.671082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.773998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:13.787230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251c539-1321-4561-a603-c345e1ffa2a5 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.824507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.825505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.825705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.826104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] PATH_INFO: `/v3/users/a941dfca3ad641da96f7bb754534f8bb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.834126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] RBAC: Authorizing `identity:get_user(user_id=a941dfca3ad641da96f7bb754534f8bb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.834126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.834126 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 116/231] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/users/a941dfca3ad641da96f7bb754534f8bb => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:13.838213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:13.844785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462ad24-61fd-48eb-be42-ad4a632d724e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:13.928281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:13.929086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:13.929367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:13.929648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:13.930426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=26793ca6aaf042b88a917ab50f8942c0) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:13.930742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:13.932271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:13.947727 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 116/232] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/role_assignments?user.id=26793ca6aaf042b88a917ab50f8942c0&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:13.960696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:14.013961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a990d0-3523-40e7-89bc-9946d963556b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:14.052163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496cdea-5fc5-447e-9d2d-967ee37acfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:14.053873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496cdea-5fc5-447e-9d2d-967ee37acfb5 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:14.054166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496cdea-5fc5-447e-9d2d-967ee37acfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:14.054443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496cdea-5fc5-447e-9d2d-967ee37acfb5 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:14.055777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496cdea-5fc5-447e-9d2d-967ee37acfb5 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:14.057310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496cdea-5fc5-447e-9d2d-967ee37acfb5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:14.059390 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 117/233] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:13 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:14.065146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:14.236838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acda43e-1b65-44b3-8d8a-e0269bbf034a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:14.275154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:14.281805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25334bac-1524-474a-834a-52cf9448be92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:14.282777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25334bac-1524-474a-834a-52cf9448be92 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:14.283276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25334bac-1524-474a-834a-52cf9448be92 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:14.283746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25334bac-1524-474a-834a-52cf9448be92 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c/users/a941dfca3ad641da96f7bb754534f8bb/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:14.286382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25334bac-1524-474a-834a-52cf9448be92 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3066e494914bca9d3624f63d34656c, user_id=a941dfca3ad641da96f7bb754534f8bb, role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:14.289064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25334bac-1524-474a-834a-52cf9448be92 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:14.314010 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 117/234] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:31:14 2022] PUT /identity/v3/projects/fc3066e494914bca9d3624f63d34656c/users/a941dfca3ad641da96f7bb754534f8bb/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:15.352863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce5fe-ab49-4b1f-b2ba-2049ea2809d0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:15.353374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce5fe-ab49-4b1f-b2ba-2049ea2809d0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:15.353448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce5fe-ab49-4b1f-b2ba-2049ea2809d0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:15.355043 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 118/235] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:15.360418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4259278-b811-479c-a59a-ede87757104e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:15.360831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4259278-b811-479c-a59a-ede87757104e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:15.361010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4259278-b811-479c-a59a-ede87757104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:15.386996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d4259278-b811-479c-a59a-ede87757104e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:15.388193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d4259278-b811-479c-a59a-ede87757104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:15.417470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ba8b-53aa-4443-89a2-fb2f42e4dcc0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:15.417779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ba8b-53aa-4443-89a2-fb2f42e4dcc0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:15.418129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ba8b-53aa-4443-89a2-fb2f42e4dcc0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:15.419796 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 119/236] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:15.425486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae54cbb-5dbb-414a-836d-c196773ee549 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:15.425797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae54cbb-5dbb-414a-836d-c196773ee549 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:15.426091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae54cbb-5dbb-414a-836d-c196773ee549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:15.448609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-eae54cbb-5dbb-414a-836d-c196773ee549 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:15.449814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eae54cbb-5dbb-414a-836d-c196773ee549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:15.492108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 118/237] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:15.539228 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 120/238] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:15.894033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4259278-b811-479c-a59a-ede87757104e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:16.009440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4259278-b811-479c-a59a-ede87757104e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:16.038257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:16.048312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:16.048944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.049224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.049501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.059987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:16.064932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:16.065806 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:16.066641 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 119/239] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:16.075239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae54cbb-5dbb-414a-836d-c196773ee549 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:16.120802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:16.152983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:16.205939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae54cbb-5dbb-414a-836d-c196773ee549 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:16.239251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:16.239713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.244408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.244408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.244408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:16.244408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:16.244408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:16.252367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 121/240] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:16.258529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:16.324673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae737c-28ab-40f7-abec-22c3ed06bdb4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:16.361911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690fc86-eaa0-4184-8150-ff5eb2e8cc4a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:16.394218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:16.395955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:16.396678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.396969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.397250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.402711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:16.403254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:16.404138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.404257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:16.404762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.405238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] PATH_INFO: `/v3/users/e7a667368c014f5f8f2ab59ea49a5135` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.406154 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 120/241] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:16.411637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:16.427933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] RBAC: Authorizing `identity:get_user(user_id=e7a667368c014f5f8f2ab59ea49a5135)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:16.429873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:16.432273 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 122/242] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:16 2022] GET /identity/v3/users/e7a667368c014f5f8f2ab59ea49a5135 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:16.438253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:16.519506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:16.556778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232422c3-affb-4cf3-9c45-230b7b88668e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:16.593294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f824dd-2195-432b-bb7a-942e37617900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:16.594337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f824dd-2195-432b-bb7a-942e37617900 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.594887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f824dd-2195-432b-bb7a-942e37617900 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.595357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f824dd-2195-432b-bb7a-942e37617900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.599925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86664897-4a34-4d00-8b5d-f0dccdbd7353 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:16.657199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:16.670583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f824dd-2195-432b-bb7a-942e37617900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:16.672402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f824dd-2195-432b-bb7a-942e37617900 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:16.682536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:16.683173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.683459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.684005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.684396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:16.685856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:16.726260 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 121/243] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:16 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:31:16.787545 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 123/244] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:16.793718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:16.882330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fbdd0-cb4d-4f9c-b05a-ddb1c3143575 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.882908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fbdd0-cb4d-4f9c-b05a-ddb1c3143575 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.883480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fbdd0-cb4d-4f9c-b05a-ddb1c3143575 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.885307 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 124/245] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:16.901499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1356e8-c925-47c4-8686-6f0b718c2a60 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:16.902878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1356e8-c925-47c4-8686-6f0b718c2a60 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:16.903376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1356e8-c925-47c4-8686-6f0b718c2a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:16.927028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1e1356e8-c925-47c4-8686-6f0b718c2a60 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:16.929341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e1356e8-c925-47c4-8686-6f0b718c2a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:16.961411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38afe09-3282-4ce0-bbd3-3f751d2884a4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:16.978673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:17.022656 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 125/246] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:17.054045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a2b42-da4b-4916-b1aa-e5043df58b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:17.054692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a2b42-da4b-4916-b1aa-e5043df58b0f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.054976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a2b42-da4b-4916-b1aa-e5043df58b0f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.055255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a2b42-da4b-4916-b1aa-e5043df58b0f None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.056407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a2b42-da4b-4916-b1aa-e5043df58b0f None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:17.057963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a2b42-da4b-4916-b1aa-e5043df58b0f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:17.059911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 122/247] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:16 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:17.065586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1356e8-c925-47c4-8686-6f0b718c2a60 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:17.070980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:17.111119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1b8de-d84d-4c7d-98b0-2d4946277c5d None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.111699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1b8de-d84d-4c7d-98b0-2d4946277c5d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.111977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1b8de-d84d-4c7d-98b0-2d4946277c5d None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.113448 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 123/248] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:31:17.119286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b572-33ac-4d31-9689-b320a5676a93 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.119822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b572-33ac-4d31-9689-b320a5676a93 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.120294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b572-33ac-4d31-9689-b320a5676a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.141298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0ad9b572-33ac-4d31-9689-b320a5676a93 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:17.142930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ad9b572-33ac-4d31-9689-b320a5676a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:17.189719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1356e8-c925-47c4-8686-6f0b718c2a60 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:17.229254 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 124/249] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:17.235932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:17.236555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.236837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.237115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.237886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] RBAC: Adding query filter params (role.id=887708bc8dee457481db8046c4f52ea8, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=e7a667368c014f5f8f2ab59ea49a5135) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:17.238206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:17.239766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:17.246065 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 126/250] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:17 2022] GET /identity/v3/role_assignments?user.id=e7a667368c014f5f8f2ab59ea49a5135&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=887708bc8dee457481db8046c4f52ea8 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:17.408955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b572-33ac-4d31-9689-b320a5676a93 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:17.530727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b572-33ac-4d31-9689-b320a5676a93 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:17.533302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:17.568425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:17.569337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.569763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.570204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.582136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:17.588243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:17.589332 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:17.590351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:17.592455 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 125/251] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:17.597899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:17.618341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:17.723964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:17.731803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e7db9-6fe1-4598-b9ec-def933daf40b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:17.763826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e57742-9529-4fe9-a80f-b0901ee14cf0 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:17.773485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb3dd3-5609-4c8a-96f4-05d734674e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:17.774407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb3dd3-5609-4c8a-96f4-05d734674e75 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.774831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb3dd3-5609-4c8a-96f4-05d734674e75 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.775645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb3dd3-5609-4c8a-96f4-05d734674e75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.788583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb3dd3-5609-4c8a-96f4-05d734674e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-46cb3dd3-5609-4c8a-96f4-05d734674e75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:17.794250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:17.795228 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:17.796032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:17.798577 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 126/252] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:17.799928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcd212-bf39-4f1a-99bb-ec8f65e02dfe None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.800389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcd212-bf39-4f1a-99bb-ec8f65e02dfe None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.800806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcd212-bf39-4f1a-99bb-ec8f65e02dfe None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.803176 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 127/253] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:17.805009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcd212-bf39-4f1a-99bb-ec8f65e02dfe None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:17.810694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f53aa-e97b-480e-ac5f-2c80a37b49ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:17.811727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f53aa-e97b-480e-ac5f-2c80a37b49ab None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.813064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f53aa-e97b-480e-ac5f-2c80a37b49ab None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.813487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f53aa-e97b-480e-ac5f-2c80a37b49ab None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.814348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f53aa-e97b-480e-ac5f-2c80a37b49ab None admin] RBAC: Adding query filter params (name=member) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:17.814795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f53aa-e97b-480e-ac5f-2c80a37b49ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:17.816391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f53aa-e97b-480e-ac5f-2c80a37b49ab None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:17.831336 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 127/254] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:17.834007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8eacb-5ded-49d0-89f6-e214addbd283 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:17.834468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8eacb-5ded-49d0-89f6-e214addbd283 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:17.834949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8eacb-5ded-49d0-89f6-e214addbd283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:17.848612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:17.878343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ead8eacb-5ded-49d0-89f6-e214addbd283 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:17.879552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ead8eacb-5ded-49d0-89f6-e214addbd283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:17.987320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcd212-bf39-4f1a-99bb-ec8f65e02dfe None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:18.023141 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 128/255] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:18.024064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8eacb-5ded-49d0-89f6-e214addbd283 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:18.055536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:18.056145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:18.056645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:18.056920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:18.057579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:18.057901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:18.059299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:18.078784 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 128/256] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:18.084839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:18.147108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:18.228593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8eacb-5ded-49d0-89f6-e214addbd283 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:18.287629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:18.300076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:18.300739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:18.301026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:18.301310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] PATH_INFO: `/v3/users/26793ca6aaf042b88a917ab50f8942c0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:18.306601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] RBAC: Authorizing `identity:get_user(user_id=26793ca6aaf042b88a917ab50f8942c0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:18.308032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:18.309969 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 129/257] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:18 2022] GET /identity/v3/users/26793ca6aaf042b88a917ab50f8942c0 => generated 297 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:18.317487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022b10-247b-4be5-bb98-c3fdc80f8652 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:18.321085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:18.414564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:18.415264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:18.415499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:18.415775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] PATH_INFO: `/v3/users/a941dfca3ad641da96f7bb754534f8bb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:18.416980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] RBAC: Authorizing `identity:get_user(user_id=a941dfca3ad641da96f7bb754534f8bb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:18.418464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:18.420057 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 129/258] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:18 2022] GET /identity/v3/users/a941dfca3ad641da96f7bb754534f8bb => generated 283 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:18.425449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:18.497005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76274106-80ad-4ab8-a6ae-d86fe62a2193 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:18.546145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:18.561260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:18.561843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:18.562126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:18.562399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:18.564214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:18.565681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:18.570062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 130/259] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:18 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:18.570860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:18.629905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a174-999b-48fb-a5bb-22a0219eb39b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:18.704800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:18.705464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:18.705753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:18.706076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:18.749204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:18.759641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c6557-96ce-45c5-9e65-47d2762fed2e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:18.832707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:18.833279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:18.833606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:18.833944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:18.839329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:18.840852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:18.863139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:18.870121 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:18.871030 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 131/260] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:18.872205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:19.007539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:19.019655 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 130/261] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:31:19.020713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:19.082187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:19.099000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b294c1f-06f1-4aab-ac10-4c51c03b143f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:19.144130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:19.144799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:19.145087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:19.145366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c/users/26793ca6aaf042b88a917ab50f8942c0/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:19.150366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3066e494914bca9d3624f63d34656c, user_id=26793ca6aaf042b88a917ab50f8942c0, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:19.151828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:19.172380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d037d-ffb3-4dc2-a612-e72a7828cd1e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:19.179997 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 132/262] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:31:18 2022] PUT /identity/v3/projects/fc3066e494914bca9d3624f63d34656c/users/26793ca6aaf042b88a917ab50f8942c0/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:19.180826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:19.275065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:19.276080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:19.276365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:19.276644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:19.277354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:19.277662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:19.279749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:19.308883 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 131/263] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:19.316763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:19.370862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142eb958-64aa-40c6-b907-43f61e6f6499 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:19.393356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:19.420087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:19.421208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:19.421636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:19.422114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:19.423741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:19.425243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:19.427594 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 133/264] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:19 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:19.433270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:19.548111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357050bd-7f32-4851-b54c-95586e16347d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:19.556040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464fc1a-0c73-4650-869d-f3c929425688 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:19.595851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:19.616642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:19.617258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:19.617542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:19.617817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:19.618581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=a941dfca3ad641da96f7bb754534f8bb) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:19.618889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:19.620340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:19.633168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:19.633738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:19.634046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:19.634321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] PATH_INFO: `/v3/users/a941dfca3ad641da96f7bb754534f8bb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:19.634866 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 134/265] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:19 2022] GET /identity/v3/role_assignments?user.id=a941dfca3ad641da96f7bb754534f8bb&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:19.635492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] RBAC: Authorizing `identity:get_user(user_id=a941dfca3ad641da96f7bb754534f8bb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:19.636954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:19.638913 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 132/266] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:19 2022] GET /identity/v3/users/a941dfca3ad641da96f7bb754534f8bb => generated 283 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:19.647069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:19.728936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:19.809411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9291-a9c6-40ee-872f-ba53b1fda66b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:19.854777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:19.855399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:19.855687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:19.855961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:19.977452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:19.978993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:20.046114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:20.117774 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 135/267] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:20.123475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:20.243093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08332887-4c72-4564-b6a9-c153d651947e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:20.284587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd77b8-cc04-48f9-b98d-50c6ba36dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:20.285231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd77b8-cc04-48f9-b98d-50c6ba36dceb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.285514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd77b8-cc04-48f9-b98d-50c6ba36dceb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.285790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd77b8-cc04-48f9-b98d-50c6ba36dceb None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.290562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd77b8-cc04-48f9-b98d-50c6ba36dceb None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:20.292031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd77b8-cc04-48f9-b98d-50c6ba36dceb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:20.294004 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 133/268] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:20 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:20.300178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:20.429089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b2360-796b-4fac-8bc8-f820a97e9f53 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:20.467422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968adc16-8413-4c6a-825b-6e185f452d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:20.468032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968adc16-8413-4c6a-825b-6e185f452d96 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.468312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968adc16-8413-4c6a-825b-6e185f452d96 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.468584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968adc16-8413-4c6a-825b-6e185f452d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.469309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968adc16-8413-4c6a-825b-6e185f452d96 None admin] RBAC: Adding query filter params (role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=a941dfca3ad641da96f7bb754534f8bb) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:20.469614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968adc16-8413-4c6a-825b-6e185f452d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:20.471145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968adc16-8413-4c6a-825b-6e185f452d96 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:20.477345 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 136/269] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:20 2022] GET /identity/v3/role_assignments?user.id=a941dfca3ad641da96f7bb754534f8bb&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:20.721950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec7a0-9b43-4f01-aae3-07dc1f139138 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.722972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec7a0-9b43-4f01-aae3-07dc1f139138 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.723453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec7a0-9b43-4f01-aae3-07dc1f139138 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.725422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 134/270] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:20.731146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974bd36-b77a-4503-9767-706501ca31cb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.732814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974bd36-b77a-4503-9767-706501ca31cb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.733146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974bd36-b77a-4503-9767-706501ca31cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.733450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffd220-3e53-41dc-85ac-864a9677b423 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.733958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffd220-3e53-41dc-85ac-864a9677b423 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.734427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffd220-3e53-41dc-85ac-864a9677b423 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.736160 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 135/271] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:31:20.743949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914a0b-bbb8-4183-9899-9632c5db4558 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.744293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914a0b-bbb8-4183-9899-9632c5db4558 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.744521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914a0b-bbb8-4183-9899-9632c5db4558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.752740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0974bd36-b77a-4503-9767-706501ca31cb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:20.757510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0974bd36-b77a-4503-9767-706501ca31cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:20.764692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-81914a0b-bbb8-4183-9899-9632c5db4558 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:20.765709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:20.765968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-81914a0b-bbb8-4183-9899-9632c5db4558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:20.832613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:20.841518 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 137/272] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:20.844066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de538a-7cd7-466d-b81f-99efd41bd44e None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.844359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de538a-7cd7-466d-b81f-99efd41bd44e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.844631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de538a-7cd7-466d-b81f-99efd41bd44e None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.849849 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 138/273] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:20.852649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481b266-72af-48b5-b2d6-90eac3bad98e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:20.852940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481b266-72af-48b5-b2d6-90eac3bad98e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:20.853212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481b266-72af-48b5-b2d6-90eac3bad98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:20.853964 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 136/274] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:20.883571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8481b266-72af-48b5-b2d6-90eac3bad98e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:20.884766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8481b266-72af-48b5-b2d6-90eac3bad98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:20.963544 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 139/275] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:21.202719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81914a0b-bbb8-4183-9899-9632c5db4558 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.215680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481b266-72af-48b5-b2d6-90eac3bad98e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.297655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:21.314565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81914a0b-bbb8-4183-9899-9632c5db4558 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.332248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481b266-72af-48b5-b2d6-90eac3bad98e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.351976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.352610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.352890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.353160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.364367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:21.369923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:21.370918 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:21.371941 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 137/276] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:21.374946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.375920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.376674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.376925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.377202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.387888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:21.392880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:21.393771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:21.394885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:21.399470 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 140/277] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:21.400174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.424971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:21.492457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386a1b8-95ea-40bb-a031-29fb001e9370 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.506318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d7386-427f-41ac-9a21-c303751032eb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.530149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.530784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.531059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.531333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.532450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:21.533933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:21.535646 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 138/278] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:21.536315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.545434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:21.552113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.553143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.553684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.554133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.555053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:21.555497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:21.557120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:21.566115 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 141/279] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:21.571108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.649730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcdef0-2643-40b1-b6cf-7f8531086dd0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.663089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:21.694171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.694588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915df05-ad1b-45b9-b2fe-c1099950b110 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.694784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.695065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.695337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.696009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:21.696313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:21.697680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:21.713173 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 139/280] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:21.713173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.734519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.735377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.735804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.736342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.736994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:21.738481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:21.762857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 142/281] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:21 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:31:21.763752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.839702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:21.858403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e9955-2bd7-411e-a27c-cd79df082c02 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.907731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.908387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.908670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.908946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] PATH_INFO: `/v3/users/e7a667368c014f5f8f2ab59ea49a5135` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.910317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] RBAC: Authorizing `identity:get_user(user_id=e7a667368c014f5f8f2ab59ea49a5135)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:21.912380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:21.914490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 140/282] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/users/e7a667368c014f5f8f2ab59ea49a5135 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:21.920479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:21.921851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800a379-3431-4086-921e-e6b3cb5a7d26 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:21.962294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:21.963564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:21.966454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:21.967025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] PATH_INFO: `/v3/users/383901f6540845b4bbf9233386be30ed` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:21.971800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:21.986669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] RBAC: Authorizing `identity:get_user(user_id=383901f6540845b4bbf9233386be30ed)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:21.988445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:21.990683 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 143/283] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/users/383901f6540845b4bbf9233386be30ed => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:21.998436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:22.050891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63f13b-0788-443d-8b2d-a2ba3f9012de None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:22.093886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:22.094458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:22.094740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:22.095017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:22.096393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:22.098312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:22.101212 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 141/284] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:22.107210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:22.117890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000b4d9-295e-4071-b7f3-43160f3e53c8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:22.129754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:22.162902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cb762-9a29-485d-b58b-85269088f6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:22.163643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cb762-9a29-485d-b58b-85269088f6d9 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:22.163927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cb762-9a29-485d-b58b-85269088f6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:22.164199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cb762-9a29-485d-b58b-85269088f6d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:22.164977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cb762-9a29-485d-b58b-85269088f6d9 None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, user.id=383901f6540845b4bbf9233386be30ed, scope.system=all) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:22.165302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cb762-9a29-485d-b58b-85269088f6d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:22.166863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cb762-9a29-485d-b58b-85269088f6d9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:22.175242 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 144/285] 10.222.0.33 () {56 vars in 1308 bytes} [Sat Sep 10 20:31:21 2022] GET /identity/v3/role_assignments?user.id=383901f6540845b4bbf9233386be30ed&scope.system=all&role.id=9d599f0800724bd4b78e84df540784c8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:22.176496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a533ab6-fd1b-43ee-8d75-7141dcd6954f None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:22.176783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a533ab6-fd1b-43ee-8d75-7141dcd6954f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:22.177065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a533ab6-fd1b-43ee-8d75-7141dcd6954f None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:22.179947 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 145/286] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:22.185743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5cada-b3a8-47ca-9669-d51df69c498b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:22.186073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5cada-b3a8-47ca-9669-d51df69c498b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:22.186355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5cada-b3a8-47ca-9669-d51df69c498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:22.210489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f2e5cada-b3a8-47ca-9669-d51df69c498b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:22.211780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f2e5cada-b3a8-47ca-9669-d51df69c498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:22.246338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71174b84-ba86-4d2d-b38c-33f4e2095edd None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:22.258863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:22.294899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:22.295621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:22.295906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:22.296184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c/users/e7a667368c014f5f8f2ab59ea49a5135/roles/887708bc8dee457481db8046c4f52ea8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:22.298297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3066e494914bca9d3624f63d34656c, user_id=e7a667368c014f5f8f2ab59ea49a5135, role_id=887708bc8dee457481db8046c4f52ea8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:22.299783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:22.326514 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 146/287] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:22.337205 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 142/288] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:31:22 2022] PUT /identity/v3/projects/fc3066e494914bca9d3624f63d34656c/users/e7a667368c014f5f8f2ab59ea49a5135/roles/887708bc8dee457481db8046c4f52ea8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:22.743249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5cada-b3a8-47ca-9669-d51df69c498b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:22.869284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5cada-b3a8-47ca-9669-d51df69c498b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:22.871439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:22.914154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:22.915160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:22.915660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:22.916145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:22.934781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:22.940663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:22.941582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:22.942500 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:22.950065 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 147/289] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:22.954140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:23.088268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a030343-2cc9-42a3-a29e-3f86894960aa None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:23.129240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:23.129869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:23.130153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:23.131017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:23.131676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:23.131978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:23.133352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:23.142082 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 143/290] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:23.147749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:23.367041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dd2d0-248d-4d80-860a-198e089e03ae None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:23.418539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:23.419538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:23.420037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:23.420497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] PATH_INFO: `/v3/users/26793ca6aaf042b88a917ab50f8942c0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:23.421935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] RBAC: Authorizing `identity:get_user(user_id=26793ca6aaf042b88a917ab50f8942c0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:23.423571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:23.425623 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 148/291] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:23 2022] GET /identity/v3/users/26793ca6aaf042b88a917ab50f8942c0 => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:23.431557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:23.468737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:23.541920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c07f5-228d-42f3-b693-0758954f27a4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:23.582819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:23.583505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:23.583787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:23.584120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:23.654845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:23.656215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:23.685788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:23.759096 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 144/292] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:23.764740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:23.797250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:23.882934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1c0d5-8aed-4ffa-9ab8-cd2415792882 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:23.932463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7199f6-3d5e-4cc2-b080-82dc597fb40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:23.933011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7199f6-3d5e-4cc2-b080-82dc597fb40c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:23.933287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7199f6-3d5e-4cc2-b080-82dc597fb40c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:23.933557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7199f6-3d5e-4cc2-b080-82dc597fb40c None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:23.934719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7199f6-3d5e-4cc2-b080-82dc597fb40c None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:23.936167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7199f6-3d5e-4cc2-b080-82dc597fb40c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:23.937897 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 149/293] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:23 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:23.944895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:24.089955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2cd4b-f671-48b3-8bdc-30be94a0d1b5 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:24.121360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:24.129179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af0594-e9a5-4b44-8ec7-4e006a6846b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:24.130845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0594-e9a5-4b44-8ec7-4e006a6846b0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:24.131330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0594-e9a5-4b44-8ec7-4e006a6846b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:24.131797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0594-e9a5-4b44-8ec7-4e006a6846b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:24.132721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af0594-e9a5-4b44-8ec7-4e006a6846b0 None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=26793ca6aaf042b88a917ab50f8942c0) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:24.133228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af0594-e9a5-4b44-8ec7-4e006a6846b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:24.135141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af0594-e9a5-4b44-8ec7-4e006a6846b0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:24.141775 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 145/294] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:23 2022] GET /identity/v3/role_assignments?user.id=26793ca6aaf042b88a917ab50f8942c0&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:24.840961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504776-cd8b-433b-96ca-a00e0c814e21 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:24.841403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504776-cd8b-433b-96ca-a00e0c814e21 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:24.841504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504776-cd8b-433b-96ca-a00e0c814e21 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:24.842930 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 150/295] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:24.849473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edbf44-08ab-4997-af4c-cbbc73c458ba None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:24.849778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edbf44-08ab-4997-af4c-cbbc73c458ba None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:24.850070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edbf44-08ab-4997-af4c-cbbc73c458ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:24.870044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f5edbf44-08ab-4997-af4c-cbbc73c458ba None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:24.871546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f5edbf44-08ab-4997-af4c-cbbc73c458ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:24.957793 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 146/296] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:25.030628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad9cbb-e7e8-4232-9007-1826db358886 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.031585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad9cbb-e7e8-4232-9007-1826db358886 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.032050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad9cbb-e7e8-4232-9007-1826db358886 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.033585 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 151/297] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:25.039207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d600-d05a-4d20-8e55-a7cde2afcfb1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.039677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d600-d05a-4d20-8e55-a7cde2afcfb1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.040098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d600-d05a-4d20-8e55-a7cde2afcfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.059408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9a22d600-d05a-4d20-8e55-a7cde2afcfb1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:25.060833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9a22d600-d05a-4d20-8e55-a7cde2afcfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:25.138030 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 147/298] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:25.292170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad9cbb-e7e8-4232-9007-1826db358886 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:25.352384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc573199-b619-408d-8657-c8c470c6b728 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.353015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc573199-b619-408d-8657-c8c470c6b728 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.353521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc573199-b619-408d-8657-c8c470c6b728 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.355383 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 148/299] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:31:25.360557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc590af-be23-4716-bc96-39e64f881a43 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.361063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc590af-be23-4716-bc96-39e64f881a43 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.361543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc590af-be23-4716-bc96-39e64f881a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.381177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5fc590af-be23-4716-bc96-39e64f881a43 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:25.382765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5fc590af-be23-4716-bc96-39e64f881a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:25.399201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:25.404395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad9cbb-e7e8-4232-9007-1826db358886 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:25.437777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:25.442491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:25.443659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.443939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.444208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.455846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:25.469992 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 149/300] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:25.470180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:25.471102 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:25.471972 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:25.472562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5275-af15-4df5-a651-bc20619ba282 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.472562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5275-af15-4df5-a651-bc20619ba282 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.472562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5275-af15-4df5-a651-bc20619ba282 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.472736 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 152/301] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:25.472819 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 150/302] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:25.474920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:25.475215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37a675-4d90-4ade-9d84-a98224d4664b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.475494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37a675-4d90-4ade-9d84-a98224d4664b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.475767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37a675-4d90-4ade-9d84-a98224d4664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.495015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bc37a675-4d90-4ade-9d84-a98224d4664b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:25.496193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bc37a675-4d90-4ade-9d84-a98224d4664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:25.565277 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 151/303] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:25.574758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37a675-4d90-4ade-9d84-a98224d4664b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:25.591919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cdd66-9560-47e9-80c9-8b907690a2a0 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:25.633364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:25.634261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.634686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.635170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.636030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:25.636553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:25.638272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:25.652842 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 153/304] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:25.654075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:25.678319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:25.714152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37a675-4d90-4ade-9d84-a98224d4664b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:25.729461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:25.753303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:25.754370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.754892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.755976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.757320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:25.758997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:25.761057 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 152/305] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:25.766803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:25.785611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc55540-7fb5-48e7-b316-62d0a42b24a8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:25.822856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:25.823500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.823774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.824054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] PATH_INFO: `/v3/users/427caed5c90247618e08749e1c486691` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.840043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] RBAC: Authorizing `identity:get_user(user_id=427caed5c90247618e08749e1c486691)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:25.841502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:25.843306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 154/306] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:25 2022] GET /identity/v3/users/427caed5c90247618e08749e1c486691 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:25.843988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:25.875664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f842f3-60b0-498a-aba8-31743e32da10 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:25.911159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:25.912427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:25.912747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:25.913026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:25.913653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:25.915118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:25.925767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:25.930926 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 153/307] 10.222.0.33 () {60 vars in 1134 bytes} [Sat Sep 10 20:31:25 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:31:25.931582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:25.955432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92980204-6396-4ecc-b4fc-0bcd036d5d6f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:25.985156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:26.002715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.003394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.003673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.003945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.015954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:26.021655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:26.022659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 155/308] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:26.028600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:26.060125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6608-3a62-491e-a6c5-a9dd52e80028 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.104167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.105255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.105255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.105583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.106436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] RBAC: Adding query filter params (role.id=887708bc8dee457481db8046c4f52ea8, user.id=427caed5c90247618e08749e1c486691, scope.system=all) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:26.106818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:26.108350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:26.114617 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 154/309] 10.222.0.33 () {56 vars in 1308 bytes} [Sat Sep 10 20:31:25 2022] GET /identity/v3/role_assignments?user.id=427caed5c90247618e08749e1c486691&scope.system=all&role.id=887708bc8dee457481db8046c4f52ea8 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:26.126919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:26.146264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bae22-7e91-4c61-a6cd-3a0ff98787e1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.203639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.204571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.204989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.205405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.215804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:26.221200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:26.227677 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:26.228631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:26.230604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 156/310] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:26.235201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:26.264972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd92761-e098-4e65-9cf2-41c74cc028b7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.286945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:26.319514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.321126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.321415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.321690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.322481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:26.322743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:26.324142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:26.335645 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 155/311] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:26.341544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:26.361291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ae8d-90a7-4819-9edd-283bc198b41c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.409425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.410369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.410789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.411217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.413011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] RBAC: Adding query filter params (name=member) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:26.413628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:26.415326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:26.429928 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 157/312] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:26.432157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:26.467876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb4e6e-075c-41bb-8b1e-8bcea37b49aa None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.515501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:26.516104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.516900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.517080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.517276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] PATH_INFO: `/v3/users/e7a667368c014f5f8f2ab59ea49a5135` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.519323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] RBAC: Authorizing `identity:get_user(user_id=e7a667368c014f5f8f2ab59ea49a5135)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:26.521367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:26.528094 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 156/313] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:26 2022] GET /identity/v3/users/e7a667368c014f5f8f2ab59ea49a5135 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:26.530014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:26.581266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a68231-98a4-428e-8235-ed4eacd8de06 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.594333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:26.626824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.628177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.628898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.629345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] PATH_INFO: `/v3/users/383901f6540845b4bbf9233386be30ed` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.630847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] RBAC: Authorizing `identity:get_user(user_id=383901f6540845b4bbf9233386be30ed)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:26.633116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:26.636087 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 158/314] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:26 2022] GET /identity/v3/users/383901f6540845b4bbf9233386be30ed => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:26.642153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:26.644103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a845e-41bd-4e3f-ac90-8ecca3f29e41 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.689736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.690636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.691242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.691686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.755902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36b8f-b8dc-4162-bf7c-9493ba791142 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:26.784007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:26.793915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:26.794179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:26.794969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:26.795344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:26.795826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] PATH_INFO: `/v3/system/users/383901f6540845b4bbf9233386be30ed/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:26.796295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:26.797460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=383901f6540845b4bbf9233386be30ed, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:26.798964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:26.819223 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 159/315] 10.222.0.33 () {58 vars in 1334 bytes} [Sat Sep 10 20:31:26 2022] PUT /identity/v3/system/users/383901f6540845b4bbf9233386be30ed/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:31:26.837270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:26.913534 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 157/316] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:26.919415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:27.035904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6c8a1-cb02-4c38-ac08-25e8e373c09b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:27.073305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0ac63-421b-4705-b478-03c1ab0a5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:27.073961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ac63-421b-4705-b478-03c1ab0a5a7c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:27.074248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ac63-421b-4705-b478-03c1ab0a5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:27.074527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ac63-421b-4705-b478-03c1ab0a5a7c None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:27.075761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0ac63-421b-4705-b478-03c1ab0a5a7c None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:27.077219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0ac63-421b-4705-b478-03c1ab0a5a7c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:27.078938 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 160/317] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:26 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:27.085173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:27.191005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07f84-b2e9-467e-84e2-6a35b6d5e48d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:27.227899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5c4cf-0c20-43cb-90ed-a6992b41554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:27.228534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c4cf-0c20-43cb-90ed-a6992b41554b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:27.228820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c4cf-0c20-43cb-90ed-a6992b41554b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:27.229100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c4cf-0c20-43cb-90ed-a6992b41554b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:27.229887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c4cf-0c20-43cb-90ed-a6992b41554b None admin] RBAC: Adding query filter params (role.id=887708bc8dee457481db8046c4f52ea8, scope.project.id=fc3066e494914bca9d3624f63d34656c, user.id=e7a667368c014f5f8f2ab59ea49a5135) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:27.230211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c4cf-0c20-43cb-90ed-a6992b41554b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:27.231713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c4cf-0c20-43cb-90ed-a6992b41554b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:27.235242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:27.242727 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 158/318] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:31:27 2022] GET /identity/v3/role_assignments?user.id=e7a667368c014f5f8f2ab59ea49a5135&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=887708bc8dee457481db8046c4f52ea8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:28.513486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f6fc-4635-4d67-acfb-5d21d1b4d047 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:28.514059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f6fc-4635-4d67-acfb-5d21d1b4d047 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:28.514059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f6fc-4635-4d67-acfb-5d21d1b4d047 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:28.515524 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 161/319] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:28.520685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89edfd-9756-4390-9024-40b30d2fad5d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:28.521303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89edfd-9756-4390-9024-40b30d2fad5d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:28.521303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89edfd-9756-4390-9024-40b30d2fad5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:28.539930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ed89edfd-9756-4390-9024-40b30d2fad5d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:28.541112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ed89edfd-9756-4390-9024-40b30d2fad5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:28.579365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c40d4-ee62-4b28-88d2-c4860b062d17 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:28.579674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c40d4-ee62-4b28-88d2-c4860b062d17 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:28.579921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c40d4-ee62-4b28-88d2-c4860b062d17 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:28.581366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 162/320] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:31:28.586256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f6ea5-5d30-422d-ac70-bae8c6a72378 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:28.586540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f6ea5-5d30-422d-ac70-bae8c6a72378 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:28.586811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f6ea5-5d30-422d-ac70-bae8c6a72378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:28.605541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-492f6ea5-5d30-422d-ac70-bae8c6a72378 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:28.606757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-492f6ea5-5d30-422d-ac70-bae8c6a72378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:28.612155 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 159/321] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:28.664736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:28.678287 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 163/322] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:28.950000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89edfd-9756-4390-9024-40b30d2fad5d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:29.002810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f6ea5-5d30-422d-ac70-bae8c6a72378 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:29.072878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89edfd-9756-4390-9024-40b30d2fad5d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:29.104457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:29.111958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:29.112442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f6ea5-5d30-422d-ac70-bae8c6a72378 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:29.112623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.112875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.113153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.114299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:29.115969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:29.117691 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 160/323] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:29.122903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:29.149432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:29.150088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.150366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.150639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.160900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:29.166821 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:29.167752 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:29.168566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 164/324] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:31:29.173445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:29.234026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:29.237682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586fe5d-7cc4-444d-bfa3-00e7e94dfe1b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:29.275315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f7268-e886-40f4-a932-db6a45dab1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:29.275918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f7268-e886-40f4-a932-db6a45dab1d7 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.276201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f7268-e886-40f4-a932-db6a45dab1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.276468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f7268-e886-40f4-a932-db6a45dab1d7 None admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.277088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f7268-e886-40f4-a932-db6a45dab1d7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:29.278566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f7268-e886-40f4-a932-db6a45dab1d7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:29.290865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c016ba4-eac0-4ade-9f6b-d6e71de453d8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:29.295197 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 161/325] 10.222.0.33 () {60 vars in 1134 bytes} [Sat Sep 10 20:31:29 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:31:29.303655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4e026-2b5b-461c-abda-a28b20bfe2ff None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.304030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4e026-2b5b-461c-abda-a28b20bfe2ff None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.304319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4e026-2b5b-461c-abda-a28b20bfe2ff None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.306021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 162/326] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:29.312027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312c9a1-7b19-4812-84b1-7811b83d24e6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.312365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312c9a1-7b19-4812-84b1-7811b83d24e6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.312585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312c9a1-7b19-4812-84b1-7811b83d24e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.330526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:29.331153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.331429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.331799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.332472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:29.332710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d312c9a1-7b19-4812-84b1-7811b83d24e6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:29.332807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:29.334304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d312c9a1-7b19-4812-84b1-7811b83d24e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:29.334430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:29.348471 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 165/327] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:29.350550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:29.400388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:29.421148 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 163/328] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:29.471024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d083eeb-f438-4ca5-8e60-6e6a1fb2d709 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:29.486359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:29.512105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:29.512747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.513028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.513306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] PATH_INFO: `/v3/users/427caed5c90247618e08749e1c486691` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.514529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] RBAC: Authorizing `identity:get_user(user_id=427caed5c90247618e08749e1c486691)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:29.515971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:29.517717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 166/329] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:29 2022] GET /identity/v3/users/427caed5c90247618e08749e1c486691 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:29.524289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312c9a1-7b19-4812-84b1-7811b83d24e6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:29.630293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312c9a1-7b19-4812-84b1-7811b83d24e6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:29.667099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:29.667997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.668432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.668864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] PATH_INFO: `/v3/system/users/427caed5c90247618e08749e1c486691/roles/887708bc8dee457481db8046c4f52ea8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.670487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=427caed5c90247618e08749e1c486691, role_id=887708bc8dee457481db8046c4f52ea8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:29.672007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:29.684098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 164/330] 10.222.0.33 () {58 vars in 1334 bytes} [Sat Sep 10 20:31:29 2022] PUT /identity/v3/system/users/427caed5c90247618e08749e1c486691/roles/887708bc8dee457481db8046c4f52ea8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:29.765386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:29.885789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f776a-ec2c-4ea8-a797-7da4800c3fcc None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:29.923964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:29.924598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:29.924875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:29.925146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:29.936940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:29.943029 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:29.943899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:29.944708 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:29.946204 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 167/331] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:29.950835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:30.007844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:30.061454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a673ef-5cbd-4d3a-b7df-e3360d0bbcfb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:30.097103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:30.097716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:30.098121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:30.098398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:30.099629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:30.099929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:30.101484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:30.110223 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 165/332] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:30.116336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:30.163590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:30.236444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d92643-6861-4d7a-ab29-b8be1a66dcc1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:30.277586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11b8d6-86a3-4916-b9af-7bf6c5437a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:30.278489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b8d6-86a3-4916-b9af-7bf6c5437a2e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:30.279019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b8d6-86a3-4916-b9af-7bf6c5437a2e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:30.279443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b8d6-86a3-4916-b9af-7bf6c5437a2e None admin] PATH_INFO: `/v3/users/383901f6540845b4bbf9233386be30ed` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:30.281125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11b8d6-86a3-4916-b9af-7bf6c5437a2e None admin] RBAC: Authorizing `identity:get_user(user_id=383901f6540845b4bbf9233386be30ed)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:30.282787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11b8d6-86a3-4916-b9af-7bf6c5437a2e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:30.284832 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 168/333] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:30 2022] GET /identity/v3/users/383901f6540845b4bbf9233386be30ed => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:30.290876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:30.420408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a828e24-db0d-4166-8cdd-a86faf0c7884 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:30.453852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:30.469020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a63f6-a3b3-4a4e-9686-40a598d7c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:30.469636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a63f6-a3b3-4a4e-9686-40a598d7c671 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:30.469976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a63f6-a3b3-4a4e-9686-40a598d7c671 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:30.470249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a63f6-a3b3-4a4e-9686-40a598d7c671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:30.470968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a63f6-a3b3-4a4e-9686-40a598d7c671 None admin] RBAC: Adding query filter params (role.id=9d599f0800724bd4b78e84df540784c8, user.id=383901f6540845b4bbf9233386be30ed, scope.system=all) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:30.471274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a63f6-a3b3-4a4e-9686-40a598d7c671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:30.472691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a63f6-a3b3-4a4e-9686-40a598d7c671 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:30.487947 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 166/334] 10.222.0.33 () {56 vars in 1308 bytes} [Sat Sep 10 20:31:30 2022] GET /identity/v3/role_assignments?user.id=383901f6540845b4bbf9233386be30ed&scope.system=all&role.id=9d599f0800724bd4b78e84df540784c8 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:32.250855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8fa89-b82c-41ab-89ab-3cbdf353a65b None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:32.251346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8fa89-b82c-41ab-89ab-3cbdf353a65b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:32.251346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8fa89-b82c-41ab-89ab-3cbdf353a65b None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:32.252885 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 169/335] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:32.259041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e9a00-c69d-4d62-a256-1b4e7848e619 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:32.259462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e9a00-c69d-4d62-a256-1b4e7848e619 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:32.259651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e9a00-c69d-4d62-a256-1b4e7848e619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:32.281718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1b6e9a00-c69d-4d62-a256-1b4e7848e619 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:32.282917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1b6e9a00-c69d-4d62-a256-1b4e7848e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:32.367141 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 167/336] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:32.918983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf62b29-7cd2-46a8-bcad-e2753d086bba None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:32.919377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf62b29-7cd2-46a8-bcad-e2753d086bba None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:32.919563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf62b29-7cd2-46a8-bcad-e2753d086bba None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:32.921099 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 170/337] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:32.926116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea08a4d-7d1a-4fd3-99af-134674548383 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:32.926404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea08a4d-7d1a-4fd3-99af-134674548383 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:32.926679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea08a4d-7d1a-4fd3-99af-134674548383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:32.955243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2ea08a4d-7d1a-4fd3-99af-134674548383 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:32.956418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2ea08a4d-7d1a-4fd3-99af-134674548383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:33.066299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 168/338] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:33.066486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eede1-3139-4ea7-a6f6-1b11584ef9e0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:33.066586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eede1-3139-4ea7-a6f6-1b11584ef9e0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:33.066843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eede1-3139-4ea7-a6f6-1b11584ef9e0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:33.069498 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 171/339] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:33.090380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307df537-7fa5-47dc-ba41-c6eff1d0b461 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:33.090776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307df537-7fa5-47dc-ba41-c6eff1d0b461 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:33.091149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307df537-7fa5-47dc-ba41-c6eff1d0b461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:33.095119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eede1-3139-4ea7-a6f6-1b11584ef9e0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:33.122916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-307df537-7fa5-47dc-ba41-c6eff1d0b461 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:33.124056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-307df537-7fa5-47dc-ba41-c6eff1d0b461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:33.173426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:33.299580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eede1-3139-4ea7-a6f6-1b11584ef9e0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:33.314194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:33.323448 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 169/340] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:33.346980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4431c-c195-40f8-9868-1fe5529d863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:33.347613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4431c-c195-40f8-9868-1fe5529d863c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:33.347891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4431c-c195-40f8-9868-1fe5529d863c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:33.348164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4431c-c195-40f8-9868-1fe5529d863c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:33.367722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4431c-c195-40f8-9868-1fe5529d863c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-91f4431c-c195-40f8-9868-1fe5529d863c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:33.372645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:33.373522 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:33.374363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:33.378185 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 172/341] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:31:33.380083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307df537-7fa5-47dc-ba41-c6eff1d0b461 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:33.537752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307df537-7fa5-47dc-ba41-c6eff1d0b461 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:33.556151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344137b5-c92a-4867-a162-4d2ba8596bfe None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:33.556444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344137b5-c92a-4867-a162-4d2ba8596bfe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:33.556703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344137b5-c92a-4867-a162-4d2ba8596bfe None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:33.558328 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 173/342] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:33.563198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a112d-e4d0-459c-874d-1c28c449fd8f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:33.563480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a112d-e4d0-459c-874d-1c28c449fd8f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:33.563755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a112d-e4d0-459c-874d-1c28c449fd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:33.602662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:33.603283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:33.603565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:33.603835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:33.604506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:33.604793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6b9a112d-e4d0-459c-874d-1c28c449fd8f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:33.604956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:33.606164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6b9a112d-e4d0-459c-874d-1c28c449fd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:33.606361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:33.620211 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 170/343] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:33.626573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:33.701664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:33.720170 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 174/344] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:33.721136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a112d-e4d0-459c-874d-1c28c449fd8f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:33.855776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cde042-e5da-42fc-9731-b54b713a9a38 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:33.919840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:33.921767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:33.922328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:33.923162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:33.924369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:33.930314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a112d-e4d0-459c-874d-1c28c449fd8f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.011822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:34.013294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:34.035099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.036135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.036670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.037331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.039561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:34.063500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:34.068710 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:34.069862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 20:31:34.070883 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:34.081850 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 175/345] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:34.081850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.129239 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 171/346] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:31:34.130416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.253772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b928e0-2118-49fa-8132-528b4b2d6d96 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.265168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4998dd3-8d0e-4f07-804b-6a1d2069742e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.287777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:34.302903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.304214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.304497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.304773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.305439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:34.305742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:34.306323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.307137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:34.307676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.308125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.308648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.317521 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 172/347] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:34.318373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.320553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:34.326007 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:34.326984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:34.327813 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:34.330475 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 176/348] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:34.337443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.363067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:34.450958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ed379-25c2-42a2-8c7a-9d52f81656ff None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.476576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801b46-c3c2-4bc5-be30-983042ad614f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.499800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.500439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.500722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.500997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.502707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:34.504460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:34.507128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 173/349] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:34.507923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.520245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.521302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.522056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.522548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.534283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:34.540521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:34.541436 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:34.542278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:34.543362 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 177/350] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:34.543825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.585803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:34.642285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:34.650924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6827d-89a6-4708-a7ea-8d8d10413495 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.675005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7542a03-cb55-457e-b5a8-be2d404029ec None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.695641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.696965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.697278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.697569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] PATH_INFO: `/v3/users/427caed5c90247618e08749e1c486691` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.699961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] RBAC: Authorizing `identity:get_user(user_id=427caed5c90247618e08749e1c486691)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:34.701452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:34.703712 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 174/351] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/users/427caed5c90247618e08749e1c486691 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:34.704496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.731363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afed4f-c082-4dc2-b68e-1c84c4836a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.732010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afed4f-c082-4dc2-b68e-1c84c4836a50 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.732290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afed4f-c082-4dc2-b68e-1c84c4836a50 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.732561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afed4f-c082-4dc2-b68e-1c84c4836a50 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.733268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afed4f-c082-4dc2-b68e-1c84c4836a50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:34.733569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afed4f-c082-4dc2-b68e-1c84c4836a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:34.735219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afed4f-c082-4dc2-b68e-1c84c4836a50 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:34.745923 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 178/352] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:34.747453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fd5e4-f807-442c-9cac-2ac3780d7894 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.747744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fd5e4-f807-442c-9cac-2ac3780d7894 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.748018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fd5e4-f807-442c-9cac-2ac3780d7894 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.749965 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 179/353] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:34.751579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fd5e4-f807-442c-9cac-2ac3780d7894 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.842823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20832fd-a6ef-41b4-8f4e-7c2b1c0608d9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.878468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:34.906181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fd5e4-f807-442c-9cac-2ac3780d7894 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:34.906412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.913543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.913543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.913543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.921223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:34.923568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:34.929949 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 175/354] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:34.930839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:34.956650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184db96b-bb2b-4f6a-89dd-8be5dac7b5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:34.957185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184db96b-bb2b-4f6a-89dd-8be5dac7b5fd None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.959170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184db96b-bb2b-4f6a-89dd-8be5dac7b5fd None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.959460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184db96b-bb2b-4f6a-89dd-8be5dac7b5fd None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.960214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184db96b-bb2b-4f6a-89dd-8be5dac7b5fd None admin] RBAC: Adding query filter params (name=member) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:34.960452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184db96b-bb2b-4f6a-89dd-8be5dac7b5fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:34.964624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184db96b-bb2b-4f6a-89dd-8be5dac7b5fd None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:34.984065 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 180/355] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:34.986301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf0035-24c4-4baa-a8b3-301bd6d0f8dc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:34.986897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf0035-24c4-4baa-a8b3-301bd6d0f8dc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:34.988689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf0035-24c4-4baa-a8b3-301bd6d0f8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:34.999001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:35.016388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a0cf0035-24c4-4baa-a8b3-301bd6d0f8dc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:35.017648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a0cf0035-24c4-4baa-a8b3-301bd6d0f8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:35.100666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775912-18ea-44d7-baa4-bc81ad3023d7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:35.121625 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 181/356] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:35.122533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf0035-24c4-4baa-a8b3-301bd6d0f8dc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:35.159704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:35.160679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:35.162020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:35.162752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:35.163701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] RBAC: Adding query filter params (role.id=887708bc8dee457481db8046c4f52ea8, user.id=427caed5c90247618e08749e1c486691, scope.system=all) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:35.164266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:35.166830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:35.186049 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 176/357] 10.222.0.33 () {56 vars in 1308 bytes} [Sat Sep 10 20:31:34 2022] GET /identity/v3/role_assignments?user.id=427caed5c90247618e08749e1c486691&scope.system=all&role.id=887708bc8dee457481db8046c4f52ea8 => generated 510 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:35.187299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:35.262182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf0035-24c4-4baa-a8b3-301bd6d0f8dc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:35.301507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:35.312895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:35.323770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a796558-e55f-4550-b96a-99e871d113ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:35.324780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a796558-e55f-4550-b96a-99e871d113ef None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:35.325293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a796558-e55f-4550-b96a-99e871d113ef None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:35.325786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a796558-e55f-4550-b96a-99e871d113ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:35.386058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721000fc-7386-4924-82ad-2bc1e8dfc540 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:35.450253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:35.450971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:35.451510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:35.451510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:35.452355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:35.452729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:35.454336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:35.462701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a796558-e55f-4550-b96a-99e871d113ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:35.463200 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 177/358] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:35 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 260 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:35.465201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a796558-e55f-4550-b96a-99e871d113ef None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:35.467263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:35.623217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 182/359] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:35.626348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35793017-f99c-44d5-b09d-c75e1919af05 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:35.626827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35793017-f99c-44d5-b09d-c75e1919af05 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:35.627481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35793017-f99c-44d5-b09d-c75e1919af05 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:35.631067 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 183/360] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:35.632241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35793017-f99c-44d5-b09d-c75e1919af05 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:35.650371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67f0e6-ef43-4222-88cf-85d2c451c30c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:35.660665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:35.680612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:35.703707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38debe-697e-4c59-bd65-a4d8f1e982e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:35.704430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38debe-697e-4c59-bd65-a4d8f1e982e1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:35.704779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38debe-697e-4c59-bd65-a4d8f1e982e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:35.705152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38debe-697e-4c59-bd65-a4d8f1e982e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:35.778335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35793017-f99c-44d5-b09d-c75e1919af05 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:35.794367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38debe-697e-4c59-bd65-a4d8f1e982e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:35.796227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38debe-697e-4c59-bd65-a4d8f1e982e1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:35.831376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:35.832011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:35.832311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:35.832576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:35.850766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:35.856994 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:35.867427 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 184/361] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:35.871729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:35.951374 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 178/362] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:35.951374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759466-69c1-4ba9-9ad5-8bcb990659a7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:35.951374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759466-69c1-4ba9-9ad5-8bcb990659a7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:35.951374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759466-69c1-4ba9-9ad5-8bcb990659a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:35.975356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-15759466-69c1-4ba9-9ad5-8bcb990659a7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:35.976976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-15759466-69c1-4ba9-9ad5-8bcb990659a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:35.979151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:35.990562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:36.008065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670716d7-11ed-4b2b-bb7e-25bd1d9085ef None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.062787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.063426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.063708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.063997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.065150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.066664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.068904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 185/363] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:35 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:36.071481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.073348 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 179/364] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:36.074628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15759466-69c1-4ba9-9ad5-8bcb990659a7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.185114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15759466-69c1-4ba9-9ad5-8bcb990659a7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.185316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82725dbe-2689-4dcb-8786-8f823aaa3d7f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.224913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.225583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.225906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.226040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.226351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.226521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.226858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.227140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.227733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.227843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:36.228102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.229406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.229526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.231088 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 180/365] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:36.231850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.240218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 186/366] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:36.241044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.241445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:36.270949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:36.363039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd7efd-a1a0-4e64-8c28-77228406375a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.365328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af1f76-7952-4d28-b667-d4c8cc5521b9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.405434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.406104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.406393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.406669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.407147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.407855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.407956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.408199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.408484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.409349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.411189 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 181/367] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:36.411548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.412055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.413145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.414902 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 187/368] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:36.418802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.518135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:36.535671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:36.549348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac09f-eed9-4541-a60c-c900c36f2f54 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.551454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed424c28-e9c4-4002-a2bf-87b9c85e7759 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.592796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.593145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.593399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.593682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.594466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:36.594779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.596978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.602846 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 188/369] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:36.603482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.603733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.604615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.605093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.605559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.706495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.708700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.751335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c79eb-bb97-498a-8ece-d12fe465253a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:36.805649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:36.806713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:36.807201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:36.807670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:36.808916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:36.810235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=fc3066e494914bca9d3624f63d34656c) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:36.810770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:36.813701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:36.823295 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 189/370] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:36.824482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:36.850830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:36.873461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 182/371] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:36.879755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:37.046304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b496ece-f319-44de-a7be-ae052eea09ec None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:37.060301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd48799-dfce-4fa9-a025-24587e6f8eea None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:37.092832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:37.093482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:37.093767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:37.094241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:37.095423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:37.097218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:37.099815 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 183/372] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:37.107107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:37.137797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:37.138453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:37.138731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:37.139000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:37.155796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:37.163232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:37.164164 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:37.164957 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 190/373] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:37.169190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:37.170549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:37.245430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03479b0c-08bb-4504-be4b-68c07d30155e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:37.286386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:37.310660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9364c5-c28c-4b80-95e6-5aa34e0ccdf3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:37.313913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:37.314576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:37.314864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:37.315144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:37.318796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:37.320280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:37.322228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 184/374] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:37 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:37.330787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:37.349429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:37.350236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:37.350518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:37.350864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:37.351448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:37.351749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:37.353127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:37.362790 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 191/375] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:37.379292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:37.489648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a3e71-ffe2-4abe-88c7-1ff9aee9cc49 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:37.506747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:37.530495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:37.531131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:37.531375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:37.531660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:37.532394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=fc3066e494914bca9d3624f63d34656c) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:37.532709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:37.534169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:37.541477 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 185/376] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:37 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:37.574587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944ad7-e69e-4c57-9e4b-63ffef7ec577 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:37.619848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:37.620475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:37.620760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:37.621039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:37.638118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:37.710580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:37.712011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:37.845140 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 192/377] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:31:37.852234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:37.990687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55e2ad-17b5-4f2f-9052-2f740f67dc53 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:38.026775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:38.027413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:38.027696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:38.027972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] PATH_INFO: `/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:38.029125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6cfd9ba910450d96d6133cd5225f2f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:38.030598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:38.032241 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 186/378] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:37 2022] GET /identity/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:38.037924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:38.108533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:38.191118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585440e-ca8b-4e74-8046-ac54d195d0fc None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:38.249210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4725be-9da4-48f6-a643-bcbd3b883c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:38.249854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4725be-9da4-48f6-a643-bcbd3b883c87 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:38.250140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4725be-9da4-48f6-a643-bcbd3b883c87 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:38.250411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4725be-9da4-48f6-a643-bcbd3b883c87 None admin] PATH_INFO: `/v3/groups/a84611346bf2435fb4e1688c7355effe` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:38.262403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4725be-9da4-48f6-a643-bcbd3b883c87 None admin] RBAC: Authorizing `identity:get_group(group_id=a84611346bf2435fb4e1688c7355effe)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:38.263874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4725be-9da4-48f6-a643-bcbd3b883c87 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:38.265677 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 193/379] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:38 2022] GET /identity/v3/groups/a84611346bf2435fb4e1688c7355effe => generated 229 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:38.272340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:38.326040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:38.396317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5744d2-62c8-4cf1-9da8-03226a19c337 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:38.433308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f835810-a633-434b-b0e7-7d4a0f979ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:38.434351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f835810-a633-434b-b0e7-7d4a0f979ee8 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:38.434833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f835810-a633-434b-b0e7-7d4a0f979ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:38.435327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f835810-a633-434b-b0e7-7d4a0f979ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:38.436259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f835810-a633-434b-b0e7-7d4a0f979ee8 None admin] RBAC: Adding query filter params (group.id=a84611346bf2435fb4e1688c7355effe, role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=ff6cfd9ba910450d96d6133cd5225f2f) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:38.436785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f835810-a633-434b-b0e7-7d4a0f979ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:38.438448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f835810-a633-434b-b0e7-7d4a0f979ee8 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:38.446204 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 187/380] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:38 2022] GET /identity/v3/role_assignments?group.id=a84611346bf2435fb4e1688c7355effe&scope.project.id=ff6cfd9ba910450d96d6133cd5225f2f&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:38.645116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6642-f2f8-4afa-a212-2b31c2234684 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:38.646208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6642-f2f8-4afa-a212-2b31c2234684 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:38.646282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6642-f2f8-4afa-a212-2b31c2234684 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:38.647900 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 194/381] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:38.654478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b40868-a233-4241-be2a-79aa2103551d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:38.655399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b40868-a233-4241-be2a-79aa2103551d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:38.655873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b40868-a233-4241-be2a-79aa2103551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:38.681857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-55b40868-a233-4241-be2a-79aa2103551d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:38.683579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-55b40868-a233-4241-be2a-79aa2103551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:38.771829 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 188/382] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:39.212712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6642-f2f8-4afa-a212-2b31c2234684 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:39.351952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6642-f2f8-4afa-a212-2b31c2234684 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:39.355437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:39.388584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:39.389571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:39.390195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:39.390676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:39.402790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:39.408820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:39.409803 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:39.410768 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:39.413276 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 195/383] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:39.419256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b40868-a233-4241-be2a-79aa2103551d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:39.463150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:39.533227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b40868-a233-4241-be2a-79aa2103551d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:39.572769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:39.573677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:39.574137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:39.574568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:39.575381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:39.575826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:39.577353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:39.587526 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 189/384] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:39.595254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:39.718791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8edd-8b4e-405e-96c5-7334437d3a53 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:39.763669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:39.764613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:39.765038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:39.765450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:39.766718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:39.768304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:39.770049 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 196/385] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:39 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:39.775973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:39.878350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:39.885629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e0bbe-2d81-4ee4-970e-88950e4c1d7c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:39.929897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e3c2-290b-43f8-8a96-47d870aba859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:39.930538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e3c2-290b-43f8-8a96-47d870aba859 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:39.930821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e3c2-290b-43f8-8a96-47d870aba859 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:39.931093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e3c2-290b-43f8-8a96-47d870aba859 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:39.932212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e3c2-290b-43f8-8a96-47d870aba859 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:39.933647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e3c2-290b-43f8-8a96-47d870aba859 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:39.935284 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 190/386] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:39 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:39.941080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:39.987794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:40.077948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ace3e5-fd59-4a95-939d-81e944472bb2 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.078298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ace3e5-fd59-4a95-939d-81e944472bb2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.078505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ace3e5-fd59-4a95-939d-81e944472bb2 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.078980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c4f02-19de-4794-9830-9ee3e4daa6f1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:40.080159 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 191/387] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:40.082255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59153b28-c908-48c9-9dd2-a2ead1152542 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.082546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59153b28-c908-48c9-9dd2-a2ead1152542 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.082821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59153b28-c908-48c9-9dd2-a2ead1152542 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.084735 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 192/388] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:40.090541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d0008-7e08-475e-a5dd-2594dad92a91 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.090828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d0008-7e08-475e-a5dd-2594dad92a91 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.091102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d0008-7e08-475e-a5dd-2594dad92a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.110086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5c6d0008-7e08-475e-a5dd-2594dad92a91 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:40.111278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5c6d0008-7e08-475e-a5dd-2594dad92a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:40.125702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816a593-27eb-40a5-8957-0a30af0e9623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:40.126638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816a593-27eb-40a5-8957-0a30af0e9623 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.127095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816a593-27eb-40a5-8957-0a30af0e9623 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.127571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816a593-27eb-40a5-8957-0a30af0e9623 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.129886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816a593-27eb-40a5-8957-0a30af0e9623 None admin] RBAC: Authorizing `identity:create_grant(project_id=62f33ff370254aa8a6e2b693a30a9b3c, group_id=79e88ae9c6914ec1b11ad004d11f29de, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:40.131761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816a593-27eb-40a5-8957-0a30af0e9623 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:40.154370 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 197/389] 10.222.0.33 () {58 vars in 1442 bytes} [Sat Sep 10 20:31:39 2022] PUT /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:40.155395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4002e7-da18-46c7-b1fa-af4861a63cac None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.155670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4002e7-da18-46c7-b1fa-af4861a63cac None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.155945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4002e7-da18-46c7-b1fa-af4861a63cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.193666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2a4002e7-da18-46c7-b1fa-af4861a63cac None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:40.197966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a4002e7-da18-46c7-b1fa-af4861a63cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:40.231311 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 193/390] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:40.301424 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 198/391] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:40.651687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d0008-7e08-475e-a5dd-2594dad92a91 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:40.696645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4002e7-da18-46c7-b1fa-af4861a63cac None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:40.730385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:40.773665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:40.780829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d0008-7e08-475e-a5dd-2594dad92a91 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:40.823680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469ca5a-5f7b-4949-9949-5a50ad14110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:40.824325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469ca5a-5f7b-4949-9949-5a50ad14110a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.824611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469ca5a-5f7b-4949-9949-5a50ad14110a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.824890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469ca5a-5f7b-4949-9949-5a50ad14110a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.832832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4002e7-da18-46c7-b1fa-af4861a63cac None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:40.843540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469ca5a-5f7b-4949-9949-5a50ad14110a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b469ca5a-5f7b-4949-9949-5a50ad14110a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:40.850210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:40.851118 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:40.851921 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:40.853277 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 194/392] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:40.855125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897548-eebd-4c3a-b8ad-fbc30a609fea None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.855407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897548-eebd-4c3a-b8ad-fbc30a609fea None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.855681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897548-eebd-4c3a-b8ad-fbc30a609fea None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.857219 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 195/393] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:40.857875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9897548-eebd-4c3a-b8ad-fbc30a609fea None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:40.883681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00325961-677d-4200-9533-e6cc1503d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:40.884316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00325961-677d-4200-9533-e6cc1503d3ea None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.884595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00325961-677d-4200-9533-e6cc1503d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.884864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00325961-677d-4200-9533-e6cc1503d3ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.900907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00325961-677d-4200-9533-e6cc1503d3ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-00325961-677d-4200-9533-e6cc1503d3ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:40.907180 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:40.908970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:40.910085 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:40.914418 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 199/394] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:40.915918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844c41-239e-4258-9158-9258d7e3cb6e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:40.916209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844c41-239e-4258-9158-9258d7e3cb6e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:40.916483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844c41-239e-4258-9158-9258d7e3cb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:40.941502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-36844c41-239e-4258-9158-9258d7e3cb6e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:40.943628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36844c41-239e-4258-9158-9258d7e3cb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:40.988205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9897548-eebd-4c3a-b8ad-fbc30a609fea None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.029887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.030502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.030783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.031115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.031784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:41.032143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:41.033520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:41.042981 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 196/395] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:41.043776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.047568 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 200/396] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:41.048505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844c41-239e-4258-9158-9258d7e3cb6e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.064918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:41.079883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:41.153861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e0780-b5b9-419a-a151-279784f8d71b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.189865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.190459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.190743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.191021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.191728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:41.192040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:41.193968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:41.202855 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 197/397] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:41.208346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.255852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844c41-239e-4258-9158-9258d7e3cb6e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.295991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:41.317155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dc4b2-f971-42b5-b6bb-8051258ac17f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.354788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.355815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.356336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.356810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.358179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:41.359968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:41.362099 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 198/398] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:41 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:41.362513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.363151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.363436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.363715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.365189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:41.368249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:41.368249 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 201/399] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:41 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:41.369706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.373772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.463661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:41.488149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9dab8-686a-4838-8685-d24362de1064 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.525262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17c915-674e-4e79-aaee-0758f61f99c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.526039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17c915-674e-4e79-aaee-0758f61f99c2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.526326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17c915-674e-4e79-aaee-0758f61f99c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.526600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17c915-674e-4e79-aaee-0758f61f99c2 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.527782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17c915-674e-4e79-aaee-0758f61f99c2 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:41.529432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17c915-674e-4e79-aaee-0758f61f99c2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:41.533866 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 202/400] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:41 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:41.533866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2f37-93d0-4836-9efa-817e2d1e5101 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.533866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2f37-93d0-4836-9efa-817e2d1e5101 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.533866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2f37-93d0-4836-9efa-817e2d1e5101 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.538160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 203/401] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:41.538749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b2f37-93d0-4836-9efa-817e2d1e5101 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.542022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285a630-3747-48ec-bb2c-74237cd4e938 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.577676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.578344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.578634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.578911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.580036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:41.581941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:41.589506 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 199/402] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:41 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:41.589506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.611030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:41.649597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b2f37-93d0-4836-9efa-817e2d1e5101 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.686963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338438a-322e-41be-a910-08e32f1caed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.687864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338438a-322e-41be-a910-08e32f1caed8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.688142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338438a-322e-41be-a910-08e32f1caed8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.688413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338438a-322e-41be-a910-08e32f1caed8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.700650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338438a-322e-41be-a910-08e32f1caed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:41.705455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0091d-cb49-40cd-95af-7829e102b3a7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-0338438a-322e-41be-a910-08e32f1caed8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:41.706215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:41.707286 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:41.708255 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 204/403] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:41.709250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582f492-bbb4-41f1-aa8e-8ea007239459 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.709250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582f492-bbb4-41f1-aa8e-8ea007239459 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.709250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582f492-bbb4-41f1-aa8e-8ea007239459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.727013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7582f492-bbb4-41f1-aa8e-8ea007239459 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:41.728119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7582f492-bbb4-41f1-aa8e-8ea007239459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:41.730122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:41.751133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:41.752287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:41.752743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:41.753018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:41.755609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3066e494914bca9d3624f63d34656c, group_id=79e88ae9c6914ec1b11ad004d11f29de, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:41.758013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:41.781852 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 200/404] 10.222.0.33 () {58 vars in 1442 bytes} [Sat Sep 10 20:31:41 2022] PUT /identity/v3/projects/fc3066e494914bca9d3624f63d34656c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:41.781852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.803639 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 205/405] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:41.804677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582f492-bbb4-41f1-aa8e-8ea007239459 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:41.955770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:42.007635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd9fd-fe90-4ec1-ba74-77107a1837dd None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.020130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582f492-bbb4-41f1-aa8e-8ea007239459 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.068707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.070151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.071067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.071891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.072571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:42.072880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:42.074301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:42.075302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.076453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.077410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.077896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.080540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:42.081570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] RBAC: Authorizing `identity:create_grant(project_id=62f33ff370254aa8a6e2b693a30a9b3c, group_id=79e88ae9c6914ec1b11ad004d11f29de, role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:42.083688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:42.085795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 206/406] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:42.091782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:42.104782 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 201/407] 10.222.0.33 () {58 vars in 1442 bytes} [Sat Sep 10 20:31:41 2022] PUT /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:42.196607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:42.241660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1f4d4-60ef-4e71-8759-eccff712eda3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.309470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.310135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.310416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.310685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.312574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:42.314075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:42.315731 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 207/408] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:42 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:42.321952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:42.390420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a0180-05ab-450a-80ba-cdd888f1e27a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.437099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:42.443398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.444178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.445205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.445205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.449488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:42.467248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:42.472619 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:42.473595 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:42.474797 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 202/409] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:31:42.484239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:42.507285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f8075-589f-4ca5-8e9a-f26c4c4de367 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.577987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.578612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.578890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.579166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.581101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:42.582580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:42.588285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 208/410] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:42 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:42.593722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:42.702976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13265e2-de88-42cd-bcdb-74ce4170d47e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.728344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5e9b9-3a4e-4716-8edd-ca6b22221af6 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.738872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.740482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.740971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.741443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.742353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:42.742857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:42.744457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:42.754216 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 203/411] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:42.760661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:42.783561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.784607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.785086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.785579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.788671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3066e494914bca9d3624f63d34656c, group_id=79e88ae9c6914ec1b11ad004d11f29de, role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:42.790387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:42.794104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:42.810462 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 209/412] 10.222.0.33 () {58 vars in 1442 bytes} [Sat Sep 10 20:31:42 2022] PUT /identity/v3/projects/fc3066e494914bca9d3624f63d34656c/groups/79e88ae9c6914ec1b11ad004d11f29de/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:31:42.853908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:42.918184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dda9d7-6103-4ba3-88ba-00716f6d4634 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:42.980812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:42.981807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:42.982330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:42.982801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] PATH_INFO: `/v3/groups/a84611346bf2435fb4e1688c7355effe` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:42.984316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] RBAC: Authorizing `identity:get_group(group_id=a84611346bf2435fb4e1688c7355effe)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:42.986015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:42.990567 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 204/413] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:42 2022] GET /identity/v3/groups/a84611346bf2435fb4e1688c7355effe => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:42.999372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:43.127439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633aebf0-5a35-4eed-a2be-93c72e3012b4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:43.165242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b08b8-8596-4fd6-8db6-ed3b6eca55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:43.166247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b08b8-8596-4fd6-8db6-ed3b6eca55e7 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:43.166729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b08b8-8596-4fd6-8db6-ed3b6eca55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:43.167221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b08b8-8596-4fd6-8db6-ed3b6eca55e7 None admin] PATH_INFO: `/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:43.168540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b08b8-8596-4fd6-8db6-ed3b6eca55e7 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6cfd9ba910450d96d6133cd5225f2f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:43.170248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b08b8-8596-4fd6-8db6-ed3b6eca55e7 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:43.172361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 210/414] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:42 2022] GET /identity/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:43.178438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:43.240077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d99c4-45ca-4ed5-8fe1-e968d7a1dadc None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:43.240660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d99c4-45ca-4ed5-8fe1-e968d7a1dadc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:43.241173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d99c4-45ca-4ed5-8fe1-e968d7a1dadc None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:43.247313 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 211/415] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:43.251344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577b7c8-3e0c-48e9-a108-49a63e94439e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:43.251811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577b7c8-3e0c-48e9-a108-49a63e94439e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:43.252255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577b7c8-3e0c-48e9-a108-49a63e94439e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:43.297784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f577b7c8-3e0c-48e9-a108-49a63e94439e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:43.299839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f577b7c8-3e0c-48e9-a108-49a63e94439e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:43.441535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b6d5d-ba39-4163-a45b-ed8f0927782e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:43.494143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:43.509097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:43.523545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:43.524562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:43.526173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:43.527287 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 212/416] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:43.534209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] PATH_INFO: `/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f/groups/a84611346bf2435fb4e1688c7355effe/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:43.537276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff6cfd9ba910450d96d6133cd5225f2f, group_id=a84611346bf2435fb4e1688c7355effe, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:43.539123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:43.560682 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 205/417] 10.222.0.33 () {58 vars in 1442 bytes} [Sat Sep 10 20:31:43 2022] PUT /identity/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f/groups/a84611346bf2435fb4e1688c7355effe/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:31:43.886272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577b7c8-3e0c-48e9-a108-49a63e94439e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:44.026159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577b7c8-3e0c-48e9-a108-49a63e94439e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:44.073703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:44.074357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:44.074683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:44.074956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:44.089896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:44.094674 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:44.095526 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:44.096315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:44.097349 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 213/418] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:44.102249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:44.246080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e48d0-b2be-4a45-9145-4b1ffdc3f862 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:44.306488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f677b2f-8c93-4b33-a7b8-53e0c83c7ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:44.307467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f677b2f-8c93-4b33-a7b8-53e0c83c7ac6 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:44.307944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f677b2f-8c93-4b33-a7b8-53e0c83c7ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:44.308427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f677b2f-8c93-4b33-a7b8-53e0c83c7ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:44.309327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f677b2f-8c93-4b33-a7b8-53e0c83c7ac6 None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:44.309854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f677b2f-8c93-4b33-a7b8-53e0c83c7ac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:44.311480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f677b2f-8c93-4b33-a7b8-53e0c83c7ac6 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:44.322247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 206/419] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:44.328289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:44.441462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:44.502364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff5c21-1736-42a0-a9fc-26ed331a4d71 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:44.556798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:44.557409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:44.557688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:44.557984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:44.658015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:44.659488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:44.708567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2b4b-f5b5-483d-97aa-bc836a5ec34f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:44.708979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2b4b-f5b5-483d-97aa-bc836a5ec34f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:44.709168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2b4b-f5b5-483d-97aa-bc836a5ec34f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:44.710713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 207/420] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:44.719289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84608-1c7f-41bd-a13a-4a81802761bc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:44.719580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84608-1c7f-41bd-a13a-4a81802761bc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:44.719861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84608-1c7f-41bd-a13a-4a81802761bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:44.730772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:44.763076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0ee84608-1c7f-41bd-a13a-4a81802761bc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:44.764203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ee84608-1c7f-41bd-a13a-4a81802761bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:44.799509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 214/421] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:44.803172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:44.847564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:44.932892 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 208/422] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:45.071884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99844021-6999-4e2d-abdb-f6e6349e4285 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:45.139627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:45.140269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:45.140549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:45.140823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:45.146563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:45.148038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:45.150027 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 215/423] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:44 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:45.159367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b748a-4fe6-4bd1-9fc6-368a5a5122d4 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:45.160250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b748a-4fe6-4bd1-9fc6-368a5a5122d4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:45.160749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b748a-4fe6-4bd1-9fc6-368a5a5122d4 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:45.161711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:45.163444 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 209/424] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:45.171176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465cca-6c30-4ddf-9f86-605ead26fe2a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:45.171801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465cca-6c30-4ddf-9f86-605ead26fe2a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:45.172299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465cca-6c30-4ddf-9f86-605ead26fe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:45.211322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:45.220048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ca465cca-6c30-4ddf-9f86-605ead26fe2a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:45.221251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ca465cca-6c30-4ddf-9f86-605ead26fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:45.362316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 210/425] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:45.374413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca465cca-6c30-4ddf-9f86-605ead26fe2a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:45.375762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37655958-b9de-4328-8785-7e51adbf1e5a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:45.462788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:45.463713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:45.463992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:45.464267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:45.467735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:45.471806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:45.471806 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 216/426] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:45 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:45.489789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:45.631817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca465cca-6c30-4ddf-9f86-605ead26fe2a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:45.642303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:45.689268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:45.689935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:45.690227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:45.690504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:45.706784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:45.715368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:45.720276 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:45.721158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:45.721975 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 211/427] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:45.727608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e04ef-1027-4290-97cc-6b695e4904d7 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:45.729804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:45.790349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:45.791080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:45.791374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:45.791634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:45.792367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:45.792676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:45.794306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:45.805372 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 217/428] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:45 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 611 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:45.836920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:45.948959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b504-29a3-4f19-a2f4-05da38d9e683 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.012758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89ea90-8f93-49b1-8286-22bc36d7705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.013786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89ea90-8f93-49b1-8286-22bc36d7705e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.014349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89ea90-8f93-49b1-8286-22bc36d7705e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.014894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89ea90-8f93-49b1-8286-22bc36d7705e None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.015642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89ea90-8f93-49b1-8286-22bc36d7705e None admin] RBAC: Adding query filter params (name=member) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:46.016063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89ea90-8f93-49b1-8286-22bc36d7705e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:46.017544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89ea90-8f93-49b1-8286-22bc36d7705e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:46.022764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f9f30-ea1a-441c-be52-9868b1e18794 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.027772 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 212/429] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:31:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:46.028933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e604-2d0f-40db-8188-f46624e1f190 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.029324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e604-2d0f-40db-8188-f46624e1f190 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.029723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e604-2d0f-40db-8188-f46624e1f190 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.036177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 213/430] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:46.036177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0e604-2d0f-40db-8188-f46624e1f190 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.067427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dde35-8145-464e-baec-b14054533460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.068095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dde35-8145-464e-baec-b14054533460 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.068373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dde35-8145-464e-baec-b14054533460 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.068649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dde35-8145-464e-baec-b14054533460 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.072534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.082345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dde35-8145-464e-baec-b14054533460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-575dde35-8145-464e-baec-b14054533460 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:46.088779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:46.089879 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:46.090712 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 218/431] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:46.091529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d56bb8-ebd3-4398-9d71-d228948b4924 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.091706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d56bb8-ebd3-4398-9d71-d228948b4924 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.091980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d56bb8-ebd3-4398-9d71-d228948b4924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.116841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-21d56bb8-ebd3-4398-9d71-d228948b4924 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:46.118248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21d56bb8-ebd3-4398-9d71-d228948b4924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:46.130838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.175693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0e604-2d0f-40db-8188-f46624e1f190 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.215883 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 219/432] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:46.216681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d56bb8-ebd3-4398-9d71-d228948b4924 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.218017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.218906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.219383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.219854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.303862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:46.305672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:46.335134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d56bb8-ebd3-4398-9d71-d228948b4924 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.347884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.382795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.383645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.384224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.384641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.385620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:46.386143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:46.387753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:46.397434 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 220/433] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:46.407039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.418801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.452688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 214/434] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:46.459553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.536440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e2d1e-08a0-478b-9786-6934be319d1a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.574019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d07009-4116-4558-a634-c762143019a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.574960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d07009-4116-4558-a634-c762143019a4 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.575576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d07009-4116-4558-a634-c762143019a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.576177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d07009-4116-4558-a634-c762143019a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.589845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22957805-585b-40ec-9251-959e65c31d9c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.630111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.636688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.638378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.638887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.639357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.640693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:46.642346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:46.644449 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 215/435] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:46.650068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.664297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d07009-4116-4558-a634-c762143019a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:46.665773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d07009-4116-4558-a634-c762143019a4 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:46.673553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.757194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5b1aa-2ace-4477-97c2-89ba30e01182 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.772772 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 221/436] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:46.777925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d07009-4116-4558-a634-c762143019a4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.792193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.792751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.793029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.793307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.794436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:46.795858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:46.797393 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 216/437] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:46.797393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.868543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.886958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d07009-4116-4558-a634-c762143019a4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.895090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:46.904553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2612c-9c3f-42a5-885f-57281a7b0149 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:46.921716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.922362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.922642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.922914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.933413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:46.938218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:46.939175 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:46.939964 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:46.940571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cd1a3-c7f3-4f37-b0b3-90ad71060410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:46.940571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cd1a3-c7f3-4f37-b0b3-90ad71060410 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.940571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cd1a3-c7f3-4f37-b0b3-90ad71060410 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.940772 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 222/438] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:46.940837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cd1a3-c7f3-4f37-b0b3-90ad71060410 None admin] PATH_INFO: `/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.940910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:46.943658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cd1a3-c7f3-4f37-b0b3-90ad71060410 None admin] RBAC: Authorizing `identity:get_project(project_id=62f33ff370254aa8a6e2b693a30a9b3c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:46.945298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cd1a3-c7f3-4f37-b0b3-90ad71060410 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:46.946946 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 217/439] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/projects/62f33ff370254aa8a6e2b693a30a9b3c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:46.950874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab50dc-5485-4cb9-b7bf-0c8c8de1bccd None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:46.951153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab50dc-5485-4cb9-b7bf-0c8c8de1bccd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:46.951844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab50dc-5485-4cb9-b7bf-0c8c8de1bccd None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:46.953056 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 218/440] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:46.954149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab50dc-5485-4cb9-b7bf-0c8c8de1bccd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.044799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e2e5-4428-4a65-bbc1-2d8531f30e56 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.058445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab50dc-5485-4cb9-b7bf-0c8c8de1bccd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.077773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.078408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.078688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.078959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.079666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=9d599f0800724bd4b78e84df540784c8, scope.project.id=fc3066e494914bca9d3624f63d34656c) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:47.079968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.081405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.087487 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 223/441] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=9d599f0800724bd4b78e84df540784c8 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:47.088507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.092365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352d432-1f58-4747-8d05-accd103675b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.093058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d432-1f58-4747-8d05-accd103675b6 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.093436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d432-1f58-4747-8d05-accd103675b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.093805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d432-1f58-4747-8d05-accd103675b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.094597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d432-1f58-4747-8d05-accd103675b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:47.094961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d432-1f58-4747-8d05-accd103675b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.096413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d432-1f58-4747-8d05-accd103675b6 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.098174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:47.113611 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 219/442] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:47.113611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe7c6f-0f35-4ca0-b853-37dc4f8e59e3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.113611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe7c6f-0f35-4ca0-b853-37dc4f8e59e3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.113611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe7c6f-0f35-4ca0-b853-37dc4f8e59e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.133002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-00fe7c6f-0f35-4ca0-b853-37dc4f8e59e3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:47.134259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00fe7c6f-0f35-4ca0-b853-37dc4f8e59e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:47.137565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:47.214317 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 220/443] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:47.215847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe7c6f-0f35-4ca0-b853-37dc4f8e59e3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.241336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d624423-5817-4e73-9acb-3668fc8e8860 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.283635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.284550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.284835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.285104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.286937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.288681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.291667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 224/444] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:47.298027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.340930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe7c6f-0f35-4ca0-b853-37dc4f8e59e3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.370212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:47.382013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.382314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.382591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.382860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.416536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:47.421633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15cfc8-1f11-41a0-a5bb-7b0ab5b9ea72 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.456984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.457282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.458531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.459044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.459217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.459370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.460045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:47.460350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.462472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.469004 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 225/445] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=62f33ff370254aa8a6e2b693a30a9b3c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:47.561504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.577627 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 221/446] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:47.584073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.617523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:47.685206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163625f9-7b78-42ed-af58-d4ca100f85bc None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.702432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7070d5a-c239-44e0-9d03-66381104e47d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.725012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.725497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.725780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.726100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.742818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:47.743755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.744392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.744675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.744948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] PATH_INFO: `/v3/projects/fc3066e494914bca9d3624f63d34656c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.746136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3066e494914bca9d3624f63d34656c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.748135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:47.748260 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:47.749122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:31:47.750040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:47.750657 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 222/447] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/projects/fc3066e494914bca9d3624f63d34656c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:47.750725 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 226/448] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:31:47.755336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.756543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.785677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:47.874866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:47.883509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee1454-4b19-4bad-88ca-9f1354fa4ac3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.887552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6bf61-c1c2-4840-917a-e3eefd555807 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:47.924627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.925238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.925929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.925929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.926548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:47.926846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.928218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.928316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:47.928889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:47.929621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:47.929621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] PATH_INFO: `/v3/groups/79e88ae9c6914ec1b11ad004d11f29de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:47.930595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] RBAC: Authorizing `identity:get_group(group_id=79e88ae9c6914ec1b11ad004d11f29de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:47.932327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:47.933591 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 223/449] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/groups/79e88ae9c6914ec1b11ad004d11f29de => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:31:47.937978 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 227/450] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:47.940140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:47.943751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:48.048208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:48.063368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed188511-2c58-4504-a9dd-1a4ac55d4b8f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:48.068942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2b24e-0c6c-47e5-841b-80f50e81cc3a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:48.098640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:48.099449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:48.099728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:48.100004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:48.100715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] RBAC: Adding query filter params (group.id=79e88ae9c6914ec1b11ad004d11f29de, role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=fc3066e494914bca9d3624f63d34656c) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:48.101020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:48.102482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:48.102960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:48.103559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:48.103835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:48.104102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:48.108089 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 224/451] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/role_assignments?group.id=79e88ae9c6914ec1b11ad004d11f29de&scope.project.id=fc3066e494914bca9d3624f63d34656c&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:48.180942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:48.182394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:48.274480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:48.292289 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 228/452] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:48.298397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:48.329477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:48.405354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814283af-002a-4b72-85c5-2aa4b45848c5 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:48.441287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:48.441863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:48.442154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:48.442431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] PATH_INFO: `/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:48.443549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] RBAC: Authorizing `identity:get_project(project_id=ff6cfd9ba910450d96d6133cd5225f2f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:48.445082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:48.446904 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 225/453] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:31:48 2022] GET /identity/v3/projects/ff6cfd9ba910450d96d6133cd5225f2f => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:48.452329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:48.552033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698272-dd26-47fb-b0e5-a389e81fd255 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:48.584709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a743a-9a61-4a64-ab0d-91ad40cd6c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:48.585323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a743a-9a61-4a64-ab0d-91ad40cd6c30 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:48.585602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a743a-9a61-4a64-ab0d-91ad40cd6c30 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:48.585898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a743a-9a61-4a64-ab0d-91ad40cd6c30 None admin] PATH_INFO: `/v3/groups/a84611346bf2435fb4e1688c7355effe` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:48.586987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a743a-9a61-4a64-ab0d-91ad40cd6c30 None admin] RBAC: Authorizing `identity:get_group(group_id=a84611346bf2435fb4e1688c7355effe)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:48.588721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a743a-9a61-4a64-ab0d-91ad40cd6c30 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:48.590292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 229/454] 10.222.0.33 () {56 vars in 1180 bytes} [Sat Sep 10 20:31:48 2022] GET /identity/v3/groups/a84611346bf2435fb4e1688c7355effe => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:48.596194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:48.688373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:48.695419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475c78b-42fa-4bb3-aa94-4023078b5c18 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:48.727957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ba106-f69a-405e-bc8a-dc089cc9fa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:48.728556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba106-f69a-405e-bc8a-dc089cc9fa5e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:48.728840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba106-f69a-405e-bc8a-dc089cc9fa5e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:48.729118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ba106-f69a-405e-bc8a-dc089cc9fa5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:48.729857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ba106-f69a-405e-bc8a-dc089cc9fa5e None admin] RBAC: Adding query filter params (group.id=a84611346bf2435fb4e1688c7355effe, role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=ff6cfd9ba910450d96d6133cd5225f2f) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:48.730186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ba106-f69a-405e-bc8a-dc089cc9fa5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:48.731618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ba106-f69a-405e-bc8a-dc089cc9fa5e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:48.737180 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 226/455] 10.222.0.33 () {56 vars in 1376 bytes} [Sat Sep 10 20:31:48 2022] GET /identity/v3/role_assignments?group.id=a84611346bf2435fb4e1688c7355effe&scope.project.id=ff6cfd9ba910450d96d6133cd5225f2f&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:51.649074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b191c-0ff2-4b79-a5de-bc3a72993332 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:51.649655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b191c-0ff2-4b79-a5de-bc3a72993332 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:51.649970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b191c-0ff2-4b79-a5de-bc3a72993332 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:51.651853 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 230/456] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:51.657529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55fa3-4903-4ae5-8449-23268d78eeeb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:51.658376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55fa3-4903-4ae5-8449-23268d78eeeb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:51.658864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55fa3-4903-4ae5-8449-23268d78eeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:51.678214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dde55fa3-4903-4ae5-8449-23268d78eeeb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:51.679867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dde55fa3-4903-4ae5-8449-23268d78eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:51.764004 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 227/457] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:51.790584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa821-27e5-4b6b-bf47-e271769451f2 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:51.791211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa821-27e5-4b6b-bf47-e271769451f2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:51.791679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa821-27e5-4b6b-bf47-e271769451f2 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:51.793181 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 231/458] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:51.798804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05177f84-5ba1-4a72-b612-8f68939c4ce2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:51.799305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05177f84-5ba1-4a72-b612-8f68939c4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:51.799768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05177f84-5ba1-4a72-b612-8f68939c4ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:51.827094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-05177f84-5ba1-4a72-b612-8f68939c4ce2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:51.828748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-05177f84-5ba1-4a72-b612-8f68939c4ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:51.922339 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 228/459] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:52.224388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faa821-27e5-4b6b-bf47-e271769451f2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:52.288997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:52.353509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faa821-27e5-4b6b-bf47-e271769451f2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:52.373335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0b652-ba31-4010-b290-78a51a8e2e2c None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.373637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0b652-ba31-4010-b290-78a51a8e2e2c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.373957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0b652-ba31-4010-b290-78a51a8e2e2c None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.375400 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 229/460] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:52.381883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df041f-b35c-41e0-8ad6-15213fcce4c6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.382167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df041f-b35c-41e0-8ad6-15213fcce4c6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.382438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df041f-b35c-41e0-8ad6-15213fcce4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.394655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:52.396197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.396694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.397166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.408147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:52.409874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:52.421177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e6df041f-b35c-41e0-8ad6-15213fcce4c6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:52.422398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e6df041f-b35c-41e0-8ad6-15213fcce4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:52.426408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:52.427484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:52.428346 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:52.428346 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:52.428346 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:52.430091 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 232/461] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:52.439534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:52.446163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:52.519442 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 230/462] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:31:52.521266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df041f-b35c-41e0-8ad6-15213fcce4c6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:52.565004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a4b5-b913-4ba0-af1c-58efcc27055e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:52.569568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:52.607242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f504028-0aec-4fa6-bbe4-f8e81a61983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:52.608282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504028-0aec-4fa6-bbe4-f8e81a61983f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.608860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504028-0aec-4fa6-bbe4-f8e81a61983f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.609429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504028-0aec-4fa6-bbe4-f8e81a61983f None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.610337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f504028-0aec-4fa6-bbe4-f8e81a61983f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:52.610875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f504028-0aec-4fa6-bbe4-f8e81a61983f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:52.612573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f504028-0aec-4fa6-bbe4-f8e81a61983f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:52.633089 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 233/463] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:52.634840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c6732-c579-4aec-a54c-a64fcdc23b82 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.635371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c6732-c579-4aec-a54c-a64fcdc23b82 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.635853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c6732-c579-4aec-a54c-a64fcdc23b82 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.637534 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 234/464] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:52.638771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c6732-c579-4aec-a54c-a64fcdc23b82 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:52.653015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df041f-b35c-41e0-8ad6-15213fcce4c6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:52.697090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e6777-dfa2-4bad-80db-65e0ef021928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:52.697734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e6777-dfa2-4bad-80db-65e0ef021928 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.698565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e6777-dfa2-4bad-80db-65e0ef021928 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.698849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e6777-dfa2-4bad-80db-65e0ef021928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.713725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e6777-dfa2-4bad-80db-65e0ef021928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:52.715561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e6777-dfa2-4bad-80db-65e0ef021928 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:52.727754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a59e6777-dfa2-4bad-80db-65e0ef021928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:52.737722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:52.738675 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:52.740350 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:52.740350 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:52.740350 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:52.747796 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 231/465] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:52.747796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03f654-4d1e-4075-b4cc-ddbf57e8bf7c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.747796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03f654-4d1e-4075-b4cc-ddbf57e8bf7c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.747796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03f654-4d1e-4075-b4cc-ddbf57e8bf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.761208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c6732-c579-4aec-a54c-a64fcdc23b82 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:52.769626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5f03f654-4d1e-4075-b4cc-ddbf57e8bf7c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:52.771763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f03f654-4d1e-4075-b4cc-ddbf57e8bf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:52.806795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:52.807787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:52.808215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:52.809377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:52.810267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:52.811862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:52.840409 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 235/466] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:52 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:31:52.842212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:52.848257 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 232/467] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:52.865764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03f654-4d1e-4075-b4cc-ddbf57e8bf7c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:52.873051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:52.981486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb8291-43d6-483a-929f-7963a6cea704 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.030040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.031042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.031487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.031903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.032759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:53.033205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.034930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.050445 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 236/468] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:53.057735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:53.158905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03f654-4d1e-4075-b4cc-ddbf57e8bf7c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.175866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:53.205077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d92d74-e727-4691-ae33-8bea9966b4b5 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.208508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:53.219530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.219530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.219755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.219984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.231866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.233321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.249447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd32962-a779-4629-875e-978a127896aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.250396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd32962-a779-4629-875e-978a127896aa None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:53.250731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:53.251615 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:53.252493 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:53.252493 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:53.252493 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:53.252493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 233/469] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:53.252714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:53.253399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd32962-a779-4629-875e-978a127896aa None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.253939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd32962-a779-4629-875e-978a127896aa None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.255360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd32962-a779-4629-875e-978a127896aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.257498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd32962-a779-4629-875e-978a127896aa None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.288408 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 237/470] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:53 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:31:53.289632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd32962-a779-4629-875e-978a127896aa None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:53.402782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685233-d43c-4c71-ac1a-603a2e75be6f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.435034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd32962-a779-4629-875e-978a127896aa None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.455464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f99e0-45e4-4884-8820-ca59139f1466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.456443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f99e0-45e4-4884-8820-ca59139f1466 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.456898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f99e0-45e4-4884-8820-ca59139f1466 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.457345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f99e0-45e4-4884-8820-ca59139f1466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.475830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f99e0-45e4-4884-8820-ca59139f1466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.477541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f99e0-45e4-4884-8820-ca59139f1466 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.483039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b73464-37f2-42ff-b6d0-036777df8f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.483979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73464-37f2-42ff-b6d0-036777df8f3b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.484481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73464-37f2-42ff-b6d0-036777df8f3b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.484959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73464-37f2-42ff-b6d0-036777df8f3b None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.485848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73464-37f2-42ff-b6d0-036777df8f3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:53.486356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73464-37f2-42ff-b6d0-036777df8f3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.487974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73464-37f2-42ff-b6d0-036777df8f3b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-961f99e0-45e4-4884-8820-ca59139f1466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:53.496069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:53.497028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:53.497909 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:53.497909 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:53.497909 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:53.497909 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 234/471] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:53.498840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f725f6-ec91-475a-aa75-79c1b093fc7d None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.499127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f725f6-ec91-475a-aa75-79c1b093fc7d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.499416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f725f6-ec91-475a-aa75-79c1b093fc7d None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.502232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:53.505926 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 235/472] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:53.507507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f725f6-ec91-475a-aa75-79c1b093fc7d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:53.510232 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 238/473] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:53.511813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50631ec-1e79-4162-8ca1-e0c7ae7d3583 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.512129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50631ec-1e79-4162-8ca1-e0c7ae7d3583 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.513395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50631ec-1e79-4162-8ca1-e0c7ae7d3583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.539778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f50631ec-1e79-4162-8ca1-e0c7ae7d3583 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:53.541405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f50631ec-1e79-4162-8ca1-e0c7ae7d3583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:53.554464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:53.633045 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 239/474] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:53.634079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50631ec-1e79-4162-8ca1-e0c7ae7d3583 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:53.649869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f725f6-ec91-475a-aa75-79c1b093fc7d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.693461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.694506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.695228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.696073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.696914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:53.697411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.700096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.715755 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 236/475] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:53.720826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:53.765412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50631ec-1e79-4162-8ca1-e0c7ae7d3583 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.808144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:53.827774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.828393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.828675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.828950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.829597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.831063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.841423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:53.850867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab30d3-77bd-4007-ba7f-0d580bb272a0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:53.873717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 240/476] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:53 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:31:53.911263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:53.912083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:53.912362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:53.912640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:53.913376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:53.914877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:53.946045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 237/477] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:53 2022] POST /identity/v3/users => generated 249 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:31:54.037525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:54.183149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be4fa-ee3b-4cad-aa9a-63996de4fece None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:54.227739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:54.228349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:54.228635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:54.228914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:54.229586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:54.231039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:54.241277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.sql.core [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 20:31:54.241277 ncdv-master-1 devstack@keystone.service[93528]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 20:31:54.241277 ncdv-master-1 devstack@keystone.service[93528]: [parameters: {'id': '385ab7dc033a4494b8ca7e1570ac2a8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 20:31:54.241277 ncdv-master-1 devstack@keystone.service[93528]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93528) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 20:31:54.259199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return ref.to_dict() Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:31:54.260098 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application yield resource Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise self.value Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application session.commit() Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 20:31:54.261002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.flush() Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application compat.raise_( Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 20:31:54.261895 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:31:54.262837 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:31:54.263659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [parameters: {'id': '385ab7dc033a4494b8ca7e1570ac2a8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:54.264430 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 20:31:54.265226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:54.268552 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 241/478] 10.222.0.33 () {60 vars in 1131 bytes} [Sat Sep 10 20:31:54 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:54.268685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:54.388084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca28121-3693-44b4-807d-42e0294ca6cb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:54.425221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ab67e-3a27-4178-965c-764918d40355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:54.425955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab67e-3a27-4178-965c-764918d40355 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:54.426845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab67e-3a27-4178-965c-764918d40355 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:54.427095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab67e-3a27-4178-965c-764918d40355 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:54.442891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ab67e-3a27-4178-965c-764918d40355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-9a3ab67e-3a27-4178-965c-764918d40355 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:54.448929 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:54.449958 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:54.451008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 238/479] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:31:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:54.456198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:54.538382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:54.609849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076d22-35cb-497a-8634-c15b243f6ab7 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:54.654520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516a202-53f6-46cf-8f72-c510ef1e8790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:54.655135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516a202-53f6-46cf-8f72-c510ef1e8790 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:54.655417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516a202-53f6-46cf-8f72-c510ef1e8790 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:54.655691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516a202-53f6-46cf-8f72-c510ef1e8790 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:54.656359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516a202-53f6-46cf-8f72-c510ef1e8790 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:54.656658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516a202-53f6-46cf-8f72-c510ef1e8790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:54.658074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516a202-53f6-46cf-8f72-c510ef1e8790 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:54.668138 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 242/480] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:31:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:55.991245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4261c-f1ce-44df-bdd5-b86c13122e58 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:55.991682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4261c-f1ce-44df-bdd5-b86c13122e58 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:55.991886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4261c-f1ce-44df-bdd5-b86c13122e58 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:55.993430 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 239/481] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:56.009160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f8501-b731-4f63-a40e-0374f850a717 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:56.010143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f8501-b731-4f63-a40e-0374f850a717 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:56.010611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f8501-b731-4f63-a40e-0374f850a717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:56.038971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1f9f8501-b731-4f63-a40e-0374f850a717 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:56.040225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1f9f8501-b731-4f63-a40e-0374f850a717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:56.132442 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 243/482] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:56.477595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c4166-aef0-4033-94db-65559f894129 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:56.478013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c4166-aef0-4033-94db-65559f894129 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:56.478203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c4166-aef0-4033-94db-65559f894129 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:56.479725 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 240/483] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:56.485524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8febe0-236e-43c9-81b0-380969bc4953 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:56.485904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8febe0-236e-43c9-81b0-380969bc4953 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:56.486259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8febe0-236e-43c9-81b0-380969bc4953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:56.515121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8e8febe0-236e-43c9-81b0-380969bc4953 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:56.516370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e8febe0-236e-43c9-81b0-380969bc4953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:56.558695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:56.611289 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 244/484] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:56.634914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c4166-aef0-4033-94db-65559f894129 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:56.648999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:56.766006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c4166-aef0-4033-94db-65559f894129 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:56.814593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d65937-36c1-4f46-a320-3c9407865e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:56.815225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d65937-36c1-4f46-a320-3c9407865e8b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:56.815509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d65937-36c1-4f46-a320-3c9407865e8b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:56.815783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d65937-36c1-4f46-a320-3c9407865e8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:56.828470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d65937-36c1-4f46-a320-3c9407865e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a2d65937-36c1-4f46-a320-3c9407865e8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:56.834016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:56.834912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:56.835704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 241/485] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:31:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:56.841196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8febe0-236e-43c9-81b0-380969bc4953 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.021237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8febe0-236e-43c9-81b0-380969bc4953 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.052116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a207-01ab-41bd-9210-b429ef727b05 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.052834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a207-01ab-41bd-9210-b429ef727b05 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.052834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a207-01ab-41bd-9210-b429ef727b05 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.054452 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 242/486] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:57.061462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144eaa-03e9-46a4-b5b1-0085488fcafc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.061750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144eaa-03e9-46a4-b5b1-0085488fcafc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.062053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144eaa-03e9-46a4-b5b1-0085488fcafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.069346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.070152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.070560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.070966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.071756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:57.072212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:57.074229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:57.083745 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 245/487] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:31:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:57.088413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.107113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dd144eaa-03e9-46a4-b5b1-0085488fcafc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:57.108338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dd144eaa-03e9-46a4-b5b1-0085488fcafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:57.141735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:57.178871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:57.185208 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 243/488] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:57.193014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd144eaa-03e9-46a4-b5b1-0085488fcafc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.262664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088f48e-bbe8-46e4-8de9-7113797689e9 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.304613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9532b-b1dc-4873-8823-e126ae08634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.305255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9532b-b1dc-4873-8823-e126ae08634b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.305537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9532b-b1dc-4873-8823-e126ae08634b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.305808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9532b-b1dc-4873-8823-e126ae08634b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.317125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9532b-b1dc-4873-8823-e126ae08634b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:57.318633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9532b-b1dc-4873-8823-e126ae08634b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ade9532b-b1dc-4873-8823-e126ae08634b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:57.333754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:57.334737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:57.335629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:57.335629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:57.335629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:57.335629 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 246/489] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:31:57.340054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624bfe6-90c4-4da2-986c-6a86947701c6 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.340054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624bfe6-90c4-4da2-986c-6a86947701c6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.340054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624bfe6-90c4-4da2-986c-6a86947701c6 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.342348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd144eaa-03e9-46a4-b5b1-0085488fcafc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.342990 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 247/490] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:57.343762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624bfe6-90c4-4da2-986c-6a86947701c6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.391463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68309f0e-1bab-4642-bc6c-ce5ee612b199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.392367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309f0e-1bab-4642-bc6c-ce5ee612b199 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.392644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309f0e-1bab-4642-bc6c-ce5ee612b199 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.392913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309f0e-1bab-4642-bc6c-ce5ee612b199 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.407953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68309f0e-1bab-4642-bc6c-ce5ee612b199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-68309f0e-1bab-4642-bc6c-ce5ee612b199 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:57.415250 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:57.416716 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:57.418391 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 244/491] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:57.422187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586d4c9-c5c5-4d9a-a89e-6dd408b7e70e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.422960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586d4c9-c5c5-4d9a-a89e-6dd408b7e70e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.423247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586d4c9-c5c5-4d9a-a89e-6dd408b7e70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.442715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d586d4c9-c5c5-4d9a-a89e-6dd408b7e70e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:57.443967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d586d4c9-c5c5-4d9a-a89e-6dd408b7e70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:57.445344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:57.474404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624bfe6-90c4-4da2-986c-6a86947701c6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.511763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:57.527469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.528287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.528641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.528990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.529720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:57.530244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:57.530656 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 245/492] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:57.531737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586d4c9-c5c5-4d9a-a89e-6dd408b7e70e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.532952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:57.559391 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 248/493] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:57.560152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.666269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586d4c9-c5c5-4d9a-a89e-6dd408b7e70e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.673375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdff4e-659d-4a42-a528-9ae38580591f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.711518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.712141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.712417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.712686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.719735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.720671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.721328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.722234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.723629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:57.724156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:57.725659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:57.726194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:57.730625 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:57.731519 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:57.732343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:57.736722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 249/494] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:57.737613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.738530 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 246/495] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:57.740232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.741885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:57.816022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:57.851261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c8852-c6c9-4ad5-a7b8-a363388ea86c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.881950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6197c-664e-4789-bfd0-403ae9461c82 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:57.890279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.890918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.891198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.891836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.900186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:57.901654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:31:57.916267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:57.917212 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:57.918152 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 250/496] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:57.918847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.924514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:57.924514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:57.924514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:57.924514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:57.924852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:57.924852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:57.924852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:57.935759 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 247/497] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:57.935759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:57.989223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.027653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a7b3e-269d-4212-85e5-8821441a79e3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.037638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a4f5b-b2e6-4adc-a1b5-20e392b405c5 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.050777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.060130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.060750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.061026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.061297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.072186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.072293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d28eb6-5e35-4c09-8f0a-03318e03d8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.072801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28eb6-5e35-4c09-8f0a-03318e03d8e4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.073067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28eb6-5e35-4c09-8f0a-03318e03d8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.073341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28eb6-5e35-4c09-8f0a-03318e03d8e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.073582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.084267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d28eb6-5e35-4c09-8f0a-03318e03d8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:58.087861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:58.089113 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:58.090035 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:58.090035 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:58.090035 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:58.090035 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 251/498] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-58d28eb6-5e35-4c09-8f0a-03318e03d8e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:31:58.090256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:58.091053 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:31:58.091842 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:58.092449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.092521 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 248/499] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:58.092521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f07909-6311-49f1-9f05-4c45eb61000b None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.092653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f07909-6311-49f1-9f05-4c45eb61000b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.093749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f07909-6311-49f1-9f05-4c45eb61000b None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.094218 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 249/500] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:31:58.097972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f07909-6311-49f1-9f05-4c45eb61000b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.195850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e4229-ed1c-4b9d-8def-c3f950545f30 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.198400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f07909-6311-49f1-9f05-4c45eb61000b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.209620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.229768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.230391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.230668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.230943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.231661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:58.231962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.232136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.232754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.233037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.233309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.233407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.244464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.245941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:58.260223 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:58.261111 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:58.262399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:58.262399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:58.262399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:58.262399 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 250/501] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:58.263114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.273884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 252/502] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:58.273884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.296410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.369716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5933b-7d7b-4a29-804c-9007bf90c17b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.370176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d38790-5198-42e3-814d-fc63ed31e5ad None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.403607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86113b-b9cd-47d5-997c-5e8b81139b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.404044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.404209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86113b-b9cd-47d5-997c-5e8b81139b14 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.404492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86113b-b9cd-47d5-997c-5e8b81139b14 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.404636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.404768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86113b-b9cd-47d5-997c-5e8b81139b14 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.404914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.405185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.405433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86113b-b9cd-47d5-997c-5e8b81139b14 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:58.405737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86113b-b9cd-47d5-997c-5e8b81139b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.405854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:58.406130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.407141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86113b-b9cd-47d5-997c-5e8b81139b14 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.407465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.415530 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 253/503] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:58.416681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061043f-113b-43a3-884d-8e75f88844b6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.416970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061043f-113b-43a3-884d-8e75f88844b6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.417248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061043f-113b-43a3-884d-8e75f88844b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.423404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 251/504] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:58.424207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.434895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8061043f-113b-43a3-884d-8e75f88844b6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:31:58.436087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8061043f-113b-43a3-884d-8e75f88844b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:31:58.462295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.503682 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 254/505] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:31:58.505113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061043f-113b-43a3-884d-8e75f88844b6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.512411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.529107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e04b7b-7e9a-46be-b0ff-aa5d9e6360f0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.564144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.564743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.565024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.565299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.565964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:58.566283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.567644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.578985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 252/506] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:58.579779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.620894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061043f-113b-43a3-884d-8e75f88844b6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.655603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.656202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.656483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.656752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.687854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa378848-8502-4336-97dc-388d9eaf87d9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.688428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.723881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36126209-9375-4214-914a-5089f632eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.724499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126209-9375-4214-914a-5089f632eecb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.724779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126209-9375-4214-914a-5089f632eecb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.725047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126209-9375-4214-914a-5089f632eecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.728496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.730111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.737127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36126209-9375-4214-914a-5089f632eecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.739164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36126209-9375-4214-914a-5089f632eecb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.745613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-36126209-9375-4214-914a-5089f632eecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:58.754137 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:58.755448 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:58.756306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:58.756306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:58.756306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:58.756306 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 253/507] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:58.756306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36126209-9375-4214-914a-5089f632eecb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.845071 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 255/508] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:58.845071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.861129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36126209-9375-4214-914a-5089f632eecb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.897329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.898188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.898462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.898729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.904057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.905845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.907657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:31:58.921529 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:58.922669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:31:58.923799 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:58.926147 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 254/509] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:58.926901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:58.943431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f1b4f-5ff5-46f7-822e-c71965d6b45d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:58.980650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:58.981282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:58.981565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:58.981849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:58.985144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:58.989462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:58.990946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:31:59.002431 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:59.003523 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 256/510] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:59.004714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.030265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ff3a7-0e53-4cbe-bb59-350f0b7bdf63 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.063577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.064246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.064527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.064824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.065447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:59.065745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.067418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.079280 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 255/511] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:59.079979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.107355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17989bb6-f678-4a44-835b-f4b025c89d1b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.130933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.142462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.143086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.143365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.143638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.154455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.155860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:59.170403 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:59.171501 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:59.172853 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:59.172853 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:59.172853 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:59.172853 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 257/512] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:59.172853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.184766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a10466-473b-43a0-a055-6959ac23d8f2 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.203471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.221474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.222141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.222421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.222689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.233325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.234872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:31:59.250880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:59.251970 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:31:59.253016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:31:59.253016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:31:59.253016 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:31:59.256764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 256/513] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:59.257627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.288190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d010-bc55-43ae-84c6-c25a9abca564 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.321861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.322474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.322761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.323032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.323758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:59.324066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.325503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.342535 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 258/514] 10.222.0.33 () {56 vars in 1137 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:59.343329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.365964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c4fd-38c7-4f68-b526-dbbf96653b9e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.376147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.399507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.400102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.400387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.400661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.401385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:59.401690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.403174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.439490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 257/515] 10.222.0.33 () {56 vars in 1137 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:59.440244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.447821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efaebd-0dd0-4ea7-9f3e-d88105ef5a0d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.468618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.482074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.482701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.482981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.483253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.490854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.492621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:31:59.505262 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:31:59.506618 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:31:59.507690 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 259/516] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:31:59.509912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.542718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd768047-2585-4078-a905-fa2b9e1656e0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.576016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.576618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.576898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.577170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.577804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:59.578138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.579521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.590748 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 258/517] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:59.591539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.602502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.609203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaac1cb-0e64-413a-a5a7-5edb37673dd4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.642925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.643527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.643807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.644081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.644720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:31:59.645257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.646668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.658044 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 260/518] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:31:59.658825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.680292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.692839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c132e34-e18c-46a8-8ea9-dc1981e790f7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.725965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.726565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.726849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.727123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.756179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17889a9-60eb-498a-a342-399e128c0327 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:31:59.789686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:31:59.790309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:31:59.790588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:31:59.790858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:31:59.794328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.795776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.814632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.858607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:31:59.860243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:31:59.881472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:31:59.894507 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 259/519] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:59.895981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.960080 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 261/520] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:31:59.960850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:31:59.999163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a02af8-f6a6-45d4-b0b9-96781d37cc1a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.020811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.033262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.033928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.034214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.034488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.035208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:00.035994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.037591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.055030 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 260/521] 10.222.0.33 () {56 vars in 1139 bytes} [Sat Sep 10 20:31:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:00.055658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.071148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44375a-02fc-42d6-8b0f-63f1570d260f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.101848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.105602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.106227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.106506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.106774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.107403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.108819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.128605 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 262/522] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:31:59 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:00.129967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.162141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee181a-4f44-47b0-bbaa-b46f05a7c8a9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.201787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.202766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.203225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.203648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.212620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.214348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:00.228539 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:00.229600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:00.230492 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 261/523] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:00.230963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.250889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff20f8-461b-4868-96c6-c5b31d4585e1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.263159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.292404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.293100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.293390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.293672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.306567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.308072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:00.324537 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:00.325553 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:00.326758 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:00.326758 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:00.326758 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:00.326758 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 263/524] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:00.326758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.358428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f4d2b-a572-4a39-a64f-b24f8c66886e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.367606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.404803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.405468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.405756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.406120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.416753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.418250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:00.434329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:00.435297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:00.436168 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:00.436168 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:00.436168 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:00.436168 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 262/525] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:00.436399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.450297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec477fa9-5559-4723-8ce0-455811070ceb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.486746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.487281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.487563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.487836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.520672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.544375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b7753-7a79-49c0-9fa7-fce7af975b16 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.562185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.563654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.582005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.582950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.583359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.583728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.584684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:00.584994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.587052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.607696 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 263/526] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:00.608538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.630046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.678858 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 264/527] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:00.678858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.724653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcff66a-be60-42e6-b69c-99c6069d03cd None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.741164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.761427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.762071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.762357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.762636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.763280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:00.763586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.764976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.777877 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 264/528] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:00.777877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.789955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5b329-b9ee-441f-ae72-8295bc0adaa4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.826905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.827507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.827787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.828063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.828816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:00.829353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.830742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.842964 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 265/529] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:00.843753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.865219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:00.888686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a09ee-8bc1-44d9-9cca-6d9ba1634c72 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:00.924318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:00.924926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:00.925214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:00.925493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:00.926823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=6307d15dca3043b987a28fc41515acc2) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:00.927136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:00.928577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:00.934021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 265/530] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/role_assignments?user.id=6307d15dca3043b987a28fc41515acc2&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:00.934808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:00.974145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041776f1-342f-4e13-a26e-9919a77219ea None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.019371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:01.042276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.043828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.044316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.044827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.061607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f0bed-f184-4cd0-a235-70aa4219c3e0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.065324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.067271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:01.088745 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:01.089724 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:01.090649 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:01.090649 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:01.090649 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:01.092613 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 266/531] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:01.093148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.105343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.106073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.106350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.106626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.115709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.117357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:01.131037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:01.132000 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:01.132906 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:01.137274 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 266/532] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:01.138436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.141556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:01.231600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d66bde-b3f2-4c5b-abc7-5ecedb429dec None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.277444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.278034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.278322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.278601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.282040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143c7c5-805b-4f6f-89f2-bd6171bf94a4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.288070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.289534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:01.307312 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:01.308257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:01.309110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 267/533] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:01.309578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.332813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.334531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.335049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.335526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.336405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:01.336940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.339799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.351497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:01.365658 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 267/534] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:01.367237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.442423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:01.444202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aa215-f6e9-4a6b-bd66-e4e5412f18bb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.482025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.482631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.482917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.483194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.483896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:01.484205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.486279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.499687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2215c23-0d8f-43ad-8462-4ca72b4c54d7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.510297 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 268/535] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:01.511278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.547862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.548503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.548738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.549021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.549720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:01.550053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.551836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.565440 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 268/536] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:01.566381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.619065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:01.626745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f464a69-f440-4626-93f1-37e881393f5e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.664913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.665927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.666439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.666909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.674022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.675711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.681660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96775b87-05a3-4dff-9de4-a0d9dc81366f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:01.687953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:01.688792 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:01.689633 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:01.697441 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 269/537] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:01.698578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.706178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:01.720869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.721532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.721834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.722140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.723145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=5cdff5f5500040439f4acafa6c9ad948) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:01.723653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.725260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.731718 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 269/538] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/role_assignments?user.id=5cdff5f5500040439f4acafa6c9ad948&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:01.732775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.809692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5450cba-2847-4655-8cb5-1acb052c307d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.847842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.848887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.849373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.849881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.850836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=5fe5d3512475454080c8528db2208241) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:01.851371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.853031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd760256-cbbd-4808-bf9c-693000397ece None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:01.853158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.859624 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 270/539] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/role_assignments?user.id=5fe5d3512475454080c8528db2208241&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:01.894287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2503b1-4e05-4610-8274-05bf637c02c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:01.895649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2503b1-4e05-4610-8274-05bf637c02c1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:01.895974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2503b1-4e05-4610-8274-05bf637c02c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:01.896252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2503b1-4e05-4610-8274-05bf637c02c1 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:01.896964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2503b1-4e05-4610-8274-05bf637c02c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:01.897274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2503b1-4e05-4610-8274-05bf637c02c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:01.898922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2503b1-4e05-4610-8274-05bf637c02c1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:01.913741 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 270/540] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:01.919902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:01.956008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:02.060792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0c2d0-200d-44c5-a184-2bc9a493a866 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:02.101094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9707da-45d3-4a65-be39-d4b04059c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:02.101749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9707da-45d3-4a65-be39-d4b04059c1b1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:02.102066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9707da-45d3-4a65-be39-d4b04059c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:02.102345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9707da-45d3-4a65-be39-d4b04059c1b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:02.103092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9707da-45d3-4a65-be39-d4b04059c1b1 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=ed817faea9e84f49bb4bfb065fe4adbd) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:02.103405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9707da-45d3-4a65-be39-d4b04059c1b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:02.104915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9707da-45d3-4a65-be39-d4b04059c1b1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:02.111696 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 271/541] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:01 2022] GET /identity/v3/role_assignments?user.id=ed817faea9e84f49bb4bfb065fe4adbd&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:02.785247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325a73-70f9-4c78-ae13-81b3949e6b87 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:02.785885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325a73-70f9-4c78-ae13-81b3949e6b87 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:02.785885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325a73-70f9-4c78-ae13-81b3949e6b87 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:02.787638 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 271/542] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:02.793198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552703-3d57-4e23-83d1-2ad0d1afd44e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:02.793641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552703-3d57-4e23-83d1-2ad0d1afd44e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:02.793865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552703-3d57-4e23-83d1-2ad0d1afd44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:02.813548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-23552703-3d57-4e23-83d1-2ad0d1afd44e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:02.818882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23552703-3d57-4e23-83d1-2ad0d1afd44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:02.908150 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 272/543] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:03.470716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b325a73-70f9-4c78-ae13-81b3949e6b87 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:03.545507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:03.617273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b325a73-70f9-4c78-ae13-81b3949e6b87 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:03.665468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:03.666178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:03.666463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:03.666737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:03.681089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:03.686066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:03.686951 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:03.688125 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 272/544] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:03.693965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552703-3d57-4e23-83d1-2ad0d1afd44e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:03.708646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:03.817756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552703-3d57-4e23-83d1-2ad0d1afd44e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:03.859356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a513e-9788-41d3-a5cc-c8336204be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:03.859988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a513e-9788-41d3-a5cc-c8336204be86 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:03.860272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a513e-9788-41d3-a5cc-c8336204be86 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:03.860550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a513e-9788-41d3-a5cc-c8336204be86 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:03.861218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a513e-9788-41d3-a5cc-c8336204be86 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:03.861524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a513e-9788-41d3-a5cc-c8336204be86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:03.862937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a513e-9788-41d3-a5cc-c8336204be86 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:03.871857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 273/545] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:03.877461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:03.986892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628a42-8eaf-4769-a8ec-bf147f81be58 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:03.987471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628a42-8eaf-4769-a8ec-bf147f81be58 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:03.987942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628a42-8eaf-4769-a8ec-bf147f81be58 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:03.991678 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 274/546] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:03.996145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089f5c1-ea63-4426-b157-1da6cdb6457b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:03.996434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089f5c1-ea63-4426-b157-1da6cdb6457b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:03.996715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089f5c1-ea63-4426-b157-1da6cdb6457b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.020295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b57803-253b-4f08-8929-7c740dd69090 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:04.033566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9089f5c1-ea63-4426-b157-1da6cdb6457b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:04.035065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9089f5c1-ea63-4426-b157-1da6cdb6457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:04.035759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:04.105129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:04.105685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:04.105997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:04.106272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.127525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:04.128987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:04.153490 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 275/547] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:04.159459 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:04.160402 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:04.162461 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:04.162461 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:04.162461 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:04.162461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 273/548] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:04.166841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089f5c1-ea63-4426-b157-1da6cdb6457b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:04.179066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:04.340210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089f5c1-ea63-4426-b157-1da6cdb6457b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:04.391858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:04.392814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:04.393287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:04.393749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.396958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:04.397509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:04.399158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:04.417882 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 276/549] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:04.422991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:04.561485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece58a7d-6047-4e86-851c-aadaf59a0514 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:04.606866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:04.607492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:04.607768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:04.608038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] PATH_INFO: `/v3/users/swift` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.621530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:04.622987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:04.637712 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:04.638852 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:04.640369 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 274/550] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:04.644173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:04.644399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:04.674112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:04.779210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bba2e-cbfb-4ecb-9965-dde7c2bb487e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:04.787228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:04.824218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4bb1c-de35-4cdc-a952-d7630e33cba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:04.825206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4bb1c-de35-4cdc-a952-d7630e33cba1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:04.825808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4bb1c-de35-4cdc-a952-d7630e33cba1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:04.826313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4bb1c-de35-4cdc-a952-d7630e33cba1 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.827387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4bb1c-de35-4cdc-a952-d7630e33cba1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:04.827916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4bb1c-de35-4cdc-a952-d7630e33cba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:04.829692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4bb1c-de35-4cdc-a952-d7630e33cba1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:04.843161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9d295-d3c9-4b04-ac78-d1ffe1e9f422 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:04.850637 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 275/551] 10.222.0.33 () {56 vars in 1135 bytes} [Sat Sep 10 20:32:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:04.852275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830134-527d-4853-a80c-d413f5eb2a29 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:04.852869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830134-527d-4853-a80c-d413f5eb2a29 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:04.853564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830134-527d-4853-a80c-d413f5eb2a29 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.855368 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 276/552] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:04.858289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830134-527d-4853-a80c-d413f5eb2a29 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:04.884223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60b3d1-64b0-4ae4-9320-6400ce3b6136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:04.884858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60b3d1-64b0-4ae4-9320-6400ce3b6136 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:04.885140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60b3d1-64b0-4ae4-9320-6400ce3b6136 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:04.885409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60b3d1-64b0-4ae4-9320-6400ce3b6136 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.896188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60b3d1-64b0-4ae4-9320-6400ce3b6136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6e60b3d1-64b0-4ae4-9320-6400ce3b6136 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:04.901746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:04.902686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:04.903959 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 277/553] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:04.904659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da8fa5-c5aa-4e2f-bd79-b6770738955e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:04.904659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da8fa5-c5aa-4e2f-bd79-b6770738955e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:04.905041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da8fa5-c5aa-4e2f-bd79-b6770738955e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:04.931517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-64da8fa5-c5aa-4e2f-bd79-b6770738955e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:04.932711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64da8fa5-c5aa-4e2f-bd79-b6770738955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:04.951710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:04.991843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830134-527d-4853-a80c-d413f5eb2a29 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.003500 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 278/554] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:05.004842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da8fa5-c5aa-4e2f-bd79-b6770738955e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.039181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f0567-9d16-4df8-b388-8211cdb683a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.041249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0567-9d16-4df8-b388-8211cdb683a4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.041746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0567-9d16-4df8-b388-8211cdb683a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.042359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0567-9d16-4df8-b388-8211cdb683a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.070442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.124964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da8fa5-c5aa-4e2f-bd79-b6770738955e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.132801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0567-9d16-4df8-b388-8211cdb683a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:05.134305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0567-9d16-4df8-b388-8211cdb683a4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:05.172447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c2bcd-3dad-430f-bb0b-f8483d282d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.173399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2bcd-3dad-430f-bb0b-f8483d282d36 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.173913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2bcd-3dad-430f-bb0b-f8483d282d36 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.174387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2bcd-3dad-430f-bb0b-f8483d282d36 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.175266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c2bcd-3dad-430f-bb0b-f8483d282d36 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:05.176335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c2bcd-3dad-430f-bb0b-f8483d282d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:05.178431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c2bcd-3dad-430f-bb0b-f8483d282d36 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:05.190462 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 279/555] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:05.192477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508ee84-23d2-4ba9-94d5-5d1af2629f13 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.192968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508ee84-23d2-4ba9-94d5-5d1af2629f13 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.193460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508ee84-23d2-4ba9-94d5-5d1af2629f13 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.195266 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 280/556] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:05.199529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508ee84-23d2-4ba9-94d5-5d1af2629f13 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.231753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.264507 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 277/557] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 20:32:05.266138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1cfd0-5806-4a70-b609-5a0843590075 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.266675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1cfd0-5806-4a70-b609-5a0843590075 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.267153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1cfd0-5806-4a70-b609-5a0843590075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.290394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-94f1cfd0-5806-4a70-b609-5a0843590075 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:05.291623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-94f1cfd0-5806-4a70-b609-5a0843590075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:05.319098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508ee84-23d2-4ba9-94d5-5d1af2629f13 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.337045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.356774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.357407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.357685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.357998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.362225 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 278/558] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:05.363364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea63c1-78e5-456e-8f5b-68df19472c97 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.365875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea63c1-78e5-456e-8f5b-68df19472c97 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.365875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea63c1-78e5-456e-8f5b-68df19472c97 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.365875 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 279/559] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:05.366352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea63c1-78e5-456e-8f5b-68df19472c97 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.371604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:05.373087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:05.388154 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:05.389248 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:05.390146 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:05.390146 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:05.390146 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:05.390146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 281/560] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:05.390852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.487493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea63c1-78e5-456e-8f5b-68df19472c97 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.496858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.499163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2c9e5-0050-474f-8b2e-d3e7799561b3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.526508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.527141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.527424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.527700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.538771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:05.539529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed442b1-c3ee-4eac-b946-0425f446dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.540254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:05.540394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed442b1-c3ee-4eac-b946-0425f446dd38 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.540659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed442b1-c3ee-4eac-b946-0425f446dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.540895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed442b1-c3ee-4eac-b946-0425f446dd38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.551527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed442b1-c3ee-4eac-b946-0425f446dd38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:05.556632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:05.557605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:05.563492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:05.563492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:05.563492 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-3ed442b1-c3ee-4eac-b946-0425f446dd38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:05.563676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:05.564600 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:05.565428 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 282/562] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:05.566159 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 280/562] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:05.566159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.566284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e167565-d3ed-4dad-baf7-4b81f67eeca2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.566284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e167565-d3ed-4dad-baf7-4b81f67eeca2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.566284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e167565-d3ed-4dad-baf7-4b81f67eeca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.578405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3e167565-d3ed-4dad-baf7-4b81f67eeca2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:05.579658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e167565-d3ed-4dad-baf7-4b81f67eeca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:05.605644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.654414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 283/563] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:05.655476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e167565-d3ed-4dad-baf7-4b81f67eeca2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.680376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98f90e-67f6-4d78-b9e3-f5b796fb9be9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.718460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.719349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.719845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.720319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.721136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:05.721696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:05.723563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:05.736486 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 281/564] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:05.737566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.758645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e167565-d3ed-4dad-baf7-4b81f67eeca2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.761732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.798807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.799554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.799812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.800241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.801849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:05.802609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:05.804146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:05.823865 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 284/565] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:05.823865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.845175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564857e9-c096-4450-8a1b-e61725324052 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.849484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.884204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.884806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.885087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.885361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.886050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:05.886357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:05.887708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:05.896439 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 282/566] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:05.897223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:05.943514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362534e-825b-421f-b50b-de1e1d3dabbd None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:05.986890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:05.987532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:05.987864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:05.988086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:05.993689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:05.998833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:06.004641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:06.005764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 285/567] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:06.006666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.018872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddae8a-8e81-46c1-b667-d32735568ebf None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.052979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.053600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.053911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.054188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.061437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.062903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:06.074996 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:06.075953 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 283/568] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:32:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:06.076876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.085151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.106648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dbf69-9f13-4995-9c13-6cb99ebc5e77 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.139517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.140139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.140422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.140693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.147810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.149272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:06.162393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:06.163477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 286/569] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:06.164582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.183668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200feb9c-d2c1-4cb5-ac2b-f5280325aad0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.217358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.218019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.218301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.218573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.227636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.229853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.231422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:06.246363 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:06.247496 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:06.248592 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:06.248592 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:06.248592 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:06.249953 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 284/570] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:06.250730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.266426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f6c47-3c53-47dc-80e2-84a4034306fd None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.300065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efba791-98a6-43af-a910-1b10046876db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.300689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efba791-98a6-43af-a910-1b10046876db None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.301014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efba791-98a6-43af-a910-1b10046876db None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.301303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efba791-98a6-43af-a910-1b10046876db None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.308345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.313388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efba791-98a6-43af-a910-1b10046876db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-4efba791-98a6-43af-a910-1b10046876db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:06.318506 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:06.319566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 287/571] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:06.320503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efba791-98a6-43af-a910-1b10046876db None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.359378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b08d05-e823-46a0-9e50-ebae01656a8d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.391633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.392227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.392510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.392781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.393429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:06.393729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.395863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.403839 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 285/572] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:06.404456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.424618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efba791-98a6-43af-a910-1b10046876db None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.459523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.460134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.460290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.460416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.460824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.461501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:06.461905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.463652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.480661 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 288/573] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:06.481467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.511371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385cc2f-610e-4983-8047-4df67c57edc3 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.537185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.546841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.547444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.547732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.548008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.548697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:06.549004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.550466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.563195 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 286/574] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:06.564049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.581086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16bab4-f97c-478e-9393-5f33699eb359 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.616607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.617207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.617485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.618201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.618847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:06.619147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.620496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.631981 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 289/575] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:06.632760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.664227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eef99-080c-4911-a420-86c72b3169d7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.677889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.698318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.698922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.699201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.699473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.700120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:06.700424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.701768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.709952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 287/576] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:06.710741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.737263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fa0a0-462a-4071-9513-d0a6c190e860 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.753507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.772661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.773222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.773503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.773778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.783931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.786013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:06.799871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:06.801615 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:06.802686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:06.802686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:06.802686 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:06.802686 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 290/577] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:06.802949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.814525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c7f23-4b86-44a9-b6a9-03a9fc106af8 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.848645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.849261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.849546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.849818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.861140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.862607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:06.877008 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:06.878164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:06.879105 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:06.879105 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:06.879105 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:06.879105 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 288/578] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:06.879361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.904258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677e368-8c98-4f7d-96c0-423583f99f0e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:06.907862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.938043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:06.938637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:06.938914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:06.939190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:06.939896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=32f1fd57a99e4c0b9e50e997c5ee74f1) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:06.940196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:06.941662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:06.946967 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 291/579] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/role_assignments?user.id=32f1fd57a99e4c0b9e50e997c5ee74f1&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:06.947721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:06.989282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:06.989673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f16f99-9875-49b2-bd82-93d2d961bc2f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.032529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.033195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.033365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.033871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] PATH_INFO: `/v3/users/glance` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.043448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.045043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:07.057767 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:07.059034 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:32:07.060299 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:07.064010 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 289/580] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:07.065380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.072337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35473c0d-6421-4509-88d3-5c2c233cd3da None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.111920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.112557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.112835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.113105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.126052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.127538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:07.143003 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:07.144217 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:07.146423 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:07.146423 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:07.146423 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:07.146423 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 292/581] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:07.150335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.176448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:07.190690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483d5a3-8500-4ea5-8b95-419f3a697c2d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.227673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.228284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.228566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.228843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.229510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:07.229815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.231200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.236738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:07.245202 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 290/582] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:07.245774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.254790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743f2c2-915a-42ce-8687-63cfdd54fa5f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.290637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.291313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.291695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.292217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.292809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:07.293230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.294758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.310758 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 293/583] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:07.315029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.380211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e84f7-37a2-4234-b68e-823a67b9f1ea None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.421391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.422039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.422325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.422596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.423493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:07.424105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.425533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.439263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c7de-23ca-4c09-8bc5-c1574e54a391 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.450948 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 291/584] 10.222.0.33 () {56 vars in 1137 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:07.450948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.465745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:07.480215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.480924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.481270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.481608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.482356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:07.482732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.484179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.496514 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 294/585] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:07.497457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.501888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:07.599192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814486b8-c9e4-45d9-b165-4eeec635833d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.635417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcb3e1-881d-468b-b2f4-b0ff3944a651 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.651392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.652509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.654787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.655153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.665793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.668953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:07.685913 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:07.686892 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 292/586] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:07.687824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.689280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.690182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.690676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.691148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.700233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.702022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:07.717014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:07.717998 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:07.718899 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:07.723097 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 295/587] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:07.724070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.768814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:07.800701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:07.808599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcd719-8e8c-46e0-afd4-03f3c98d9b41 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.847499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938656f-d309-4a28-9694-a10797f631d6 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:07.851700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.852339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.853372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.853660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.867777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.869422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:07.884508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:07.885478 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:07.886401 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:07.886401 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:07.886401 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:07.886401 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 293/588] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:07.887045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:07.891383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:07.892428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:07.893001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:07.893468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:07.903546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:07.905003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:07.917984 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:07.918946 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:32:07.919915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:07.924467 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 296/589] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:07.925307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.028283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f075d-c793-4e8e-977c-5320b3bcab08 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.050177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:08.071755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3e4a-5061-4dbb-ab2c-4f540e626067 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.079616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.079616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.079616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.080219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.080956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:08.081571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.084108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.102178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:08.110083 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 294/590] 10.222.0.33 () {56 vars in 1137 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:08.112007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.116273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.117159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.117662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.118272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.119174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:08.119709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.121348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.135960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 297/591] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:08.137184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.220263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101fba4-a1c0-4055-95ed-76ca61f9d82e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.246437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152519cc-9db0-451c-ae03-0495e6b8c017 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.263736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.264343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.264621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.264891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.265529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:08.265927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.267450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.284662 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 295/592] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:08.286102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.301571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.302338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.302802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.303089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.303914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:08.304228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.306180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.326106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:08.330193 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 298/593] 10.222.0.33 () {56 vars in 1139 bytes} [Sat Sep 10 20:32:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:08.331332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.402964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:08.415055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36476e35-b957-4c96-a463-dde034e62f23 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.466226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.466866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.467149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.467428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.477772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39b2ec-336f-4e12-8002-e1d235e046fc None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.485647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.489093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:08.508196 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:08.509351 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:08.510483 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:08.510483 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:08.510483 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:08.512299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 296/594] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:08.512853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.524993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.525743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.526056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.526335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/6307d15dca3043b987a28fc41515acc2/roles/34d4065b512e49b883776816da214951` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.546269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=6307d15dca3043b987a28fc41515acc2, role_id=34d4065b512e49b883776816da214951)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.548545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.566107 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 299/595] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:08 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/6307d15dca3043b987a28fc41515acc2/roles/34d4065b512e49b883776816da214951 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:08.567910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.701115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:08.714154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:08.731299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec1ac-d78b-454d-9bc8-62a92bfe65da None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.731977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f098b22-6d81-4bb9-a930-1eff3fb71dad None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:08.801395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.802422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.802873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.803859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.805166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:08.806160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:08.807738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:08.808343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:08.816552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.818353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.828435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:08.829741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:08.836360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:08.837293 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:08.838502 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 297/596] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:08.841252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:08.857425 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:08.858461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:08.859845 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:08.859845 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:08.859845 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:08.859845 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 300/597] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:08.859845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:08.984635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6bfba-42f4-4e5d-8fa3-3441f7e28ac2 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.018690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e9053-570a-470a-9550-2c4149149d69 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.038581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.039560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.040337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.040818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.041895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:09.042353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.044004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.065173 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 298/598] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:09.067397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:09.072569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.073765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.074293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.074667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.078173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:09.078637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.080036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:09.082114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.105599 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 301/599] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:09.106719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:09.112990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:09.204017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24413e48-5a78-4bfe-b8d9-27c309460c97 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.256424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341858-325c-4198-8115-e24c31163224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.257421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341858-325c-4198-8115-e24c31163224 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.257947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341858-325c-4198-8115-e24c31163224 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.258429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341858-325c-4198-8115-e24c31163224 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.259277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341858-325c-4198-8115-e24c31163224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:09.259788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341858-325c-4198-8115-e24c31163224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.261370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341858-325c-4198-8115-e24c31163224 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.283435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 299/600] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:09.284750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341858-325c-4198-8115-e24c31163224 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:09.285161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1955217-b879-45d2-ba34-22d5d49d41f1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.333413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.334695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.334977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.335246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.353619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.355124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:09.373979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:09.374922 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:09.375820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:09.382550 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 302/601] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:09.387003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:09.415685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:09.469964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341858-325c-4198-8115-e24c31163224 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.499978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:09.516770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.517535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.517951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.518352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/5cdff5f5500040439f4acafa6c9ad948/roles/34d4065b512e49b883776816da214951` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.538462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=5cdff5f5500040439f4acafa6c9ad948, role_id=34d4065b512e49b883776816da214951)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.539917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.557676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf6aa6-7323-4542-8b36-ab5110dd50f9 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.566140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 300/602] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:09 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/5cdff5f5500040439f4acafa6c9ad948/roles/34d4065b512e49b883776816da214951 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:09.568006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:09.626576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a9029-1ff3-4aa1-bb81-b09e56251aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.627233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a9029-1ff3-4aa1-bb81-b09e56251aba None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.627515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a9029-1ff3-4aa1-bb81-b09e56251aba None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.627790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a9029-1ff3-4aa1-bb81-b09e56251aba None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.648882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a9029-1ff3-4aa1-bb81-b09e56251aba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.650609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a9029-1ff3-4aa1-bb81-b09e56251aba None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-365a9029-1ff3-4aa1-bb81-b09e56251aba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:09.680651 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:09.681663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:09.683065 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 303/603] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:09.683569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39821e21-5870-498f-b3e2-388d524821a9 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.684132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39821e21-5870-498f-b3e2-388d524821a9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.684132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39821e21-5870-498f-b3e2-388d524821a9 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.686628 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 304/604] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:09.688301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39821e21-5870-498f-b3e2-388d524821a9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:09.775276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119f261-f0bd-4a67-89ca-f3c296959955 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.825735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:09.858829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fbfd1-066f-4972-921f-4629621ac1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.859934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fbfd1-066f-4972-921f-4629621ac1e7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.859934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fbfd1-066f-4972-921f-4629621ac1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.860134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fbfd1-066f-4972-921f-4629621ac1e7 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.860779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fbfd1-066f-4972-921f-4629621ac1e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:09.861084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fbfd1-066f-4972-921f-4629621ac1e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.862561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fbfd1-066f-4972-921f-4629621ac1e7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.878457 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 301/605] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:09.883251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fa86d-90b5-4dc5-9a6b-086bc26f4a26 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.885083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fa86d-90b5-4dc5-9a6b-086bc26f4a26 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.886337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fa86d-90b5-4dc5-9a6b-086bc26f4a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.911743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39821e21-5870-498f-b3e2-388d524821a9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:09.913496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b96fa86d-90b5-4dc5-9a6b-086bc26f4a26 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:09.914910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b96fa86d-90b5-4dc5-9a6b-086bc26f4a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:09.939625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:09.962480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:09.964573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:09.965117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:09.966270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:09.967273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:09.967801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:09.969599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:09.993445 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 305/606] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:09.995897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:10.013954 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 302/607] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:10.013954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fa86d-90b5-4dc5-9a6b-086bc26f4a26 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:10.143171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fa86d-90b5-4dc5-9a6b-086bc26f4a26 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:10.156096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:10.188815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:10.189943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:10.190462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:10.191095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/ed817faea9e84f49bb4bfb065fe4adbd/roles/34d4065b512e49b883776816da214951` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:10.209156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=ed817faea9e84f49bb4bfb065fe4adbd, role_id=34d4065b512e49b883776816da214951)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:10.212045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:10.236845 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 303/608] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:10 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/ed817faea9e84f49bb4bfb065fe4adbd/roles/34d4065b512e49b883776816da214951 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:10.276320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38ead2-2901-4c3d-bfcd-2bc02936d478 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:10.348918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:10.349622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:10.349919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:10.350202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/5fe5d3512475454080c8528db2208241/roles/34d4065b512e49b883776816da214951` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:10.378063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=5fe5d3512475454080c8528db2208241, role_id=34d4065b512e49b883776816da214951)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:10.380512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:10.407292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 306/609] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:09 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/5fe5d3512475454080c8528db2208241/roles/34d4065b512e49b883776816da214951 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:10.502586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:10.654512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dea85-ff5d-4bb6-8386-a3809240f0f3 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:10.696188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:10.697235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:10.697854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:10.698543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:10.711143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:10.716851 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:10.717894 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:10.718758 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:10.721644 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 304/610] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:10.727590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:10.787420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:10.865335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892246a5-564e-4fd0-a603-3bc90e5fc1e5 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:10.908334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:10.908970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:10.909253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:10.909599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:10.911254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:10.911643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:10.913136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:10.928985 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 307/611] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:10.934907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:10.991399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:11.221786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80981d93-c38f-40cb-b48b-0b22dd9d87ce None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:11.309228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803526a-4df1-4eec-91cf-b5087106ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:11.310138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803526a-4df1-4eec-91cf-b5087106ecda None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:11.310578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803526a-4df1-4eec-91cf-b5087106ecda None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:11.311038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803526a-4df1-4eec-91cf-b5087106ecda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:11.337708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803526a-4df1-4eec-91cf-b5087106ecda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:11.340032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803526a-4df1-4eec-91cf-b5087106ecda None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b803526a-4df1-4eec-91cf-b5087106ecda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:11.363962 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:11.365174 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:11.366191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:11.366191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:11.366191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:11.367888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 305/612] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:11.373131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:11.526485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:11.534485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d137d7-528a-4da8-8a5b-7fcbeadb0e39 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:11.594695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:11.595302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:11.595585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:11.595859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:11.597500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:11.597500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:11.599046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:11.609849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bedb03-1551-461a-b430-258e737f3a86 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:11.609849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bedb03-1551-461a-b430-258e737f3a86 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:11.609849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bedb03-1551-461a-b430-258e737f3a86 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:11.609849 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 306/613] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:11.619437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677dd10-7ded-4387-8665-0a8d653d314b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:11.619437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677dd10-7ded-4387-8665-0a8d653d314b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:11.619684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677dd10-7ded-4387-8665-0a8d653d314b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:11.631397 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 308/614] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:11.639047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:11.652688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d677dd10-7ded-4387-8665-0a8d653d314b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:11.655753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d677dd10-7ded-4387-8665-0a8d653d314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:11.794481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:11.806098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 307/615] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:11.865816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e588ed-ffde-493e-87f8-bf0b9a2acaef None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:11.951477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:11.952120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:11.952401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:11.952673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:11.970971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:11.972521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:11.995451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:11.996440 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:11.997396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 309/616] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:12.004415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677dd10-7ded-4387-8665-0a8d653d314b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:12.248640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677dd10-7ded-4387-8665-0a8d653d314b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:12.316005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:12.330228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:12.331311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:12.331791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:12.332825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:12.334027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:12.334658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:12.336431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:12.357321 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 308/617] 10.222.0.33 () {56 vars in 1135 bytes} [Sat Sep 10 20:32:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:12.363186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:12.364535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:12.434693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:12.486219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea1d11-d0b6-4be1-8d0c-9bd5b65f4744 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:12.535493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:12.536519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:12.536998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:12.537482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:12.551364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:12.553131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:12.569722 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:12.570804 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:12.571684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:12.571684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:12.571684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:12.573778 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 309/618] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:12.587083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:12.621669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26491720-bc1d-44bd-91fb-319a62236005 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:12.670649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1586734-6def-42a3-a0de-2c7f30b05323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:12.671289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1586734-6def-42a3-a0de-2c7f30b05323 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:12.671572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1586734-6def-42a3-a0de-2c7f30b05323 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:12.671844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1586734-6def-42a3-a0de-2c7f30b05323 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:12.692996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1586734-6def-42a3-a0de-2c7f30b05323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-c1586734-6def-42a3-a0de-2c7f30b05323 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:12.701931 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:12.702877 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:12.703747 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:12.704414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 310/619] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:12.705460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21027691-2502-4d3d-b4bb-fee7a190b56e None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:12.705742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21027691-2502-4d3d-b4bb-fee7a190b56e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:12.707030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21027691-2502-4d3d-b4bb-fee7a190b56e None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:12.710219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb64376-9a56-4925-aa81-e208ccd7034d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:12.714083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 311/620] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:12.714917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21027691-2502-4d3d-b4bb-fee7a190b56e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:12.723373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:12.774640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59222fde-0364-495e-b168-4ab7262d735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:12.775501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59222fde-0364-495e-b168-4ab7262d735f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:12.775956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59222fde-0364-495e-b168-4ab7262d735f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:12.776377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59222fde-0364-495e-b168-4ab7262d735f None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:12.777169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59222fde-0364-495e-b168-4ab7262d735f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:12.777620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59222fde-0364-495e-b168-4ab7262d735f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:12.779164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59222fde-0364-495e-b168-4ab7262d735f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:12.808316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 310/621] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:12.808316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f3465-efc6-4b0a-bc9f-d846a35dd1c3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:12.808316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f3465-efc6-4b0a-bc9f-d846a35dd1c3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:12.808316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f3465-efc6-4b0a-bc9f-d846a35dd1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:12.845417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5c8f3465-efc6-4b0a-bc9f-d846a35dd1c3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:12.846658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5c8f3465-efc6-4b0a-bc9f-d846a35dd1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:12.874463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:12.924429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21027691-2502-4d3d-b4bb-fee7a190b56e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:12.993213 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 311/622] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:12.994860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f3465-efc6-4b0a-bc9f-d846a35dd1c3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:12.998863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:12.999719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.000203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.000739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.003345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:13.005447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:13.007105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:13.030075 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 312/623] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:13.037914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:13.223342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:13.224845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dace060-0aea-482f-b94f-4b7b6d417b64 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:13.235892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f3465-efc6-4b0a-bc9f-d846a35dd1c3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:13.272708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:13.273995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.274519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.275079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.285820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:13.298693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:13.300167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:13.332913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:13.334753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.335264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.335843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.344407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:13.346226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:13.351388 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:13.352378 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:13.353306 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:13.353306 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:13.353306 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:13.353306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 313/624] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:13.359368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:13.364632 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:13.365613 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:13.366551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:13.369590 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 312/625] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:13.383059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:13.531744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c9887-7708-4d64-ad12-d8637dc14e47 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:13.581812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:13.582856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.583588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.584070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.584985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:13.585630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:13.587872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:13.590751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250405f-feca-46d0-a169-4674931ce6b8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:13.609035 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 313/626] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:13.609035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:13.634333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:13.638707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd1a58-e337-4caf-8e87-4b614a30f86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:13.639988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd1a58-e337-4caf-8e87-4b614a30f86a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.640436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd1a58-e337-4caf-8e87-4b614a30f86a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.640868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd1a58-e337-4caf-8e87-4b614a30f86a None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.641759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd1a58-e337-4caf-8e87-4b614a30f86a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:13.642239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd1a58-e337-4caf-8e87-4b614a30f86a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:13.643876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd1a58-e337-4caf-8e87-4b614a30f86a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:13.657210 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 314/627] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:13.663748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18445f26-66cb-42de-afc9-73761ba2fb9a None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.664206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18445f26-66cb-42de-afc9-73761ba2fb9a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.664619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18445f26-66cb-42de-afc9-73761ba2fb9a None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.666550 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 315/628] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:13.667523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18445f26-66cb-42de-afc9-73761ba2fb9a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:13.724441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617dd94-daa5-4702-b268-790403039c3b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:13.728595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:13.760603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:13.761239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.761522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.761796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.774446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:13.779652 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:13.780586 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:13.781481 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 314/629] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:13.782514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:13.794210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18445f26-66cb-42de-afc9-73761ba2fb9a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:13.835997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf503a8a-c6fe-4288-944c-782666663f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:13.836781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf503a8a-c6fe-4288-944c-782666663f19 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.837066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf503a8a-c6fe-4288-944c-782666663f19 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.837341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf503a8a-c6fe-4288-944c-782666663f19 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/32f1fd57a99e4c0b9e50e997c5ee74f1/roles/34d4065b512e49b883776816da214951` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.853782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf503a8a-c6fe-4288-944c-782666663f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=32f1fd57a99e4c0b9e50e997c5ee74f1, role_id=34d4065b512e49b883776816da214951)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:13.856096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf503a8a-c6fe-4288-944c-782666663f19 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:13.870365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:13.874033 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 316/630] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:13 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/32f1fd57a99e4c0b9e50e997c5ee74f1/roles/34d4065b512e49b883776816da214951 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:13.875413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99cc9c-3956-4131-8057-6038d3d02d04 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.875709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99cc9c-3956-4131-8057-6038d3d02d04 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.876011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99cc9c-3956-4131-8057-6038d3d02d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.895704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0f99cc9c-3956-4131-8057-6038d3d02d04 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:13.896441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d800c4-c2f6-4fd9-b322-3425bfb4249b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:13.896923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f99cc9c-3956-4131-8057-6038d3d02d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:13.943959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ef0f2-1252-47a2-afc0-d854841c5e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:13.944767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef0f2-1252-47a2-afc0-d854841c5e56 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.945052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef0f2-1252-47a2-afc0-d854841c5e56 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.945330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef0f2-1252-47a2-afc0-d854841c5e56 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.956765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ef0f2-1252-47a2-afc0-d854841c5e56 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:13.959599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ef0f2-1252-47a2-afc0-d854841c5e56 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-911ef0f2-1252-47a2-afc0-d854841c5e56 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:13.976624 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:13.977928 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:13.979106 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:13.982008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 315/631] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:32:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:13.983525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9af4d-40de-409b-9ca1-84e3a11a1020 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:13.983884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9af4d-40de-409b-9ca1-84e3a11a1020 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:13.984227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9af4d-40de-409b-9ca1-84e3a11a1020 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:13.985818 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 316/632] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:13.991223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9af4d-40de-409b-9ca1-84e3a11a1020 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:14.003111 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 317/633] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:14.004821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99cc9c-3956-4131-8057-6038d3d02d04 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:14.070925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:14.137439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9af4d-40de-409b-9ca1-84e3a11a1020 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:14.185464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99cc9c-3956-4131-8057-6038d3d02d04 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:14.193246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4ad61-e2fc-493d-9945-ea7567ee1799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:14.194021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ad61-e2fc-493d-9945-ea7567ee1799 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:14.194304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ad61-e2fc-493d-9945-ea7567ee1799 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:14.194571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ad61-e2fc-493d-9945-ea7567ee1799 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:14.195228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ad61-e2fc-493d-9945-ea7567ee1799 None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:14.195528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ad61-e2fc-493d-9945-ea7567ee1799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:14.196896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ad61-e2fc-493d-9945-ea7567ee1799 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:14.211134 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 317/634] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:14.212499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549e03-1676-4b55-912b-3fa6d6095b51 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:14.212791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549e03-1676-4b55-912b-3fa6d6095b51 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:14.213078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549e03-1676-4b55-912b-3fa6d6095b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:14.220803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:14.236577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:14.237576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:14.238078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:14.238897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:14.239996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:14.240656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:14.241187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a6549e03-1676-4b55-912b-3fa6d6095b51 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:14.243818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a6549e03-1676-4b55-912b-3fa6d6095b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:14.245415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:14.267752 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 318/635] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:32:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:14.269090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:14.331645 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 318/636] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:14.337998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6549e03-1676-4b55-912b-3fa6d6095b51 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:14.446487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:14.475094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76103-2ef7-4c4b-84d0-68365aa2dd97 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:14.528246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6549e03-1676-4b55-912b-3fa6d6095b51 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:14.552775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:14.553777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:14.555619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:14.556103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:14.578211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:14.580089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:14.595661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:14.605945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:14.607493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:14.607889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:14.608371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:14.609511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:14.610737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:14.610737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:14.610737 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:14.612468 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 319/637] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:14.614183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:14.614718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:14.739105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:14.740595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:14.775889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b9edf-7037-4718-8261-5ec24f799d96 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:14.822512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:14.823260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:14.823613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:14.823964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:14.833024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:14.837445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:14.843321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:14.844315 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:14.845185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:14.846689 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 320/638] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:14.846794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:14.884026 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 319/639] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:14.885197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:14.921129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:14.974243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ac3a6-3af0-48c1-9188-d2c486e2ae38 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.011642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3304603-0080-4a01-967d-7919a74fe7c3 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.016417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.017028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.017312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.017589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.018343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:15.018647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.020246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.033633 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 321/640] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:15.034379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.058491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.059538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.059926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.060301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.073767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:15.078830 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:15.080007 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:15.081070 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:15.085309 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 320/641] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:15.086333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.115163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:15.173805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fc6c2-5d8b-4e82-b4cd-758038e64f13 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.216374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:15.222767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.223391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.223679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.223957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.224813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:15.225655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.227459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.228299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55ce7a-7526-4894-b892-48c202150355 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.238932 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 322/642] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:15.240015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.276692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.277329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.277613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.277927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.290287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.291857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:15.309803 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:15.310893 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:15.311795 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:15.311795 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:15.311795 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:15.311795 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 321/643] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:15.312475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.366523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1bea3-4b7f-4c1f-9bf9-f742f5a0656d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.393639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:15.407970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.408927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.409598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.409939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] PATH_INFO: `/v3/users/glance` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.421140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.422664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:15.436341 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:15.437366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 20:32:15.438511 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:15.441341 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 323/644] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:15.441341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.568469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae47642-472a-4847-85ec-79161ec804b9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.581254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0376eb-63ce-466e-956a-c0b97b72abef None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.620855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.621699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.622034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.622311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.622977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:15.623289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.624675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.628439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.629557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.630761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.631225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.633772 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 322/645] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:15.640043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.645451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.647232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.650117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:15.666227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:15.667273 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:15.668177 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:15.668177 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:15.668177 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:15.670599 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 324/646] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:15.671483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.714221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:15.771510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55e51-5ddb-4c2c-8427-f8c42035d32d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.803610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce266-abb2-47f6-9c52-cb48ecd5e9ea None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:15.819087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.820224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.820707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.821289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.822048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:15.822459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.824179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.838188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 323/647] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:15.839389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.849890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:15.850685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:15.853047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:15.853047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:15.853047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:15.853047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:15.854211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:15.879744 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 325/648] 10.222.0.33 () {56 vars in 1137 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:15.880726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:15.938464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:15.988864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438e6a7-abbe-4317-94c0-e3e607f8b4b2 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.015224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:16.026096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4eb8f-dd37-467b-8832-27e3d5caa79c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.038157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.038795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.040030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.040030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.054142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.055972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.071095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.071950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.072382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.072835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.073646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:16.074168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:16.075605 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:16.076728 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:16.077817 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:16.077817 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:16.077817 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:16.078209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.084824 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 324/649] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:16.086146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.101483 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 326/650] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:16.102948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.221924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b510d1-0de3-4bd1-ab50-a5cefff73fae None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.240165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f7808-fc98-4166-9a9b-9ec686a72c26 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.242568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:16.266093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.267194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.267485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.267763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.277855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.280226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.286695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.287311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.287623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.287894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:16.295341 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:16.296245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:16.297160 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:16.302191 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 325/651] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:16.303244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.320186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:16.390896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.392416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.428619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28af99-db12-462b-8ae5-fcebbfedfdef None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.469280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.469905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.470187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.470458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.471091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:16.471394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.472888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.486348 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 326/652] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:16.487190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.516171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 327/653] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:16.517016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.536654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:16.575555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:16.611217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0b996-27be-4870-94c2-bc43f4b606a3 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.628699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267478c5-7e70-47ca-bb35-263b51a16354 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.649621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55674a-862a-41f3-a9eb-64c09be3ad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.650299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55674a-862a-41f3-a9eb-64c09be3ad58 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.650582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55674a-862a-41f3-a9eb-64c09be3ad58 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.650857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55674a-862a-41f3-a9eb-64c09be3ad58 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.658549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55674a-862a-41f3-a9eb-64c09be3ad58 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.660979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55674a-862a-41f3-a9eb-64c09be3ad58 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.667038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.667646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.667920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.668192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.668883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:16.669192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.670681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-bd55674a-862a-41f3-a9eb-64c09be3ad58 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:16.674645 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:16.675868 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:16.676860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 327/654] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:16.677461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfd614-c739-4207-bf3f-43cf49672090 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.677540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfd614-c739-4207-bf3f-43cf49672090 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.677792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfd614-c739-4207-bf3f-43cf49672090 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.681970 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 328/655] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:16.682765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfd614-c739-4207-bf3f-43cf49672090 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.683777 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 328/656] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:16.684812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.792783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:16.795280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3e935-dfd4-4127-b4ff-8ddd13b315ac None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.799190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfd614-c739-4207-bf3f-43cf49672090 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:16.809218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:16.835404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc707fda-6dd3-426f-8d62-0e5d58b4bf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.836129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc707fda-6dd3-426f-8d62-0e5d58b4bf13 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.836484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc707fda-6dd3-426f-8d62-0e5d58b4bf13 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.836819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc707fda-6dd3-426f-8d62-0e5d58b4bf13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.841784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:16.842522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.842874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.843214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.848536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc707fda-6dd3-426f-8d62-0e5d58b4bf13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.850347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc707fda-6dd3-426f-8d62-0e5d58b4bf13 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.852291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:16.853727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:16.866198 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:16.867381 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 329/657] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:16.868508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-dc707fda-6dd3-426f-8d62-0e5d58b4bf13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:16.870310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:16.871234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:16.872171 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:16.872171 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:16.872171 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:16.872171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 329/658] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:16.872766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ac81-6e4c-497e-af83-ebbb110df0cc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:16.873057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ac81-6e4c-497e-af83-ebbb110df0cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:16.873334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ac81-6e4c-497e-af83-ebbb110df0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:16.895098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5bc1ac81-6e4c-497e-af83-ebbb110df0cc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:16.896489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5bc1ac81-6e4c-497e-af83-ebbb110df0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:16.975311 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 330/659] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:16.976063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1ac81-6e4c-497e-af83-ebbb110df0cc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.000188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe020fe-37c7-4cf9-81aa-b048bd206f0f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.043025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.043860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.044312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.044738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.045628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:17.046123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:17.047736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:17.068838 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 330/660] 10.222.0.33 () {56 vars in 1139 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:17.069964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.085667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:17.093706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1ac81-6e4c-497e-af83-ebbb110df0cc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.093880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:17.140913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.141900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.142387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.142885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.143793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=6307d15dca3043b987a28fc41515acc2) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:17.144292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:17.146430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:17.152957 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 331/661] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:16 2022] GET /identity/v3/role_assignments?user.id=6307d15dca3043b987a28fc41515acc2&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:17.154215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.203630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9d4ce-5555-4142-beb9-994b4f2395f6 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.265915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.266909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.267421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.267907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.268894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:17.269440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:17.271155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:17.309687 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 331/662] 10.222.0.33 () {56 vars in 1137 bytes} [Sat Sep 10 20:32:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:17.310997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.335599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd24d2-2bc2-476c-9e80-32f5dda5a406 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.389516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.390812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.391792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.392296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.393400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:17.393715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:17.397096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:17.411338 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 332/663] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:17.412251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.422800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:17.436587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:17.470278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7141b-ff9c-4278-8456-b777f9b2adb4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.516111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.517654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.518291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.518575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.567924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b5e4d-20ed-4206-a193-b8b4f5577782 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.609200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.609565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.609762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.610206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.612571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:17.615411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:17.621895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:17.626871 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:17.627815 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 333/664] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:17.630002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.720929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:17.727585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:17.745960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae7b2-db44-4792-85e9-744ac749dbab None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.764541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 332/665] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:17.765787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.785693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.786485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.786821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.787104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.871827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:17.873848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:17.888378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdd8eb-f11a-40fe-a229-e9667558bed2 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:17.933769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:17.934767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:17.935050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:17.935325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:17.943915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:17.945405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:17.958771 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:17.959707 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:17.960603 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:17.962703 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 333/666] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:17.964191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:17.981221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:18.018380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:18.018680 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 334/667] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:18.020938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.103608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beca7ba-36c4-4b1f-b2ca-0923b149f742 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.146590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.147642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.148331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.148811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.149680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:18.150183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1307ad-c8aa-44ac-8da4-6f1838d2c36f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.150435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.152534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.168395 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 334/668] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:18.170136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.196898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.197597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.197914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.198191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.212820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.214572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:18.235330 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:18.236529 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:18.237466 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:18.237466 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:18.237466 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:18.237466 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 335/669] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:18.237771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.279820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:18.310408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:18.320400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57ff77-daed-4421-92d6-aff41b1b9516 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.371902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.373239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.374330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.374618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.375467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:18.375781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.377452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.382214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3891d4-3802-4478-af6f-52dbf308861a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.396141 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 335/670] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:18.397935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.427284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.428224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.428683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.429323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.443076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.445211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:18.461754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:18.462806 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:18.463965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:18.463965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:18.463965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:18.470456 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 336/671] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:18.471565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.524677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847af493-4627-4675-9e48-1b459fae6fa9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.573716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.574673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.574962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.575240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.575685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:18.591809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.593259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.610682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce407c9-fddc-40e1-960e-770b3cd7f509 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:18.614354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:18.615785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:18.616863 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:18.616863 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:18.616863 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:18.624595 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 336/672] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:18.624595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.637355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:18.669915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.670463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.670744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.671018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.671669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:18.671973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.674529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.695071 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 337/673] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:18.696003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.809172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcceda49-e2a0-466e-87f8-07f179a97cf0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.834041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9206a9-8357-4371-a7b7-9c5821581ac4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:18.860577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.861357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.861730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.861946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.862909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=5cdff5f5500040439f4acafa6c9ad948) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:18.863407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.866411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.874731 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 337/674] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/role_assignments?user.id=5cdff5f5500040439f4acafa6c9ad948&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:18.875559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.881905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:18.883246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:18.883820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:18.884300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:18.885231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:18.885992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:18.888191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:18.903740 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 338/675] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:18.905126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:18.909128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:18.992968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:19.042837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615536f9-968a-42ed-83a6-7bc03de37efd None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:19.067188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ee580-77b2-47cb-83f4-34b23fe70880 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:19.091826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:19.092876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:19.093366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:19.093883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:19.102559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:19.104427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:19.119929 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:19.120923 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:19.121850 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:19.130797 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 339/676] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:19.131798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:19.146454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:19.147076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:19.147346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:19.147619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:19.148361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:19.148564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:19.150609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:19.171466 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 338/677] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:19.173080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:19.220284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:19.277121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43482df3-74c3-4125-8351-3c5ce98b9c66 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:19.367944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58127df-9285-4e1f-9894-e57b57902c7f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:19.383982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:19.385306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:19.386224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:19.386954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:19.401970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:19.403701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:19.408952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:19.416556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:19.416556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:19.416886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:19.417066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:19.417856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:19.418260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:19.419835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:19.424688 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:19.425641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:19.427232 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:19.429974 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 340/678] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:19.431419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:19.441479 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 339/679] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:19.441479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:19.554740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd142a9-12d9-4a94-a54d-b81f762748b4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:19.634232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:19.637281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:19.639114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:19.640411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:19.641621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:19.643546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:19.646850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:19.676571 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 340/680] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:19.682217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:19.694467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3909e0-4dd6-4773-9741-1ab3b51bc8ec None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:19.754802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:19.767272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:19.779058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:19.785662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:19.786149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:19.786594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:19.802109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:19.808193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:19.826890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93fd1c-08fd-45b4-a8a0-05d3d3e9b3c1 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:19.843692 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:19.844953 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:19.846114 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:19.850527 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 341/681] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:19.851579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:19.870998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:19.871778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:19.872157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:19.875823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:19.876679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=ed817faea9e84f49bb4bfb065fe4adbd) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:19.877101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:19.878705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:19.885342 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 341/682] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:19 2022] GET /identity/v3/role_assignments?user.id=ed817faea9e84f49bb4bfb065fe4adbd&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:19.887574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:20.119222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72487692-c30e-4641-ae87-83cb55255cb0 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:20.168915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef43b-74ef-4249-ac2e-9848a8efabcd None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:20.206031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e43d-39c2-4fcd-be86-3d5dc18a05f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:20.206658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e43d-39c2-4fcd-be86-3d5dc18a05f0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:20.206964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e43d-39c2-4fcd-be86-3d5dc18a05f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:20.207284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e43d-39c2-4fcd-be86-3d5dc18a05f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:20.209381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5e43d-39c2-4fcd-be86-3d5dc18a05f0 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=5fe5d3512475454080c8528db2208241) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:20.209381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5e43d-39c2-4fcd-be86-3d5dc18a05f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:20.211085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5e43d-39c2-4fcd-be86-3d5dc18a05f0 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:20.212882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:20.224158 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 342/683] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:19 2022] GET /identity/v3/role_assignments?user.id=5fe5d3512475454080c8528db2208241&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 608 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:20.225731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcf937-c99a-4b20-aaf9-d133c2bad840 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:20.227249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcf937-c99a-4b20-aaf9-d133c2bad840 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:20.227931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcf937-c99a-4b20-aaf9-d133c2bad840 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:20.234485 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 343/684] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:20.240735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb7f9-5aa6-469c-98fd-87b48e375875 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:20.241618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb7f9-5aa6-469c-98fd-87b48e375875 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:20.242362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb7f9-5aa6-469c-98fd-87b48e375875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:20.264335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:20.265293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:20.265779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:20.266325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:20.267291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:20.268455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:20.271412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:20.287264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4b6cb7f9-5aa6-469c-98fd-87b48e375875 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:20.290822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4b6cb7f9-5aa6-469c-98fd-87b48e375875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:20.302500 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 342/685] 10.222.0.33 () {56 vars in 1135 bytes} [Sat Sep 10 20:32:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:20.313609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:20.408248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:20.444223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcfc99-dc87-4430-919c-c7425bec2762 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:20.484615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:20.485216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:20.485502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:20.485779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:20.508100 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 344/686] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:20.537516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:20.571755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:20.577594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:20.798726 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 343/687] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:20.802169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb7f9-5aa6-469c-98fd-87b48e375875 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:20.920375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:21.056994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:21.094625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddba970-0538-43f2-b3d1-1b65b5cd6e26 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:21.132196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb7f9-5aa6-469c-98fd-87b48e375875 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:21.155579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:21.156222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:21.159560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:21.159560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:21.176495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:21.183475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:21.184434 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:21.185282 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:21.186441 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 344/688] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:21.193850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:21.263230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:21.263885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:21.264171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:21.264450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:21.285241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:21.287923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:21.328176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:21.329182 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:21.330124 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:21.330124 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:21.330124 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:21.337795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 345/689] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 538 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:21.341795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:21.403295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31583a5-a79a-4770-88f6-b8276d8ca01f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:21.405242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:21.449784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:21.450268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:21.450554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:21.450832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:21.451500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:21.451804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:21.453175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:21.454951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:21.473533 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 345/690] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:21.479065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:21.525513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98f0c3-4428-4ec4-baa8-068866364ac7 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:21.589386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:21.590395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:21.590907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:21.591380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:21.591835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0619fc5-f472-4bea-b554-df50ef3fe3bf None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:21.592457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:21.592962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:21.594586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:21.622200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 346/691] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:21.627277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:21.634785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:21.635715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:21.636206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:21.636683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:21.649205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:21.651053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:21.666777 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:21.667874 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:21.668772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:21.668772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:21.668772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:21.670856 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 346/692] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:21.676343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:21.771244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:21.826276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcac7b-f487-422d-8f58-9280dfa213e7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:21.860913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:21.863705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:21.864466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:21.864854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:21.865876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:21.866635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:21.867045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:21.868520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:21.882719 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 347/693] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:21.889795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:21.898878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7504ab8-7dab-43ef-b3ab-f142767bbe77 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:21.974246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:21.981722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:21.981722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:21.981722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:21.993613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:21.995373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:22.015591 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:22.016514 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:22.017447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 347/694] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:22.023512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:22.126287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e64aed-33f8-40a0-abb9-65b06e2803a6 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:22.150196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:22.174691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c3f0d-da76-4da6-ba55-10a36710cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:22.175365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3f0d-da76-4da6-ba55-10a36710cf19 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:22.175684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3f0d-da76-4da6-ba55-10a36710cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:22.176545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c3f0d-da76-4da6-ba55-10a36710cf19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:22.177209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f2edb-690a-41b0-9b3b-75f5f8e5fd9a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:22.185699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c3f0d-da76-4da6-ba55-10a36710cf19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:22.187665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c3f0d-da76-4da6-ba55-10a36710cf19 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e95c3f0d-da76-4da6-ba55-10a36710cf19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:22.202582 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:22.203581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:22.204610 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 348/695] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:32:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:22.205225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef0263-7cd2-418d-a753-9ded7b7c459f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:22.205225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef0263-7cd2-418d-a753-9ded7b7c459f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:22.205361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef0263-7cd2-418d-a753-9ded7b7c459f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:22.207413 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 349/696] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:22.210203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef0263-7cd2-418d-a753-9ded7b7c459f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:22.223971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2f8f8-1cec-4279-a8ef-30c10a4cb3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:22.224652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f8f8-1cec-4279-a8ef-30c10a4cb3bb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:22.225261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f8f8-1cec-4279-a8ef-30c10a4cb3bb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:22.225545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f8f8-1cec-4279-a8ef-30c10a4cb3bb None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:22.226540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2f8f8-1cec-4279-a8ef-30c10a4cb3bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:22.226852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2f8f8-1cec-4279-a8ef-30c10a4cb3bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:22.228824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2f8f8-1cec-4279-a8ef-30c10a4cb3bb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:22.245426 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 348/697] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:22.251196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f91b4-1245-4ce5-9148-309a7a0bb624 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:22.251488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f91b4-1245-4ce5-9148-309a7a0bb624 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:22.251767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f91b4-1245-4ce5-9148-309a7a0bb624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:22.287087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-376f91b4-1245-4ce5-9148-309a7a0bb624 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:22.288296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-376f91b4-1245-4ce5-9148-309a7a0bb624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:22.290667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:22.348419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef0263-7cd2-418d-a753-9ded7b7c459f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:22.396485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:22.397427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:22.398229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:22.398707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:22.399767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:22.400480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:22.402484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:22.426529 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 350/698] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:32:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:22.427752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:22.464071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:22.475418 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 349/699] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:22.477382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f91b4-1245-4ce5-9148-309a7a0bb624 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:22.555836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a64dc-33b4-44d2-9c45-84816beca882 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:22.602520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:22.603694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:22.604398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:22.604894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:22.606326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=32f1fd57a99e4c0b9e50e997c5ee74f1) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:22.606889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:22.609167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:22.617013 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 351/700] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:22 2022] GET /identity/v3/role_assignments?user.id=32f1fd57a99e4c0b9e50e997c5ee74f1&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:22.721739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f91b4-1245-4ce5-9148-309a7a0bb624 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:22.767548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:22.768192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:22.768476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:22.768754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:22.782268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:22.854691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:22.856449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:22.968946 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 350/701] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:22.974674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:22.989563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:23.071472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:23.113965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:23.130516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5c1f1-1451-4fba-8398-72fdb41cf959 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:23.159674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70131-8fc3-4cef-b453-61baa6f1c74d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:23.180564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec294f-c176-4678-937d-ac7a9fff4027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:23.181291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec294f-c176-4678-937d-ac7a9fff4027 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.181640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec294f-c176-4678-937d-ac7a9fff4027 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.182027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec294f-c176-4678-937d-ac7a9fff4027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.194604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec294f-c176-4678-937d-ac7a9fff4027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:23.196698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec294f-c176-4678-937d-ac7a9fff4027 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:23.196839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9e0af-b06d-4153-8515-4a453dc32d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:23.197404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e0af-b06d-4153-8515-4a453dc32d99 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.197687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e0af-b06d-4153-8515-4a453dc32d99 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.197985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9e0af-b06d-4153-8515-4a453dc32d99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.210555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9e0af-b06d-4153-8515-4a453dc32d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:23.212364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9e0af-b06d-4153-8515-4a453dc32d99 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b6ec294f-c176-4678-937d-ac7a9fff4027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:23.214749 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:23.215790 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:23.217116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:23.217116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:23.217116 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:23.217116 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 352/702] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:23.217405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4912a2-9cfe-479a-8e66-17ba254e1930 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.217608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4912a2-9cfe-479a-8e66-17ba254e1930 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.223266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4912a2-9cfe-479a-8e66-17ba254e1930 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.225185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 353/703] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:23.226228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4912a2-9cfe-479a-8e66-17ba254e1930 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-cdb9e0af-b06d-4153-8515-4a453dc32d99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:23.235137 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:23.236391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:23.237337 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:23.237337 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:23.237337 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:23.244815 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 351/704] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:23.246352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a32c4-ffab-4e0d-b862-ec96e44d2b48 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.246643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a32c4-ffab-4e0d-b862-ec96e44d2b48 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.246926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a32c4-ffab-4e0d-b862-ec96e44d2b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.269576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-812a32c4-ffab-4e0d-b862-ec96e44d2b48 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:23.270749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-812a32c4-ffab-4e0d-b862-ec96e44d2b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:23.354485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4912a2-9cfe-479a-8e66-17ba254e1930 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:23.378449 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 352/705] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:23.380069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a32c4-ffab-4e0d-b862-ec96e44d2b48 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:23.403686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:23.404914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.405429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.405984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.407137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:23.408254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:23.408932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:23.412581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:23.427768 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 354/706] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:23.433511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:23.437090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:23.512830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a32c4-ffab-4e0d-b862-ec96e44d2b48 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:23.564893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dd912-5e15-4c10-8006-03ed9d49ce53 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:23.579744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:23.580730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.581213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.582501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.583389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:23.583949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:23.585540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:23.609604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 353/707] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:23.618039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:23.637318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4716a3-4a69-418b-9962-bcd775230fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:23.637962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4716a3-4a69-418b-9962-bcd775230fbf None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.638243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4716a3-4a69-418b-9962-bcd775230fbf None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.638522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4716a3-4a69-418b-9962-bcd775230fbf None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.639171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4716a3-4a69-418b-9962-bcd775230fbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:23.640588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4716a3-4a69-418b-9962-bcd775230fbf None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:23.687370 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 355/708] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:32:23 2022] POST /identity/v3/users => generated 254 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:32:23.688688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b426ab-571a-474e-9c0a-08e2d2b7d46c None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.689050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b426ab-571a-474e-9c0a-08e2d2b7d46c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.689329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b426ab-571a-474e-9c0a-08e2d2b7d46c None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.690972 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 356/709] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:23.697188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1791e-a49b-44a2-bba3-88f2d027d95b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.697508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1791e-a49b-44a2-bba3-88f2d027d95b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.697817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1791e-a49b-44a2-bba3-88f2d027d95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.721183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:23.738957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d6d1791e-a49b-44a2-bba3-88f2d027d95b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:23.740825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d6d1791e-a49b-44a2-bba3-88f2d027d95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:23.761372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572913b7-0044-4a3f-a944-3e22c5327111 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:23.828285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:23.829564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:23.830363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:23.830862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:23.835397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:23.845100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:23.849198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:23.860887 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 357/710] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:23.861492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1791e-a49b-44a2-bba3-88f2d027d95b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:23.869738 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:23.870725 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:23.871655 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:23.880855 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 354/711] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:23.886992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:24.005577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1791e-a49b-44a2-bba3-88f2d027d95b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.039415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41551591-5e98-4f1b-9cb2-13f4572ad9f3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.051789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.052705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.053423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.053893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] PATH_INFO: `/v3/services/network` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.054906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.057187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:24.071234 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:24.072279 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:24.073295 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 358/712] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:23 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:24.082398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:24.083765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.084386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.084676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.084955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.085654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:24.086002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.087445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.101177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:24.111306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 355/713] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:24.117796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:24.146845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:24.215850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149df3da-2703-431b-bb3d-4f04ab459272 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.237125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915283b3-0e4f-444c-9ce2-a8a5c7194408 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.263424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.264324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.264809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.265288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.266024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:24.266391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.268014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.274397 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 359/714] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:24.274789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.275324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:24.275791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.276278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.280709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.280709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=6307d15dca3043b987a28fc41515acc2) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:24.280709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.280709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.292831 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 356/715] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:24 2022] GET /identity/v3/role_assignments?user.id=6307d15dca3043b987a28fc41515acc2&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:24.294446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:24.443365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:24.462510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dae10-2c48-43e1-9998-85e0b03d58a7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.482668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:24.504369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.505381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.505930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.506453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.507259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.509082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:24.520110 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:24.521414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:24.522277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:24.524963 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 360/716] 10.222.0.33 () {56 vars in 1129 bytes} [Sat Sep 10 20:32:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:24.530913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:24.531090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ccbef-b1cd-4953-b699-55e29970662b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.603889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.604504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.604783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.605052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.605718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] RBAC: Adding query filter params (type=network) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:24.606053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.607585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.619437 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 357/717] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:24.691759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77da3c9-dcbd-4e95-a8cd-d49dff907372 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.731488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cd96f-3e6f-44ab-9bf6-4184692f8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.732020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cd96f-3e6f-44ab-9bf6-4184692f8981 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.732300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cd96f-3e6f-44ab-9bf6-4184692f8981 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.732572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cd96f-3e6f-44ab-9bf6-4184692f8981 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.733201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cd96f-3e6f-44ab-9bf6-4184692f8981 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:24.733502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cd96f-3e6f-44ab-9bf6-4184692f8981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.734910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cd96f-3e6f-44ab-9bf6-4184692f8981 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.740594 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 361/718] 10.222.0.33 () {56 vars in 1124 bytes} [Sat Sep 10 20:32:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:24.750073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:24.912369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fcfe5-84ac-4e1c-9043-684dde612970 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:24.949169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:24.960009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9987f36-be99-4657-ac12-3287de80eb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:24.963840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9987f36-be99-4657-ac12-3287de80eb54 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:24.964216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9987f36-be99-4657-ac12-3287de80eb54 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:24.964494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9987f36-be99-4657-ac12-3287de80eb54 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:24.965134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9987f36-be99-4657-ac12-3287de80eb54 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:24.965437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9987f36-be99-4657-ac12-3287de80eb54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:24.966848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9987f36-be99-4657-ac12-3287de80eb54 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:24.978263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 358/719] 10.222.0.33 () {56 vars in 1124 bytes} [Sat Sep 10 20:32:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:25.856493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77e13-c6ce-4352-aa19-c08a01b42ed2 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:25.857019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77e13-c6ce-4352-aa19-c08a01b42ed2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:25.857019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77e13-c6ce-4352-aa19-c08a01b42ed2 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:25.858666 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 362/720] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:25.864839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfd580-08b6-4ef4-95a2-026f8ec3dc09 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:25.865293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfd580-08b6-4ef4-95a2-026f8ec3dc09 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:25.865480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfd580-08b6-4ef4-95a2-026f8ec3dc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:25.891550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8edfd580-08b6-4ef4-95a2-026f8ec3dc09 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:25.892744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8edfd580-08b6-4ef4-95a2-026f8ec3dc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:26.125185 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 359/721] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:26.517984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77e13-c6ce-4352-aa19-c08a01b42ed2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:26.567497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:26.630131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77e13-c6ce-4352-aa19-c08a01b42ed2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:26.664110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:26.664733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:26.665012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:26.665282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:26.676744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:26.682097 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:26.683032 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:26.684275 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 363/722] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:26.689329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfd580-08b6-4ef4-95a2-026f8ec3dc09 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:26.792359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfd580-08b6-4ef4-95a2-026f8ec3dc09 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:26.815952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:26.828953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd4db4-efdf-4273-8813-96be6b4f0a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:26.829567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd4db4-efdf-4273-8813-96be6b4f0a0f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:26.829870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd4db4-efdf-4273-8813-96be6b4f0a0f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:26.830153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd4db4-efdf-4273-8813-96be6b4f0a0f None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:26.830827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd4db4-efdf-4273-8813-96be6b4f0a0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:26.831131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd4db4-efdf-4273-8813-96be6b4f0a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:26.832535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd4db4-efdf-4273-8813-96be6b4f0a0f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:26.842499 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 360/723] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:26.850913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:26.879294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce9c43-083b-445a-a1e4-48f3559dff9d None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:26.879957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce9c43-083b-445a-a1e4-48f3559dff9d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:26.879957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce9c43-083b-445a-a1e4-48f3559dff9d None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:26.881553 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 361/724] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:26.888061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35edd9-ffea-4294-b7cd-5db8f8af4ae8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:26.888426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35edd9-ffea-4294-b7cd-5db8f8af4ae8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:26.888689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35edd9-ffea-4294-b7cd-5db8f8af4ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:26.911052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-be35edd9-ffea-4294-b7cd-5db8f8af4ae8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:26.912310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be35edd9-ffea-4294-b7cd-5db8f8af4ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:26.998856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fe777-da35-40fb-9322-5ebd356302da None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:27.001977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:27.002182 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 362/725] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 20:32:27.049306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:27.049872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.050159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.050687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.064787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:27.066818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:27.087711 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:27.089012 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:27.089960 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:27.089960 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:27.089960 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:27.093590 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 364/726] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:27.099283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35edd9-ffea-4294-b7cd-5db8f8af4ae8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:27.220667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35edd9-ffea-4294-b7cd-5db8f8af4ae8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:27.255068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:27.271157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:27.271772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.272054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.272328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.272972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:27.273275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:27.274744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:27.291457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 363/727] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:27.297152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:27.401370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ed582-1221-4857-92a3-e08666ecc847 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:27.435030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:27.460958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:27.461603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.461918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.462198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] PATH_INFO: `/v3/users/swift` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.479353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:27.480820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:27.512398 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:27.513482 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:27.514475 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:27.518111 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 365/728] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:27.519838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:27.563150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:27.637664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441549-5ad4-40d4-9f6d-901e2c35b509 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:27.693140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12d416-13a2-4f0a-b384-96dd6a2d2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:27.693689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12d416-13a2-4f0a-b384-96dd6a2d2526 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.694024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12d416-13a2-4f0a-b384-96dd6a2d2526 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.694453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12d416-13a2-4f0a-b384-96dd6a2d2526 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.698349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682de57f-3be7-4158-976a-57e49cbc032a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:27.707945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12d416-13a2-4f0a-b384-96dd6a2d2526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6b12d416-13a2-4f0a-b384-96dd6a2d2526 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:27.715578 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:27.716535 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:27.718045 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 364/729] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:27.718749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f50ba-458f-464d-ae01-73082cf4aa73 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.718749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f50ba-458f-464d-ae01-73082cf4aa73 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.718878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f50ba-458f-464d-ae01-73082cf4aa73 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.724657 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 365/730] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:27.724657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f50ba-458f-464d-ae01-73082cf4aa73 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:27.745601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1efc3-0227-4cf5-a04f-3eda424d993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:27.746483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1efc3-0227-4cf5-a04f-3eda424d993c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.746945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1efc3-0227-4cf5-a04f-3eda424d993c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.747424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1efc3-0227-4cf5-a04f-3eda424d993c None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.748419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1efc3-0227-4cf5-a04f-3eda424d993c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:27.748924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1efc3-0227-4cf5-a04f-3eda424d993c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:27.750648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1efc3-0227-4cf5-a04f-3eda424d993c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:27.769896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:27.774463 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 366/731] 10.222.0.33 () {56 vars in 1135 bytes} [Sat Sep 10 20:32:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:27.776055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a989190-e75e-415a-8797-af0b68771438 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.776506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a989190-e75e-415a-8797-af0b68771438 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.776942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a989190-e75e-415a-8797-af0b68771438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.799315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0a989190-e75e-415a-8797-af0b68771438 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:27.801197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a989190-e75e-415a-8797-af0b68771438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:27.859758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f50ba-458f-464d-ae01-73082cf4aa73 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:27.891106 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 367/732] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:27.894767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a989190-e75e-415a-8797-af0b68771438 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:27.906458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:27.910512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:27.911760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:27.912255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:27.912740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:27.914299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:27.914822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:27.916408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:27.927828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 366/733] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:27.933359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.055202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a989190-e75e-415a-8797-af0b68771438 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.060357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.068735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d083d-08b8-453e-8f97-5aa29e992565 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.093719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2e1dc-9465-405d-b6d3-c672e6803ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.094520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e1dc-9465-405d-b6d3-c672e6803ddd None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.094803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e1dc-9465-405d-b6d3-c672e6803ddd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.095071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e1dc-9465-405d-b6d3-c672e6803ddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.105708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8d8d-04ba-4996-ae26-2f7e62d3e0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.106364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8d8d-04ba-4996-ae26-2f7e62d3e0de None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.106649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8d8d-04ba-4996-ae26-2f7e62d3e0de None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.106982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8d8d-04ba-4996-ae26-2f7e62d3e0de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.117927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced8d8d-04ba-4996-ae26-2f7e62d3e0de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.119373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced8d8d-04ba-4996-ae26-2f7e62d3e0de None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-fced8d8d-04ba-4996-ae26-2f7e62d3e0de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:28.133954 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:28.134981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:28.135976 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:28.135976 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:28.135976 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:28.139731 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 367/734] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:28.140739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382354e-5cc1-423b-ad04-0b809a7ec21f None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.141017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382354e-5cc1-423b-ad04-0b809a7ec21f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.141299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382354e-5cc1-423b-ad04-0b809a7ec21f None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.142680 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 368/735] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:28.149750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0350a45-e926-4af0-b031-bc74966acdd1 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.149750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0350a45-e926-4af0-b031-bc74966acdd1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.149750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0350a45-e926-4af0-b031-bc74966acdd1 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.149750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 369/736] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:28.149750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0350a45-e926-4af0-b031-bc74966acdd1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.179563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2e1dc-9465-405d-b6d3-c672e6803ddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.181011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2e1dc-9465-405d-b6d3-c672e6803ddd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.183799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.257429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0350a45-e926-4af0-b031-bc74966acdd1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.288305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 368/737] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 20:32:28.288305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30778db-e10e-4b1b-881a-9acf5d4a337a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.288305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30778db-e10e-4b1b-881a-9acf5d4a337a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.288305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30778db-e10e-4b1b-881a-9acf5d4a337a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.297148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ca5dd-7db7-4e00-8293-364c2231d246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.297753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ca5dd-7db7-4e00-8293-364c2231d246 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.298072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ca5dd-7db7-4e00-8293-364c2231d246 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.298352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ca5dd-7db7-4e00-8293-364c2231d246 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.298993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ca5dd-7db7-4e00-8293-364c2231d246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:28.299306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ca5dd-7db7-4e00-8293-364c2231d246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.300735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ca5dd-7db7-4e00-8293-364c2231d246 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.304289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c30778db-e10e-4b1b-881a-9acf5d4a337a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:28.305546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c30778db-e10e-4b1b-881a-9acf5d4a337a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:28.312855 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 370/738] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:28.314104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bdb6d-7084-4b9f-af63-84f753835fb0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.314389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bdb6d-7084-4b9f-af63-84f753835fb0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.314660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bdb6d-7084-4b9f-af63-84f753835fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.332701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0f6bdb6d-7084-4b9f-af63-84f753835fb0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:28.333904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f6bdb6d-7084-4b9f-af63-84f753835fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:28.335832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.374266 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 369/739] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:28.375106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30778db-e10e-4b1b-881a-9acf5d4a337a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.406181 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 371/740] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 20:32:28.408948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bdb6d-7084-4b9f-af63-84f753835fb0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.418292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.503086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30778db-e10e-4b1b-881a-9acf5d4a337a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.527258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bdb6d-7084-4b9f-af63-84f753835fb0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.540614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.541374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.541758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.542176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.553107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:28.558249 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:28.559422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:28.560476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:28.562474 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 370/741] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:28.563451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.576351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.577601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.578217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.578780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.588995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.595321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.598214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:28.619021 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:28.620228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:28.621338 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:28.621338 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:28.621338 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:28.623464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 372/742] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:28.624870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.683675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.693557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17977d41-1f4d-4eb5-bf6d-a2db701d25eb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.734081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5d49f-acce-4104-b2ee-9763d96937b6 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.736132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.737552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.737887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.738422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.745635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.747105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:28.758658 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:28.759686 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 371/743] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:32:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:28.760640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.772441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.773044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.773321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.773593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.774287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:28.774587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.775943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.784367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 373/744] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:28.785654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.837390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.861179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558675a-ce5f-4a8f-b4d0-2890267bde8a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.885784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac6afd-351a-4115-b9b3-513259ec6765 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:28.893512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.894825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.895120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.895461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.896097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:28.896398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.897797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.909462 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 372/745] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:28.911129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:28.922697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:28.923304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:28.923586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:28.923863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:28.924505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:28.925956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:28.926495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:28.940638 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 374/746] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:32:28 2022] POST /identity/v3/services => generated 240 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:28.941493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.031630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4160882-bb7a-46d7-9690-f442f67e1959 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.076509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eca71a-0c96-459e-a088-9b4f1a2a8664 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.085137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.085758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.086111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.086356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.087226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.088260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:29.088925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.113936 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 373/747] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:32:28 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:29.113936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.133019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.134072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.134512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.135040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.136018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:29.136486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.138511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.161083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 375/748] 10.222.0.33 () {56 vars in 1149 bytes} [Sat Sep 10 20:32:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:29.162070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.223510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:29.270270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5d54d-8208-461e-9fbe-dc43f29067fb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.288898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0962eb-8cc9-4474-8fb4-37413c0b6064 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.314177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.315109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.315541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.316513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.329229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.331014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.343273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.343884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.344168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.344443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:29.350217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:29.351366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:29.352414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:29.352414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:29.352414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:29.352588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.354118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.354290 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 374/749] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:29.359051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:29.368034 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:29.368988 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:29.369978 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 376/750] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:29.370527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.398538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:29.492775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad194a-e7fa-4b33-83c9-eef3f0611e04 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.498188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fbd94-144a-4d72-ae6e-7f46753e0a26 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.521490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:29.534313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.535107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.535398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.535675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.536328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:29.536656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.538042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.538344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.539188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.539714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.540164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.553908 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 377/751] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:29.554281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.632234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.634052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.658608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:29.681809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06821461-3fbd-4965-a113-635aeb7e27eb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.727639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.728720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.729011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.729284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.730012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:29.730321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.732482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.748620 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 378/752] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:29.750026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.763944 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 375/753] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:32:29.764912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.791962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:29.872500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03269c9-e333-4622-b94a-86bb07ef434d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.881261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20879b-a277-490c-ab45-4a6adf925f44 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:29.919959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.920489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.920773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.921070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.921857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=32f1fd57a99e4c0b9e50e997c5ee74f1) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:29.922188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.923271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:29.923720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.924561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:29.925052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:29.925921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:29.929729 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 376/754] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/role_assignments?user.id=32f1fd57a99e4c0b9e50e997c5ee74f1&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:29.930729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.937642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:29.940303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:29.958186 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:29.959228 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:29.960196 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:29.963779 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 379/755] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:29.979667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:29.990837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:30.102457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:30.134681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4d90e-56eb-4871-921c-a9c995266d8b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.163235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a6bbe-9a32-4f85-8cbe-50c5fb4db2a2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.173145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.174052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.174181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.174388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.187273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.188712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:30.205721 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:30.207419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:30.208528 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:30.208528 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:30.208528 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:30.210426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.211282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.211405 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 377/756] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:30.211871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.212340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.212713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:30.213889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:30.214417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.216221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.238588 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 380/757] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:32:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:30.256787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:30.327611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6ebca-d2e2-49f0-ac96-408c9df29ada None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.333430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:30.369393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.370284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.370708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.371140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.371929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:30.372419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.374075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.382863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:30.393659 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 378/758] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:30.395782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:30.411412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ab4d-d1c8-43f9-9e2d-62836d0548c1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.486067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.486604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.486888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.487159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.506338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284d54b-dd49-4e52-99a1-55a43248bcb2 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.506542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.508045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:30.536781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:30.537805 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:30.538741 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:30.538741 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:30.538741 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:30.546204 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 381/759] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:30.547022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.547205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:30.548192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.548657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.549121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.557607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.559427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:30.572423 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:30.573378 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:30.574329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:30.581429 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 379/760] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:30.582983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:30.648071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:30.695546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8a765-5fc7-4c56-ad91-7f18318156d2 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.736734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.737799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.738769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.739804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.740867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:30.742922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a97361-71d9-4ad1-bddb-41dae061ab0d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.743434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.745086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.770031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 380/761] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:30.772247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:30.778667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:30.808716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.809662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.810181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.811163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.812157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:30.812659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.814324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.830121 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 382/762] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:30.840308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:30.918125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452c3f2-6c11-4d96-9d86-07c666d9fe3d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:30.936088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:30.955836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:30.956832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:30.957280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:30.957706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:30.958657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:30.959107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:30.960678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:30.969340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 381/763] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:30 2022] GET /identity/v3/role_assignments?user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:31.053381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e848f7-6289-4009-9f92-73150eff5c25 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:31.105221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:31.105874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:31.106161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:31.106439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:31.119171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:31.120646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:31.133218 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:31.134185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:31.135129 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:31.140062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 383/764] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:31.146598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:31.316280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581236d8-f9ce-4b11-9036-2f52d4389a39 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:31.354354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2fc6b-c89b-4423-8edc-d1b9a4e0bb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:31.355026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2fc6b-c89b-4423-8edc-d1b9a4e0bb8d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:31.355318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2fc6b-c89b-4423-8edc-d1b9a4e0bb8d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:31.355648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2fc6b-c89b-4423-8edc-d1b9a4e0bb8d None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:31.356409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2fc6b-c89b-4423-8edc-d1b9a4e0bb8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:31.356714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2fc6b-c89b-4423-8edc-d1b9a4e0bb8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:31.358396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2fc6b-c89b-4423-8edc-d1b9a4e0bb8d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:31.371910 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 382/765] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:31.377803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:31.428798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:31.525761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456b500-c79c-4e4a-8a0e-8b7c6e5a8b79 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:31.577049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aebab2-e0ed-4a81-bb49-324aa1aa65c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:31.577659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aebab2-e0ed-4a81-bb49-324aa1aa65c5 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:31.577972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aebab2-e0ed-4a81-bb49-324aa1aa65c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:31.578253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aebab2-e0ed-4a81-bb49-324aa1aa65c5 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/6307d15dca3043b987a28fc41515acc2/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:31.580319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aebab2-e0ed-4a81-bb49-324aa1aa65c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=6307d15dca3043b987a28fc41515acc2, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:31.582607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aebab2-e0ed-4a81-bb49-324aa1aa65c5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:31.604033 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 384/766] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:31 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/6307d15dca3043b987a28fc41515acc2/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:31.895264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81031bb-e6d7-4a6a-be40-2a2d0ae7ece3 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:31.895609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81031bb-e6d7-4a6a-be40-2a2d0ae7ece3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:31.895789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81031bb-e6d7-4a6a-be40-2a2d0ae7ece3 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:31.897318 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 383/767] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:31.902806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef46a45-a25d-4abc-837c-3b893f8e03a6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:31.903093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef46a45-a25d-4abc-837c-3b893f8e03a6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:31.903370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef46a45-a25d-4abc-837c-3b893f8e03a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:31.923072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0ef46a45-a25d-4abc-837c-3b893f8e03a6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:31.924269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ef46a45-a25d-4abc-837c-3b893f8e03a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:32.017081 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 385/768] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:32.068531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d522-e957-431b-a652-42aeecfd79e2 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:32.068843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d522-e957-431b-a652-42aeecfd79e2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:32.069087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10d522-e957-431b-a652-42aeecfd79e2 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:32.070843 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 384/769] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:32.076258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a98e20-e6cb-4043-8e8e-06d4e98e91e6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:32.076697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a98e20-e6cb-4043-8e8e-06d4e98e91e6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:32.076869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a98e20-e6cb-4043-8e8e-06d4e98e91e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:32.106872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-91a98e20-e6cb-4043-8e8e-06d4e98e91e6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:32.108082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-91a98e20-e6cb-4043-8e8e-06d4e98e91e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:32.129125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:32.201684 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 386/770] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:32.467677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d522-e957-431b-a652-42aeecfd79e2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:32.524254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:32.601779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10d522-e957-431b-a652-42aeecfd79e2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:32.602479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a98e20-e6cb-4043-8e8e-06d4e98e91e6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:32.657484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:32.658165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:32.658451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:32.658728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] PATH_INFO: `/v3/services/network` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:32.659367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:32.661262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:32.671770 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:32.673197 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:32.674227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 385/771] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:32 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:32.679315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:32.796333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a98e20-e6cb-4043-8e8e-06d4e98e91e6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:32.822983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b677c2e-f750-4ad9-ae26-43b42835d034 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:32.830387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:32.866964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:32.867580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:32.867862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:32.868136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:32.868776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] RBAC: Adding query filter params (name=network) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:32.869081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:32.870472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:32.873670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:32.874337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:32.874617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:32.874893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:32.875474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:32.876822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:32.885970 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 386/772] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:32.890192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:32.894272 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:32.895764 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:32.897257 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 387/773] 10.222.0.33 () {56 vars in 1129 bytes} [Sat Sep 10 20:32:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:32.905651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:32.926339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:33.032837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef4252-91a9-4da5-b1d9-eff734e1c9d6 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.033048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874500f-7da4-4b8f-a116-3955539d62da None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.074749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c556-2a7d-4bb8-a3c0-4a19d458beaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.075513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c556-2a7d-4bb8-a3c0-4a19d458beaa None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.075898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c556-2a7d-4bb8-a3c0-4a19d458beaa None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.076275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c556-2a7d-4bb8-a3c0-4a19d458beaa None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.077010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c556-2a7d-4bb8-a3c0-4a19d458beaa None admin] RBAC: Adding query filter params (type=network) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:33.077581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c556-2a7d-4bb8-a3c0-4a19d458beaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:33.079876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c556-2a7d-4bb8-a3c0-4a19d458beaa None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:33.084082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.084721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.085004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.085282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.086252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:33.086560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:33.088025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:33.088152 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 387/774] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:33.089889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e3e1-6092-4620-9405-746e4f155725 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.090285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e3e1-6092-4620-9405-746e4f155725 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.090662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e3e1-6092-4620-9405-746e4f155725 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.092382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 388/775] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:33.093576 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 388/776] 10.222.0.33 () {56 vars in 1124 bytes} [Sat Sep 10 20:32:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:33.098946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:33.099281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15efa6-d801-4839-8a1c-0e59c40b673a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.099687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15efa6-d801-4839-8a1c-0e59c40b673a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.100020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15efa6-d801-4839-8a1c-0e59c40b673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.118732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4f15efa6-d801-4839-8a1c-0e59c40b673a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:33.119924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4f15efa6-d801-4839-8a1c-0e59c40b673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:33.148065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:33.190133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:33.201398 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 389/777] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:33.203307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15efa6-d801-4839-8a1c-0e59c40b673a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:33.217395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3e473-de62-428a-b048-aa0ee0825e80 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.260805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.261422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.261703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.262009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.262715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] RBAC: Adding query filter params (interface=public, service_id=4631ea41e11c4c0d9961dfa698b630f4, region_id=RegionOne) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:33.263025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:33.264396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:33.281628 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 389/778] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:32:33 2022] GET /identity/v3/endpoints?service_id=4631ea41e11c4c0d9961dfa698b630f4&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:33.294864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:33.356881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15efa6-d801-4839-8a1c-0e59c40b673a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.398460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.399909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.400345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.400766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.401555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:33.402028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:33.403637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:33.410471 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 390/779] 10.222.0.33 () {56 vars in 1124 bytes} [Sat Sep 10 20:32:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:33.416685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:33.470789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:33.504179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935f35-8656-4ccc-b2d1-3e56b1d04f09 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.573086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.573694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.574024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.574299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.575403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:33.577036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:33.583827 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 390/780] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:32:33 2022] GET /identity/v3/services => generated 772 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:33.604004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac0aea-bb84-4fd2-82ca-514392a7222c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.625217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:33.641917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:33.658926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.659556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.659890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.660168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.660892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] RBAC: Adding query filter params (interface=public, service_id=156b192efa3d4b03b8f99d6e0418cd43, region_id=RegionOne) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:33.661198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:33.662605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:33.676277 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 391/781] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:32:33 2022] GET /identity/v3/endpoints?service_id=156b192efa3d4b03b8f99d6e0418cd43&interface=public®ion_id=RegionOne => generated 194 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:33.682298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:33.837706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bf279-2a5a-4e61-98a7-a20582631f08 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.866057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:33.880245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb5214-e2bd-4f37-8ea1-1d503f2d86fc None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:33.886364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.887281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.887767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.888709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.903596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:33.910645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:33.911631 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:33.912498 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:33.915728 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 391/782] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:33.922392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:33.927203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:33.927826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:33.928952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:33.929261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:33.929949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:33.931380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:33.941340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 392/783] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:32:33 2022] GET /identity/v3/services => generated 772 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:34.013989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:34.054880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231eb76-4b8a-4b35-8572-22c72893327d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:34.095991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753d393-d73e-4c57-be8f-d2227f78b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:34.096682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d393-d73e-4c57-be8f-d2227f78b852 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.096926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d393-d73e-4c57-be8f-d2227f78b852 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.097240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d393-d73e-4c57-be8f-d2227f78b852 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.098045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753d393-d73e-4c57-be8f-d2227f78b852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:34.098271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753d393-d73e-4c57-be8f-d2227f78b852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:34.099724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753d393-d73e-4c57-be8f-d2227f78b852 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:34.110015 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 392/784] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:34.122132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:34.276013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabf5ee-14fb-4da6-80a7-1294c3490aee None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:34.317884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:34.318506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.318773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.319046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.331247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcd59d-f20a-4a91-b875-67b30fcaee74 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.332075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcd59d-f20a-4a91-b875-67b30fcaee74 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.332342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcd59d-f20a-4a91-b875-67b30fcaee74 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.332588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:34.333955 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 393/785] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:34.334318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:34.339807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbabc1-31ca-4922-81ff-c475e1363714 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.340093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbabc1-31ca-4922-81ff-c475e1363714 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.340372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbabc1-31ca-4922-81ff-c475e1363714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:34.357719 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:34.358785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:34.360324 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:34.360324 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:34.360324 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:34.360324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 393/786] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:34.360585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-13cbabc1-31ca-4922-81ff-c475e1363714 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:34.361923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-13cbabc1-31ca-4922-81ff-c475e1363714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:34.364768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:34.384394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:34.461440 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 394/787] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:34.499413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee866fb5-6697-4157-aace-c70fe20aaebe None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:34.534754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c4431-126e-4686-907d-414e93e0fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:34.535261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4431-126e-4686-907d-414e93e0fefe None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.535539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4431-126e-4686-907d-414e93e0fefe None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.535814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4431-126e-4686-907d-414e93e0fefe None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.536455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c4431-126e-4686-907d-414e93e0fefe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:34.536754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c4431-126e-4686-907d-414e93e0fefe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:34.538258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c4431-126e-4686-907d-414e93e0fefe None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:34.551041 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 394/788] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:34.556592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbabc1-31ca-4922-81ff-c475e1363714 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:34.570668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca230d-fee8-4413-942a-813182037ca1 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.570969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca230d-fee8-4413-942a-813182037ca1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.571211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca230d-fee8-4413-942a-813182037ca1 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.572770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 395/789] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:32:34.577693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceef15-93e0-41ac-b22f-9ae598879366 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.578013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceef15-93e0-41ac-b22f-9ae598879366 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.578289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fceef15-93e0-41ac-b22f-9ae598879366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.615056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8fceef15-93e0-41ac-b22f-9ae598879366 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:34.616263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8fceef15-93e0-41ac-b22f-9ae598879366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:34.626282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:34.708594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbabc1-31ca-4922-81ff-c475e1363714 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:34.722217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:34.729469 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 396/790] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:32:34.764439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:34.765077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:34.765406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:34.765681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] PATH_INFO: `/v3/users/swift` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:34.777043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:34.778520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:34.795008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:34.795981 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:34.796939 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 395/791] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:34.803019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceef15-93e0-41ac-b22f-9ae598879366 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:34.935695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:34.952230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fceef15-93e0-41ac-b22f-9ae598879366 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.019736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.020709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.021173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.021661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.022589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:35.023958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.025637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.062076 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 397/792] 10.222.0.33 () {56 vars in 1135 bytes} [Sat Sep 10 20:32:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:35.068063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.124607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adf023-7e6e-4a7a-94b4-c105e128381d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.161249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:35.170882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:35.181334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.182185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.182569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.182947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.196765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:35.202610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:35.203740 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:35.204784 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:35.207242 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 396/793] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:35.208257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.243169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020233c-1f99-4d14-af99-efc2fc265b0b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.281408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.282309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.282588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.282855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.294133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.295558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:35.312979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:35.314192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:35.315244 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:35.315244 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:35.315244 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:35.319596 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 398/794] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:35.320618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.327731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2c69-d21c-45f1-b425-de516d021c32 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.366567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.367564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.368070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.368488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.381882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:35.387757 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:35.388861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:35.389908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:35.392343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 397/795] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:35.393500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.432757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41c00b-3b83-4069-b113-2e876c836676 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.443920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:35.451374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:35.474323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.475197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.475484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.475759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.476411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:35.476712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.478292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.492345 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 399/796] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:35.493205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.523101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da99c31-4463-4cff-bf09-f1de698afcc8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.562484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.563538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.564068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.564542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.565484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:35.566661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.568234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.582463 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 398/797] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:35.583761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.611678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9836a8b-b0ea-4b18-b601-f6e897694a1a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.656208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.656836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.657376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.657376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.658045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:35.658349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.659809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.670087 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 400/798] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:35.670957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.711427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:35.723533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:35.729106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1ce43-6e1e-4d20-bd74-db47c68fdc41 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.772445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.773428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.773947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.774424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.788396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.789929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:35.805068 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:35.806107 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:35.807012 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:35.807012 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:35.807012 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:35.807012 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 399/799] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:35.807750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.812382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2e106-a421-444a-93ce-6f6c0a753543 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.864984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.866016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.866534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.867013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.874884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.876725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:35.889979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:35.890985 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:35.891910 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:35.895107 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 401/800] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:35.896239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:35.922875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a90152-44eb-4d74-92ce-5f8490480f1c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:35.962284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:35.962978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:35.963294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:35.963844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:35.976791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:35.978828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:35.995382 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:35.996433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:35.997343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:35.997343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:35.997343 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:35.997343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 400/801] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:35.997576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.001416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:36.016257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:36.027088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c2741-f7b3-4152-a220-c24cc8e15426 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.065443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.067030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.067552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.068372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.069496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:36.070108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.072295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.086669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 402/802] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:36.087935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.121554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7197-671f-41ba-9613-ad62defbbe63 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.163893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.164531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.164824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.165048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.165808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:36.166146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.167707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.182784 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 401/803] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:36.183652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.208353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ced7-a4a4-466c-9b23-392cd2707928 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.254407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.255027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.255312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.255586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.257134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:36.257416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:36.258405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.260298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.279830 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 403/804] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:36.279830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.293277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:36.314322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe1c3-7683-4178-856d-f2f6dcaef0fd None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.357128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.358350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.358863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.359379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.368036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.370175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:36.384912 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:36.385925 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:36.386896 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:36.390636 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 402/805] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:32:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:36.392323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.417141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc9d2d-3baf-4a5b-bdd5-ecb8dd06bf0e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.465148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.466242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.466769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.467255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/32f1fd57a99e4c0b9e50e997c5ee74f1/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.469492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=32f1fd57a99e4c0b9e50e997c5ee74f1, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.471563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.491585 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 404/806] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:36 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/32f1fd57a99e4c0b9e50e997c5ee74f1/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:36.493051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.536913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73142e5-a8d2-40ef-bc64-22ebf1cc448c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.572164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:36.587677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65721d-94f3-4164-ac59-cac6758ddb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.588757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65721d-94f3-4164-ac59-cac6758ddb7c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.589013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65721d-94f3-4164-ac59-cac6758ddb7c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.589291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65721d-94f3-4164-ac59-cac6758ddb7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.599758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65721d-94f3-4164-ac59-cac6758ddb7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.601381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65721d-94f3-4164-ac59-cac6758ddb7c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.604304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-3a65721d-94f3-4164-ac59-cac6758ddb7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:36.619339 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:36.620366 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:32:36.621321 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:36.626694 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 403/807] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:32:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:36.629155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efddd67-7fb7-4472-beda-81c6e6e2c6be None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.634238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efddd67-7fb7-4472-beda-81c6e6e2c6be None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.634878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efddd67-7fb7-4472-beda-81c6e6e2c6be None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.636996 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 404/808] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:36.638185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efddd67-7fb7-4472-beda-81c6e6e2c6be None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.654951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df732fbb-80b7-4a79-a88a-14ec0a7a76d7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.708150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a231e7b-d2c7-4cce-9da0-093e554d058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.708766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a231e7b-d2c7-4cce-9da0-093e554d058c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.709053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a231e7b-d2c7-4cce-9da0-093e554d058c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.709332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a231e7b-d2c7-4cce-9da0-093e554d058c None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.710131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a231e7b-d2c7-4cce-9da0-093e554d058c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:36.710444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a231e7b-d2c7-4cce-9da0-093e554d058c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.711929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a231e7b-d2c7-4cce-9da0-093e554d058c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.739541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 405/809] 10.222.0.33 () {56 vars in 1149 bytes} [Sat Sep 10 20:32:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:36.741193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef33533-bdec-44d9-81c9-7d3de8ea99e1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.741650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef33533-bdec-44d9-81c9-7d3de8ea99e1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.742653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef33533-bdec-44d9-81c9-7d3de8ea99e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.768064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0ef33533-bdec-44d9-81c9-7d3de8ea99e1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:36.769897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ef33533-bdec-44d9-81c9-7d3de8ea99e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:36.783397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efddd67-7fb7-4472-beda-81c6e6e2c6be None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:36.826966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:36.827879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:36.828367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:36.828812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:36.829706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:36.830236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:36.833268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:36.858980 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 405/810] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:32:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:36.864498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.871791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 406/811] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:36.872762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef33533-bdec-44d9-81c9-7d3de8ea99e1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:36.947483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.011502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.041208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1a5af-33ae-4743-ba7c-014433a2b3fc None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.056783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef33533-bdec-44d9-81c9-7d3de8ea99e1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.079050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f28a2-bbe1-4bd0-a101-fd643bae2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.080176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f28a2-bbe1-4bd0-a101-fd643bae2b22 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.080845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f28a2-bbe1-4bd0-a101-fd643bae2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.081300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f28a2-bbe1-4bd0-a101-fd643bae2b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.094642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db41fc4-e6a1-409b-ada3-b055d245dbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.095256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db41fc4-e6a1-409b-ada3-b055d245dbe3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.095540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db41fc4-e6a1-409b-ada3-b055d245dbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.095813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db41fc4-e6a1-409b-ada3-b055d245dbe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.108290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f28a2-bbe1-4bd0-a101-fd643bae2b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.113355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f28a2-bbe1-4bd0-a101-fd643bae2b22 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-826f28a2-bbe1-4bd0-a101-fd643bae2b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:37.133017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:37.134333 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:37.135438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:37.135438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:37.135438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:37.139131 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 406/812] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:37.140436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9fb60-e068-408f-9147-e30a80b4b76a None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.140719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9fb60-e068-408f-9147-e30a80b4b76a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.140997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9fb60-e068-408f-9147-e30a80b4b76a None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.142553 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 407/813] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:37.143263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9fb60-e068-408f-9147-e30a80b4b76a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.186154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db41fc4-e6a1-409b-ada3-b055d245dbe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.187930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db41fc4-e6a1-409b-ada3-b055d245dbe3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.255807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.271455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.282921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9fb60-e068-408f-9147-e30a80b4b76a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.301894 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 407/814] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:32:37.303481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc009-6750-4b78-b9fb-cb5d1bb11aad None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.303772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc009-6750-4b78-b9fb-cb5d1bb11aad None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.304050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc009-6750-4b78-b9fb-cb5d1bb11aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.323429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9a7bc009-6750-4b78-b9fb-cb5d1bb11aad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:37.324712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9a7bc009-6750-4b78-b9fb-cb5d1bb11aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:37.326196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.326812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.327098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.327374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.328021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:37.328326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.329730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.346869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 408/815] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:37.348296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.403633 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 408/816] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:37.404592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc009-6750-4b78-b9fb-cb5d1bb11aad None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.476570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783bd5e-2517-4bac-b83c-0ba93db821aa None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.516782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.518048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.518780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.520004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] PATH_INFO: `/v3/services/network` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.521098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.522674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.526262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:37.537013 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:37.538059 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 20:32:37.538942 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:37.539688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc009-6750-4b78-b9fb-cb5d1bb11aad None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.547848 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 409/817] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:37.548871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.574348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.591052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.592189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.592671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.593117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.608553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.610784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:37.631669 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:37.632881 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:37.634043 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:37.634043 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:37.634043 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:37.640971 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 409/818] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:37.640971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.677859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cfe69-44f5-46d0-a2b7-55ab4ca850a1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.722417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.724417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.724890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.725313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.735178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.738156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:37.755309 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:37.756391 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:37.757487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:37.757971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ec512-0a03-4097-82f8-91a4655e2f36 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.760396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 410/819] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:37.762145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.795837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.796444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.796830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.797209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.797989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] RBAC: Adding query filter params (name=network) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:37.798300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.799674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.804640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 410/820] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:37.805453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.816386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.855946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:37.879425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780173a-c516-4af4-b364-6b06559153b3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.916880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.917494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.917777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.918080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.918724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69918f-97fe-4f68-a3e5-edd372dc445d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:37.918824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:37.919028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.920562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.934823 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 411/821] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:37.936207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:37.958248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:37.958864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:37.959154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:37.959430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:37.960122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:37.960430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:37.964328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:37.978130 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 411/822] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:37.979442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.052264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:38.057201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff589bd8-8796-4d95-8bd0-156eaad7f94d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.088045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25647820-144b-4cd5-8d90-15b68c840ba3 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.095708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.096344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.096899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.096899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.097556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.098963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:38.107963 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:38.108979 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:38.114664 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 412/823] 10.222.0.33 () {56 vars in 1129 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:38.115511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.117962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:38.133390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.134118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.134484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.134830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.135527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] RBAC: Adding query filter params (type=network) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:38.135903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.137328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.142433 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 412/824] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:38.148004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.223424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9c42e-38fc-40c7-98ba-0acacf9e7777 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.260068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.260693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.260977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.261140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9eac4-9bbc-4c68-a14d-50acaa65eb0e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.261241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.267951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.269610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:38.280882 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:38.282250 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 413/825] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:38.283139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.296285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:38.303763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.304717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.305093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.305700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/dcb3cf5d0ccd4ee2a1f71963dbcdfb18/roles/34d4065b512e49b883776816da214951` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.324050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18, role_id=34d4065b512e49b883776816da214951)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.325583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.345860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 413/826] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:38 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/dcb3cf5d0ccd4ee2a1f71963dbcdfb18/roles/34d4065b512e49b883776816da214951 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:32:38.345860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.369709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:38.416942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff16dcd-63c1-4f4f-bf9e-b8d64e324d7a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.456171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.457522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.457842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.458128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.458777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:38.459133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.460553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.470949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 414/827] 10.222.0.33 () {56 vars in 1124 bytes} [Sat Sep 10 20:32:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:38.470949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.480336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f6903-32b4-4445-a860-13a473d9fcba None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.522411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.523094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.523378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.523652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.524345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.525914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.556815 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 414/828] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:32:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:38.558044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.621762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685278-6e99-4a68-bb9d-a8b667a6248d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.623104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:38.658907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:38.678911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.679947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.680488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.681011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.682075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:38.682651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.684438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.701606 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 415/829] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:38.710346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.711906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fc7b4-e893-4bff-8c6d-c3f15cde7276 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.759185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.759858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.760074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.760347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.760991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:38.761298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.763439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.771032 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 415/830] 10.222.0.33 () {56 vars in 1124 bytes} [Sat Sep 10 20:32:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:38.783898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:38.876055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1829ff7-d3f3-4005-bbdc-500deae5871a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.926189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ed37-40ab-485a-9a67-d9ba521fb30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:38.927336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ed37-40ab-485a-9a67-d9ba521fb30a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:38.927766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ed37-40ab-485a-9a67-d9ba521fb30a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:38.928204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ed37-40ab-485a-9a67-d9ba521fb30a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:38.929248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480ed37-40ab-485a-9a67-d9ba521fb30a None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=6307d15dca3043b987a28fc41515acc2) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:38.929723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480ed37-40ab-485a-9a67-d9ba521fb30a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:38.932071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480ed37-40ab-485a-9a67-d9ba521fb30a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:38.939464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 416/831] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:38 2022] GET /identity/v3/role_assignments?user.id=6307d15dca3043b987a28fc41515acc2&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:38.961657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a85d-6808-4008-ae3e-d9d25f2ec596 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:38.971827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:39.007147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f10e7-102b-4c1c-88c2-5e4d5f7d44ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:39.008178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f10e7-102b-4c1c-88c2-5e4d5f7d44ac None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:39.008710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f10e7-102b-4c1c-88c2-5e4d5f7d44ac None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:39.009188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f10e7-102b-4c1c-88c2-5e4d5f7d44ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:39.010193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f10e7-102b-4c1c-88c2-5e4d5f7d44ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:39.011828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f10e7-102b-4c1c-88c2-5e4d5f7d44ac None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:39.038256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 416/832] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:32:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:39.194779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959f9db-d5d7-4ab6-bad1-c80565387cd5 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:39.194952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959f9db-d5d7-4ab6-bad1-c80565387cd5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:39.195233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959f9db-d5d7-4ab6-bad1-c80565387cd5 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:39.197121 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 417/833] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:39.204613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dc76c-33f7-4264-8dba-fa1beb99d6bd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:39.204931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dc76c-33f7-4264-8dba-fa1beb99d6bd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:39.205175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dc76c-33f7-4264-8dba-fa1beb99d6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:39.227820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-632dc76c-33f7-4264-8dba-fa1beb99d6bd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:39.229394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-632dc76c-33f7-4264-8dba-fa1beb99d6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:39.309422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 417/834] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:39.646663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959f9db-d5d7-4ab6-bad1-c80565387cd5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:39.704818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:39.763733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959f9db-d5d7-4ab6-bad1-c80565387cd5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:39.800989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:39.801689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:39.802008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:39.802284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:39.813288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:39.818433 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:39.819344 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:39.820199 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 418/835] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:39.825563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632dc76c-33f7-4264-8dba-fa1beb99d6bd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:39.911406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:39.930594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632dc76c-33f7-4264-8dba-fa1beb99d6bd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:39.966832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:39.967832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:39.968315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:39.968788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:39.969660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:39.973203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:39.973203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:39.985757 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 418/836] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:39.993857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:40.096757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50876-08ca-46ce-b256-41fe604890ed None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:40.104962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:40.134584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:40.135257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:40.135544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:40.135813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:40.146603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:40.148152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:40.163285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:40.164213 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:40.165120 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:40.165120 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:40.165120 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:40.169018 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 419/837] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:40.170394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:40.280375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cd0d9-70d4-4471-a47b-f99506aa37fb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:40.318035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:40.319043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:40.319528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:40.320002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:40.320851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:40.321356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:40.323378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:40.337183 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 419/838] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:40.344280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:40.466209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea921a-ee6b-40cf-8202-00f677595318 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:40.502249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:40.502881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:40.503704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:40.504117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:40.512346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:40.513809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:40.526111 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:40.527076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:40.528027 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 420/839] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:32:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:40.533190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:40.537479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:40.647949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd9afd-e114-4988-8a19-e4636910ef1b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:40.683589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe13cc6-5dce-4874-8989-07739b8c3592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:40.684552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe13cc6-5dce-4874-8989-07739b8c3592 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:40.685038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe13cc6-5dce-4874-8989-07739b8c3592 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:40.685513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe13cc6-5dce-4874-8989-07739b8c3592 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:40.686497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe13cc6-5dce-4874-8989-07739b8c3592 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:40.687007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe13cc6-5dce-4874-8989-07739b8c3592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:40.688620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe13cc6-5dce-4874-8989-07739b8c3592 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:40.708489 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 420/840] 10.222.0.33 () {56 vars in 1135 bytes} [Sat Sep 10 20:32:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:40.714303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:40.735314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:40.826286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09159856-92ca-4445-b520-2804aa1fe7b5 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:40.862410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:40.864167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:40.864651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:40.865144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:40.940477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:40.942845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:40.962393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:41.053557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ec2ae-042c-420e-ae3e-913d62fa28ce None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.054101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ec2ae-042c-420e-ae3e-913d62fa28ce None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.054181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ec2ae-042c-420e-ae3e-913d62fa28ce None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.055696 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 421/841] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:41.057200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 421/842] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:32:41.062403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca9a9-5664-4f12-b0d5-1194cffc2188 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.062687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca9a9-5664-4f12-b0d5-1194cffc2188 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.062961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ca9a9-5664-4f12-b0d5-1194cffc2188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.067086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:41.085177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-866ca9a9-5664-4f12-b0d5-1194cffc2188 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:41.086469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-866ca9a9-5664-4f12-b0d5-1194cffc2188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:41.149410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:41.165987 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 422/843] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:41.203052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf63104-a7a6-49f5-9127-932256a731e8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:41.224945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:41.243519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:41.244477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.244990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.245483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.257147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:41.258899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:41.274872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:41.276015 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:41.277056 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:41.277056 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:41.277056 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:41.279261 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 422/844] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:41.285770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ca9a9-5664-4f12-b0d5-1194cffc2188 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:41.405851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ca9a9-5664-4f12-b0d5-1194cffc2188 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:41.458423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:41.459245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.459654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.460054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.460824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:41.461275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:41.462778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:41.476016 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 423/845] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:41.481676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:41.575135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:41.599786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ce1db-06c4-4cd8-b2ae-4ad6da596019 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:41.634013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:41.645072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2507e5-f226-41db-923a-70202018b46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:41.645726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2507e5-f226-41db-923a-70202018b46e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.646049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2507e5-f226-41db-923a-70202018b46e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.646333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2507e5-f226-41db-923a-70202018b46e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.653594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2507e5-f226-41db-923a-70202018b46e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:41.655421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2507e5-f226-41db-923a-70202018b46e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7a2507e5-f226-41db-923a-70202018b46e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:41.667104 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:41.668005 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:41.668927 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 423/846] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:41.669920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aeb68-6591-4830-a1f5-e639f8c6e284 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.670032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aeb68-6591-4830-a1f5-e639f8c6e284 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.670523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aeb68-6591-4830-a1f5-e639f8c6e284 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.672974 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 424/847] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:41.673622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aeb68-6591-4830-a1f5-e639f8c6e284 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:41.699702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874cf06-151a-4ef1-838f-6236ddb5b0a7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:41.706083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:41.739568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646ac51-2b80-46c4-b8df-0e05a3cbccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:41.740210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ac51-2b80-46c4-b8df-0e05a3cbccb3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.740492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ac51-2b80-46c4-b8df-0e05a3cbccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.740765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646ac51-2b80-46c4-b8df-0e05a3cbccb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.750933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646ac51-2b80-46c4-b8df-0e05a3cbccb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c646ac51-2b80-46c4-b8df-0e05a3cbccb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:41.756297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:41.762176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 424/848] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:41.763069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eead62-8167-46c2-9838-fda37dff7087 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.763861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eead62-8167-46c2-9838-fda37dff7087 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.763861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eead62-8167-46c2-9838-fda37dff7087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.779895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-13eead62-8167-46c2-9838-fda37dff7087 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:41.781061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-13eead62-8167-46c2-9838-fda37dff7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:41.786779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aeb68-6591-4830-a1f5-e639f8c6e284 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:41.820499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:41.821096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.821379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.821654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.822377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:41.822685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:41.824115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:41.836473 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 425/849] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:41.837258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:41.851042 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 425/850] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:41.852325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391fbda-6e85-4991-b6ae-ddccf6571a13 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.852610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391fbda-6e85-4991-b6ae-ddccf6571a13 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.852881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391fbda-6e85-4991-b6ae-ddccf6571a13 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.857945 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 426/851] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:41.857945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391fbda-6e85-4991-b6ae-ddccf6571a13 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:41.906210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:41.938631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:41.949230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8f96-fa23-41cf-8cbf-6a2b6dd2f5d2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:41.972451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391fbda-6e85-4991-b6ae-ddccf6571a13 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:41.992557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3803b7b-f043-45d3-aad2-fc0d2ac80e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:41.993162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803b7b-f043-45d3-aad2-fc0d2ac80e6f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:41.993440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803b7b-f043-45d3-aad2-fc0d2ac80e6f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:41.993709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803b7b-f043-45d3-aad2-fc0d2ac80e6f None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:41.994395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803b7b-f043-45d3-aad2-fc0d2ac80e6f None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:41.994698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803b7b-f043-45d3-aad2-fc0d2ac80e6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:41.996053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803b7b-f043-45d3-aad2-fc0d2ac80e6f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.004816 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 426/852] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:42.007405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91876-4c88-4f3f-bff7-8806acbb94cf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.007875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91876-4c88-4f3f-bff7-8806acbb94cf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.008324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad91876-4c88-4f3f-bff7-8806acbb94cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.010516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.011189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.011552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.011829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.012746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=32f1fd57a99e4c0b9e50e997c5ee74f1) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:42.013055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.014576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.026151 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 427/853] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:41 2022] GET /identity/v3/role_assignments?user.id=32f1fd57a99e4c0b9e50e997c5ee74f1&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:42.027355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.035821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4ad91876-4c88-4f3f-bff7-8806acbb94cf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:42.037295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4ad91876-4c88-4f3f-bff7-8806acbb94cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:42.122902 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 427/854] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:42.165651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9e243-1bd1-4975-83d5-115373dd902a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.173968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:42.182701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91876-4c88-4f3f-bff7-8806acbb94cf None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.221989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.221989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.221989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.221989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.234774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.236976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:42.258058 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:42.259277 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:42.260391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:42.260391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:42.260391 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:42.261588 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 428/855] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:42.267592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.292275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:42.317254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad91876-4c88-4f3f-bff7-8806acbb94cf None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.362071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.363549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.363690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.364024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.364570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.365965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:42.375234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:42.376401 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:42.377502 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 428/856] 10.222.0.33 () {56 vars in 1132 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:42.384393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.415077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89927c-aed6-447e-970d-d39bd5fd1743 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.453129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.454083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.454366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.454691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.455337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:42.455639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.457200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.471267 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 429/857] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:42.475291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.498042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:42.518206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9521-bd81-4c7d-805c-acaecff8126e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.553833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.554449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.554728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.555037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.555605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:42.555905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.557258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.561638 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 429/858] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:42.562439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.580708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:42.593746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4c327-f372-4f2e-85dd-dae7cf30e21d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.635622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.636266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.636550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.636822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.637400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.638787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:42.649107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:42.650256 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:42.651092 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 430/859] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:42.651875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.688942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db65ca-212a-4f67-af19-915c2c4cb6c7 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.725061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.725746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.726009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.726271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.733855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.735288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.738126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:42.748438 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:42.749453 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 430/860] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:42.750538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.763963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbfab2-7934-4efa-b485-6c385a7c05d9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.800804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.801502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.801876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.802229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.802920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:42.803301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.804713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.809307 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 431/861] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:42.810016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.839547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:42.877666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686b14b-deaf-4354-8d39-68692a6cf87d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.916312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.917216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.917651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.918192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.919048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:42.919586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.921201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:42.927703 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 431/862] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:42.934045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ffcc-6975-4cb7-9f36-767096aa9acd None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:42.934224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:42.978018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:42.978866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:42.979296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:42.979729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:42.980691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:42.981175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:42.983063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:43.004688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 432/863] 10.222.0.33 () {56 vars in 1149 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:43.011229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:43.017142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:43.065482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048eb3-3b9e-47d8-8f99-1cb7e025a77b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:43.105092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:43.106091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:43.106439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:43.106842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:43.107870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:43.109006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:43.110465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:43.118151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:43.118298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 432/864] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:43.151535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ade5b-cb04-49f1-8866-a405470e1926 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:43.191008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:43.191774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:43.192035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:43.192198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:43.271742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:43.273133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:43.276224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:43.401931 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 433/865] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:32:43.404000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:43.524471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d44ac-3f5e-4c9e-8228-ddf5c2fbf631 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:43.560629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:43.561254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:43.561529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:43.561797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:43.573861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:43.575315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:43.590841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:43.591842 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:43.592769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:43.592769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:43.592769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:43.595631 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 433/866] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:43.598754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:43.706218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee97ae-bab1-46be-b70b-a7285f454f36 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:43.709412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:43.748321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:43.749468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:43.750001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:43.750517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:43.751352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:43.751854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:43.754171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:43.766945 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 434/867] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:43.772707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:43.825769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:43.878745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea85e2f-8649-4c3f-9897-33bd416c07a7 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:43.913620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:43.914283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:43.914562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:43.914829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:43.922228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:43.923673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:43.936091 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:43.937050 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:43.937985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:43.941929 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 434/868] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:43.943907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:44.058551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4226a4c-5867-463c-949a-efaf45145346 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:44.095473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671948f-0078-4119-9919-a750a6ebae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:44.096045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671948f-0078-4119-9919-a750a6ebae36 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:44.096328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671948f-0078-4119-9919-a750a6ebae36 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:44.096596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671948f-0078-4119-9919-a750a6ebae36 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:44.097540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671948f-0078-4119-9919-a750a6ebae36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:44.097868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671948f-0078-4119-9919-a750a6ebae36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:44.099292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671948f-0078-4119-9919-a750a6ebae36 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:44.117733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 435/869] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:44.119221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:44.226906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e479c63-db2d-410f-bd94-c0ebcf963da9 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:44.237792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:44.263044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95a15-d086-47e7-901c-418c32c2b6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:44.263663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95a15-d086-47e7-901c-418c32c2b6ec None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:44.263920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95a15-d086-47e7-901c-418c32c2b6ec None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:44.264192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95a15-d086-47e7-901c-418c32c2b6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:44.264910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95a15-d086-47e7-901c-418c32c2b6ec None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:44.265215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95a15-d086-47e7-901c-418c32c2b6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:44.268494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95a15-d086-47e7-901c-418c32c2b6ec None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:44.282010 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 435/870] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:44 2022] GET /identity/v3/role_assignments?user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:44.569156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225bf37-d566-4974-8ef3-76db69f3d581 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:44.569674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225bf37-d566-4974-8ef3-76db69f3d581 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:44.569674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225bf37-d566-4974-8ef3-76db69f3d581 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:44.571206 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 436/871] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:44.576793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95f659-9573-43ce-967b-5a4432767f0f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:44.577122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95f659-9573-43ce-967b-5a4432767f0f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:44.577358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95f659-9573-43ce-967b-5a4432767f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:44.599319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-de95f659-9573-43ce-967b-5a4432767f0f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:44.602073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de95f659-9573-43ce-967b-5a4432767f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:44.674642 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 436/872] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:45.004428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225bf37-d566-4974-8ef3-76db69f3d581 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:45.032577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:45.119841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225bf37-d566-4974-8ef3-76db69f3d581 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:45.155491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:45.156114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:45.156391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:45.156657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:45.157228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:45.158601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:45.168454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:45.169467 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:45.170358 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 437/873] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:45.177988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95f659-9573-43ce-967b-5a4432767f0f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:45.289915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95f659-9573-43ce-967b-5a4432767f0f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:45.292295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:45.331826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05217dea-c460-41f5-a084-5f29610ac12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:45.332753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05217dea-c460-41f5-a084-5f29610ac12d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:45.333032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05217dea-c460-41f5-a084-5f29610ac12d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:45.333307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05217dea-c460-41f5-a084-5f29610ac12d None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:45.333968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05217dea-c460-41f5-a084-5f29610ac12d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:45.334270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05217dea-c460-41f5-a084-5f29610ac12d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:45.335894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05217dea-c460-41f5-a084-5f29610ac12d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:45.341475 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 437/874] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:45.347096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:45.440295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:45.455713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7db80-1f00-4a0d-a803-8029ea885e3e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:45.492335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba692f1-09c9-48c8-b346-00794a430aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:45.492937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba692f1-09c9-48c8-b346-00794a430aac None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:45.493219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba692f1-09c9-48c8-b346-00794a430aac None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:45.493494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba692f1-09c9-48c8-b346-00794a430aac None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:45.494149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba692f1-09c9-48c8-b346-00794a430aac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:45.494455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba692f1-09c9-48c8-b346-00794a430aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:45.495818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba692f1-09c9-48c8-b346-00794a430aac None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:45.501187 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 438/875] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:45.509503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad81101-446d-4aa8-a6e6-e102a63bbdf5 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:45.510454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad81101-446d-4aa8-a6e6-e102a63bbdf5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:45.510454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad81101-446d-4aa8-a6e6-e102a63bbdf5 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:45.512604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 438/876] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:45.520375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14098d2-ff5a-4714-b075-9096943ca6f5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:45.520675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14098d2-ff5a-4714-b075-9096943ca6f5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:45.520953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14098d2-ff5a-4714-b075-9096943ca6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:45.542145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e14098d2-ff5a-4714-b075-9096943ca6f5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:45.543318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e14098d2-ff5a-4714-b075-9096943ca6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:45.655539 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 439/877] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:45.707649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048278b5-6085-4053-a0b1-5bfb32356439 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:45.708228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048278b5-6085-4053-a0b1-5bfb32356439 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:45.708681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048278b5-6085-4053-a0b1-5bfb32356439 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:45.710579 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 439/878] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:45.716180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449fafd-e93c-4083-a84d-b5d09a982bfc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:45.716729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449fafd-e93c-4083-a84d-b5d09a982bfc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:45.717182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449fafd-e93c-4083-a84d-b5d09a982bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:45.758063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4449fafd-e93c-4083-a84d-b5d09a982bfc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:45.759270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4449fafd-e93c-4083-a84d-b5d09a982bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:45.834860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 440/879] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:46.110726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048278b5-6085-4053-a0b1-5bfb32356439 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:46.227011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449fafd-e93c-4083-a84d-b5d09a982bfc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:46.242614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:46.252805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048278b5-6085-4053-a0b1-5bfb32356439 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:46.294697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a3107-508a-46fb-add0-fe968dd8bcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:46.295511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3107-508a-46fb-add0-fe968dd8bcf2 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:46.295804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3107-508a-46fb-add0-fe968dd8bcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:46.296082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3107-508a-46fb-add0-fe968dd8bcf2 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:46.296970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a3107-508a-46fb-add0-fe968dd8bcf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:46.298364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a3107-508a-46fb-add0-fe968dd8bcf2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:46.321998 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 440/880] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:32:46 2022] POST /identity/v3/services => generated 269 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:46.354547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449fafd-e93c-4083-a84d-b5d09a982bfc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:46.402788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd4aba-7b14-4ad1-8e33-6962d0fd0b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:46.403824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd4aba-7b14-4ad1-8e33-6962d0fd0b49 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:46.404303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd4aba-7b14-4ad1-8e33-6962d0fd0b49 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:46.404773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd4aba-7b14-4ad1-8e33-6962d0fd0b49 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:46.405809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd4aba-7b14-4ad1-8e33-6962d0fd0b49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:46.407534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd4aba-7b14-4ad1-8e33-6962d0fd0b49 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:46.429326 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 441/881] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:32:46 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:32:46.972987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d755e-d2c3-4294-91d3-bb9d6f60d356 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:46.972987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d755e-d2c3-4294-91d3-bb9d6f60d356 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:46.972987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d755e-d2c3-4294-91d3-bb9d6f60d356 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:46.974094 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 441/882] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:46.979205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d085-ccfd-4581-80eb-a49d9b9882ad None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:46.979599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d085-ccfd-4581-80eb-a49d9b9882ad None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:46.979767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05d085-ccfd-4581-80eb-a49d9b9882ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:46.999195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5a05d085-ccfd-4581-80eb-a49d9b9882ad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:47.000384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5a05d085-ccfd-4581-80eb-a49d9b9882ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:47.065173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:47.081965 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 442/883] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:47.409112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d755e-d2c3-4294-91d3-bb9d6f60d356 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:47.438350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:47.524014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d755e-d2c3-4294-91d3-bb9d6f60d356 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:47.563941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:47.564938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:47.565414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:47.566022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:47.577082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:47.582778 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:47.583702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:32:47.584557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:47.587196 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 442/884] 10.222.0.33 () {56 vars in 1120 bytes} [Sat Sep 10 20:32:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:47.593870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d085-ccfd-4581-80eb-a49d9b9882ad None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:47.703762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05d085-ccfd-4581-80eb-a49d9b9882ad None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:47.741234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:47.741873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:47.742160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:47.742436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:47.743217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:47.743528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:47.744888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:47.754398 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 443/885] 10.222.0.33 () {56 vars in 1115 bytes} [Sat Sep 10 20:32:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:47.759939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:47.862063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:47.875252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6c637-1fda-4d50-aceb-9eb3798bfddb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:47.911773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:47.912764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:47.913297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:47.913781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:47.925193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:47.927242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:47.943045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:47.944036 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:47.945014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:47.945014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:47.945014 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:47.947098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 443/886] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:47.955775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:48.022663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:48.068464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65d472-a4a1-46a1-a9fe-ba4117664b08 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:48.103368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bd538-3393-466d-8ed8-64f5b8c74334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:48.104098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bd538-3393-466d-8ed8-64f5b8c74334 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.104381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bd538-3393-466d-8ed8-64f5b8c74334 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.104657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bd538-3393-466d-8ed8-64f5b8c74334 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.105300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bd538-3393-466d-8ed8-64f5b8c74334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:48.105606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bd538-3393-466d-8ed8-64f5b8c74334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:48.107013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2bd538-3393-466d-8ed8-64f5b8c74334 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:48.119246 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 444/887] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:48.124636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:48.230714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166766b-78ce-4c57-a9f0-b63d265774af None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:48.256068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb3834-9694-4566-8a30-d9bf7c3561a1 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.256690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb3834-9694-4566-8a30-d9bf7c3561a1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.257213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb3834-9694-4566-8a30-d9bf7c3561a1 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.259048 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 445/888] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:32:48.264474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667da9-14fe-47f5-81d3-fb5da39702b9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.265019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667da9-14fe-47f5-81d3-fb5da39702b9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.265502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a667da9-14fe-47f5-81d3-fb5da39702b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.273914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:48.274548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.274827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.275099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.283896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:48.285853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2a667da9-14fe-47f5-81d3-fb5da39702b9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:48.286122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:48.287875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2a667da9-14fe-47f5-81d3-fb5da39702b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:48.292821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:48.300796 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:48.301978 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:48.302936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 444/889] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:32:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:48.310996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:48.380694 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 446/890] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:48.435173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab0f91-dd8b-405e-88e0-2fa5c09b30f2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:48.474044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:48.475168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.475655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.476132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.477096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:48.477644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:48.479377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:48.505708 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 445/891] 10.222.0.33 () {56 vars in 1149 bytes} [Sat Sep 10 20:32:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:48.508528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667da9-14fe-47f5-81d3-fb5da39702b9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:48.568036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:48.633869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a667da9-14fe-47f5-81d3-fb5da39702b9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:48.670554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:48.671151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.671430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.671704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.716504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:48.743706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:48.745158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:48.791630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:48.795566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:48.851934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04536ebc-70bb-4872-babd-d7418b10e847 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:48.858037 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 447/892] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:32:48.864661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:48.900166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bc6d7-053d-46a1-982b-6ff6e8e1ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:48.900776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bc6d7-053d-46a1-982b-6ff6e8e1ff0d None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.901057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bc6d7-053d-46a1-982b-6ff6e8e1ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.901329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bc6d7-053d-46a1-982b-6ff6e8e1ff0d None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.902093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bc6d7-053d-46a1-982b-6ff6e8e1ff0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:48.903391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bc6d7-053d-46a1-982b-6ff6e8e1ff0d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:48.919352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 446/893] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:32:48 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:32:48.920723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa461c-1128-4e40-8eb6-b1ba8fba31d1 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.921018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa461c-1128-4e40-8eb6-b1ba8fba31d1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.921290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa461c-1128-4e40-8eb6-b1ba8fba31d1 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.923028 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 447/894] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:48.928725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae05f8c-956a-4cd8-985c-43f1b3c07171 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:48.929047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae05f8c-956a-4cd8-985c-43f1b3c07171 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:48.929284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae05f8c-956a-4cd8-985c-43f1b3c07171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:48.949403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-aae05f8c-956a-4cd8-985c-43f1b3c07171 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:48.950679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aae05f8c-956a-4cd8-985c-43f1b3c07171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:48.992598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd7507-3773-4b42-ab6d-e6b910fae171 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.039102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.039781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.040000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.040269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.050783 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 448/895] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:32:49.052640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.054331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.065440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:49.068128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb199-ede6-4236-a76e-cd4f6c17732c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.068642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb199-ede6-4236-a76e-cd4f6c17732c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.069069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb199-ede6-4236-a76e-cd4f6c17732c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.070956 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 449/896] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:49.075274 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:49.076297 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:49.077217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:49.077217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:49.077217 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:49.077217 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 448/897] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:49.080257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d244fd0-ff57-4f0b-927b-29e91c60a714 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.080436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.081063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d244fd0-ff57-4f0b-927b-29e91c60a714 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.081597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d244fd0-ff57-4f0b-927b-29e91c60a714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.102082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2d244fd0-ff57-4f0b-927b-29e91c60a714 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:49.103925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2d244fd0-ff57-4f0b-927b-29e91c60a714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:49.162211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:49.191612 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 450/898] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:49.203277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f7ec-cd92-49e0-863c-80119d1fc758 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.240605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.241213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.241491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.241766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.242439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:49.242740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.244108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.256896 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 449/899] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:49.262711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d244fd0-ff57-4f0b-927b-29e91c60a714 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.369527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d244fd0-ff57-4f0b-927b-29e91c60a714 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.405325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.406144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.406439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.406718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.414073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.415525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.422320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:49.428858 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:49.430190 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:49.431991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 451/900] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:49.440053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.486784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:49.493495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:49.554507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79226ff-d4dd-4889-b243-03d17863254e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.580376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ef09-0370-43da-bc48-edf26cfaac45 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.592194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.592828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.593095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.593364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.593978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.595330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:49.604464 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:49.605516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:49.607031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 450/901] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:49.607851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.622639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.623282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.623761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.623992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.624692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:49.625003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.626746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.639800 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 452/902] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:49.640845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.729067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841239c-bc5d-4172-97a6-f63617f4de18 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.747634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:49.749675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:49.755977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd166a0-e7bb-47df-b047-ca1502e99a1b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.765534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.766188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.766468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.766741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.767313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.768748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:49.778157 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:49.779306 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:49.780412 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 451/903] 10.222.0.33 () {56 vars in 1132 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:49.781209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.797780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.798417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.798695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.798966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.799586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:49.800446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.802336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.807414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 453/904] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:49.808616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.899697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cf72d-888e-4452-8e6b-5e7ec1614591 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.910450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61bc1b-7386-41d9-a000-21428f25df19 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:49.943126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.943719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.943993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.944261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.945024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] RBAC: Adding query filter params (role.id=2bd6237d9303489c830fba54a2be93f5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:49.945387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.946074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:49.946737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:49.946832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.947026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:49.947301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:49.947952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:49.948256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:49.949610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:49.953173 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 452/905] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/role_assignments?user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=2bd6237d9303489c830fba54a2be93f5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:49.954177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:49.960033 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 454/906] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:49.960925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:50.010000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:50.021892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:50.101637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd3f24-117c-42b8-aca5-591c3d7483bb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:50.112062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c5e5b-379a-47a1-84eb-b2e861b62769 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:50.140045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:50.140662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:50.140939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:50.141392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:50.142090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:50.142386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:50.143988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:50.150456 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 455/907] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:50.155670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:50.167523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:50.169021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:50.169460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:50.169933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:50.171151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:50.171622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:50.173809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:50.182493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 453/908] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:50.191828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:50.266613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:50.278847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ce286-9d4a-4d06-ab37-d88f32cb076f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:50.300417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c0c44-410f-4781-b51e-6ddc66df875f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:50.311645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:50.319124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:50.320002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:50.320502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:50.320946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:50.322210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] RBAC: Adding query filter params (interface=public, service_id=bdc76dde98e842609dc2a90e7dc0dc3c, region_id=RegionOne) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:50.322700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:50.324332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:50.330553 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 456/909] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:32:50 2022] GET /identity/v3/endpoints?service_id=bdc76dde98e842609dc2a90e7dc0dc3c&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:50.342807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:50.345759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:50.347390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:50.347739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:50.348090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:50.348970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] RBAC: Adding query filter params (interface=public, service_id=1dd5c84f46e54265a2a9e50578c24c3a, region_id=RegionOne) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:50.349357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:50.350836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:50.357977 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 454/910] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:32:50 2022] GET /identity/v3/endpoints?service_id=1dd5c84f46e54265a2a9e50578c24c3a&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:50.363690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:50.488130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447af95-71e3-409f-8fb1-df9e20b06d28 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:50.491182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b03f5-b835-46c8-8029-bef6f46cc2c3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:50.524911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dd492-baa6-4140-96b9-791225bdfffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:50.525538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd492-baa6-4140-96b9-791225bdfffa None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:50.525814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd492-baa6-4140-96b9-791225bdfffa None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:50.526117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd492-baa6-4140-96b9-791225bdfffa None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:50.526712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dd492-baa6-4140-96b9-791225bdfffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:50.528431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dd492-baa6-4140-96b9-791225bdfffa None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:50.535880 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 455/911] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:32:50 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:32:50.547748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e24947-6bc2-46b0-ad8d-32d4a3a7f42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:50.548540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e24947-6bc2-46b0-ad8d-32d4a3a7f42d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:50.551745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e24947-6bc2-46b0-ad8d-32d4a3a7f42d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:50.551745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e24947-6bc2-46b0-ad8d-32d4a3a7f42d None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:50.551745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e24947-6bc2-46b0-ad8d-32d4a3a7f42d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:50.556140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e24947-6bc2-46b0-ad8d-32d4a3a7f42d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:50.563609 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 457/912] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:32:50 2022] GET /identity/v3/services => generated 1497 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:32:51.465540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35b5c4-a466-47e5-aeae-f8ef00779dbd None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:51.466157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35b5c4-a466-47e5-aeae-f8ef00779dbd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:51.466157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35b5c4-a466-47e5-aeae-f8ef00779dbd None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:51.467568 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 456/913] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:51.473629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa82de-1df0-4acf-a8ef-c841454de819 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:51.473939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa82de-1df0-4acf-a8ef-c841454de819 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:51.474214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa82de-1df0-4acf-a8ef-c841454de819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:51.494952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f1fa82de-1df0-4acf-a8ef-c841454de819 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:51.496875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1fa82de-1df0-4acf-a8ef-c841454de819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:51.511190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:51.577383 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 458/914] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:51.909806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35b5c4-a466-47e5-aeae-f8ef00779dbd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:51.952259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:52.014684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35b5c4-a466-47e5-aeae-f8ef00779dbd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:52.049760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:52.050431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:52.050911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:52.051196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:52.051785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:52.053156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:52.062312 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:52.063329 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:52.064224 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 457/915] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:52.070539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa82de-1df0-4acf-a8ef-c841454de819 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:52.183189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa82de-1df0-4acf-a8ef-c841454de819 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:52.220473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:52.221446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:52.222013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:52.222490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:52.223346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:52.223852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:52.225430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:52.231189 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 459/916] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:52.238766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:52.338074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecb41a-b57e-40c5-b662-c2a3cecedf0a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:52.339935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:52.375120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:52.375727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:52.376011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:52.376286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:52.376916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:52.377240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:52.379212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:52.384489 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 458/917] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:52.390391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:52.405738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:52.496717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f31ab-f115-42e2-8c97-39652b5ea6b9 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:52.532434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bcff1-edec-4f06-8e41-1b9d480c8f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:52.533059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bcff1-edec-4f06-8e41-1b9d480c8f6f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:52.533343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bcff1-edec-4f06-8e41-1b9d480c8f6f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:52.533624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bcff1-edec-4f06-8e41-1b9d480c8f6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:52.534977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bcff1-edec-4f06-8e41-1b9d480c8f6f None admin] RBAC: Adding query filter params (interface=public, service_id=842a16222fb74ecc9b586d59c43245b1, region_id=RegionOne) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:52.535286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bcff1-edec-4f06-8e41-1b9d480c8f6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:52.536776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bcff1-edec-4f06-8e41-1b9d480c8f6f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:52.542651 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 460/918] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:32:52 2022] GET /identity/v3/endpoints?service_id=842a16222fb74ecc9b586d59c43245b1&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:52.548347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:52.631988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9dd0-c57b-4b09-9b6d-dd920c45c0d9 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:52.632316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9dd0-c57b-4b09-9b6d-dd920c45c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:52.632608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a9dd0-c57b-4b09-9b6d-dd920c45c0d9 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:52.635511 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 461/919] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:32:52.640775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0db73e-efdf-48e9-91b6-82bbb1596fcb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:52.641297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0db73e-efdf-48e9-91b6-82bbb1596fcb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:52.641764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0db73e-efdf-48e9-91b6-82bbb1596fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:52.663672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fe0db73e-efdf-48e9-91b6-82bbb1596fcb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:52.664617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d220a11-a79e-45a5-8b63-70f28ea5e986 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:52.665415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fe0db73e-efdf-48e9-91b6-82bbb1596fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:52.708171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:52.708989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:52.709371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:52.709742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:52.710715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:52.713002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:52.720391 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 459/920] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:32:52 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:32:52.762840 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 462/921] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:32:53.115057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:53.141931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955b7f-c765-4e5c-8cd6-f6379d376fbc None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.142254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955b7f-c765-4e5c-8cd6-f6379d376fbc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.142500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955b7f-c765-4e5c-8cd6-f6379d376fbc None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.144123 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 463/922] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:32:53.149301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed7e6-67c6-47b9-a0a9-ac590b76afbc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.149651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed7e6-67c6-47b9-a0a9-ac590b76afbc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.149969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed7e6-67c6-47b9-a0a9-ac590b76afbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.153604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:53.163201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:53.171239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1f6ed7e6-67c6-47b9-a0a9-ac590b76afbc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:53.172957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1f6ed7e6-67c6-47b9-a0a9-ac590b76afbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:53.232600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d87db-ce92-48b1-83a6-7094b471fcc6 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:53.248587 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 464/923] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:32:53.248587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1b925-0520-4ca4-93c8-5cfead2a1b3c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.248587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1b925-0520-4ca4-93c8-5cfead2a1b3c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.248587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1b925-0520-4ca4-93c8-5cfead2a1b3c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.249279 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 465/924] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:53.255458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c4128-6af3-42d7-90ae-53e70165efa4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.255744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c4128-6af3-42d7-90ae-53e70165efa4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.256016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c4128-6af3-42d7-90ae-53e70165efa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.272130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:53.273067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.273532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.274015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.275322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e42c4128-6af3-42d7-90ae-53e70165efa4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:53.276673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e42c4128-6af3-42d7-90ae-53e70165efa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:53.285930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:53.291948 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:53.292880 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:32:53.293813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:53.296625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 460/925] 10.222.0.33 () {56 vars in 1120 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:53.302840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:53.373682 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 466/926] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:53.422542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:53.434154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e55e-b7de-4f8a-bd99-44fd2ee38125 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:53.469559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:53.470555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.471069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.471659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.472620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:53.473140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:53.474724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:53.488848 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 461/927] 10.222.0.33 () {56 vars in 1115 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:53.490615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c4128-6af3-42d7-90ae-53e70165efa4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:53.567153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:53.585466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:53.612803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c4128-6af3-42d7-90ae-53e70165efa4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:53.659590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:53.660138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.660420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.660694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.673480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:53.675866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:53.695076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:53.696074 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:53.696989 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:53.696989 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:53.696989 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:53.697899 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 467/928] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:53.702518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:53.705504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaa155-439f-4127-9790-3139c44a1f20 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:53.742909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:53.744311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.744606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.744887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.745473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:53.746898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:53.757446 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:53.758551 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:53.759440 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 462/929] 10.222.0.33 () {56 vars in 1132 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:53.760251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:53.778628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:53.823028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad993106-185f-445f-8416-dbe6423abbd1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:53.844198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:53.869120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:53.870103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.870563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.871014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.872063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:53.872618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:53.874975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:53.892231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd2fd6-b696-4abd-a146-cdce2fe19b8c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:53.898154 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 468/930] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:53.899255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:53.935161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:53.935699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:53.935978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:53.936247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:53.936817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:53.938202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:53.948474 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:53.949498 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:53.950766 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 463/931] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:53.951590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.014153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1b66-052d-4923-8174-694cef2be575 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.047323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:54.054028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.054666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.054949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.055221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.055848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:54.056155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.057527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.062559 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 469/932] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:54.068313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.070488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00c9c1-dd02-45cf-b6f7-7012aae1fa7f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.109869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.110496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.110780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.111056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.116490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:54.119508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.121574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:54.133296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:54.134404 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 464/933] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:32:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:54.135349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.188326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4cd75-5301-44ab-aba0-f2996f487ea9 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.234555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.235171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.235423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.235693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.236307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:54.236610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.238110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.243575 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 470/934] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:54.245328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.246998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ab5a-016f-4689-8bbe-c61f2170f92a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.283600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.284795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.285271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.285620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.286299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:54.286603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.288126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.298734 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 465/935] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:54.298734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.301844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:54.352811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503fbfe-7f8b-409d-9b37-257cdd08da36 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.361435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:54.392352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.392960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.393241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.394406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.395133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:54.395739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.397195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.415016 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 471/936] 10.222.0.33 () {56 vars in 1149 bytes} [Sat Sep 10 20:32:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:54.415648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.427170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ce4b3-2d43-4bce-b938-4bbb37d142a4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.466704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.467233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.467525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.467785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.468416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:54.468717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.470109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.480171 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 466/937] 10.222.0.33 () {56 vars in 1114 bytes} [Sat Sep 10 20:32:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:54.480171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.525020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adde3b-24a2-4428-ab01-b97ba1c6be50 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.557466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:54.561607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.562252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.562535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.562814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.563476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.564843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.589877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 472/938] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:32:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:54.590796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.599531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40ee1-0177-4ae5-86a3-59298db9f813 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.642540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:54.655432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.656592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.657072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.657535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.671612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.673545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:54.691612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:54.692628 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:54.693522 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:54.693522 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:54.693522 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:54.700412 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 467/939] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:54.701711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:54.764546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c4c1a-0ea1-4a0c-973b-2b6734d6ae48 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.820325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.820954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.821362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.821669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.822497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.824711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.827990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80d1d-bd2b-4ede-9c7f-1eeae8c2a2de None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:54.849294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:54.852941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 473/940] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:32:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:32:54.882368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:54.883002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:54.883282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:54.883554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:54.884217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:54.884572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:54.887540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:54.904013 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 468/941] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:54.911256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:55.012496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:55.044684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd220c38-3fac-48b2-9641-142e34f1f896 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:55.084961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecee6f6-42c8-4daa-ad13-d9bb65326e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:55.085610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee6f6-42c8-4daa-ad13-d9bb65326e4e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:55.086002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee6f6-42c8-4daa-ad13-d9bb65326e4e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:55.086283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee6f6-42c8-4daa-ad13-d9bb65326e4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:55.093989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee6f6-42c8-4daa-ad13-d9bb65326e4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:55.095461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee6f6-42c8-4daa-ad13-d9bb65326e4e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5ecee6f6-42c8-4daa-ad13-d9bb65326e4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:55.108365 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:55.109284 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:55.110197 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:55.110633 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 474/942] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:55.116439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:55.201640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2bb62-6db5-4bb2-95a3-058b8e455a4c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:55.202018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2bb62-6db5-4bb2-95a3-058b8e455a4c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:55.202239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2bb62-6db5-4bb2-95a3-058b8e455a4c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:55.203565 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 475/943] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:32:55.209456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69dbc3-6347-4aa5-a1d6-c1d5ba241031 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:55.209743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69dbc3-6347-4aa5-a1d6-c1d5ba241031 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:55.210043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69dbc3-6347-4aa5-a1d6-c1d5ba241031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:55.230318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e775e-c775-427b-af38-b1f6a6d01af4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:55.231790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9a69dbc3-6347-4aa5-a1d6-c1d5ba241031 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:55.233403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9a69dbc3-6347-4aa5-a1d6-c1d5ba241031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:55.271035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:55.271643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:55.271924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:55.272193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:55.272996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:55.273297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:55.274887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:55.289918 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 469/944] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:55.295323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:55.323635 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 476/945] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:32:55.331314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:55.404382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a8b2-b75d-4059-a40a-469ea62b1c83 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:55.441771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:55.442470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:55.442871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:55.443144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/dcb3cf5d0ccd4ee2a1f71963dbcdfb18/roles/2bd6237d9303489c830fba54a2be93f5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:55.445356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18, role_id=2bd6237d9303489c830fba54a2be93f5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:55.446807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:55.466003 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 470/946] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:32:55 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/dcb3cf5d0ccd4ee2a1f71963dbcdfb18/roles/2bd6237d9303489c830fba54a2be93f5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:32:55.667223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69dbc3-6347-4aa5-a1d6-c1d5ba241031 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:55.742157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:55.780621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69dbc3-6347-4aa5-a1d6-c1d5ba241031 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:55.818502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:55.819129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:55.819406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:55.819675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:55.821046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:55.822454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:55.832771 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:55.833872 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:55.835053 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 477/947] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:55.842749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:55.947827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:55.954501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccea25-214e-4bf4-8218-6eaaddb68ec2 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:55.996357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:55.996975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:55.997252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:55.997525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:55.998257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:55.998562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:55.999908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:56.005579 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 471/948] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:56.011141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:56.123685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e83de9-ae3d-49b6-9424-a11a0b9461ce None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:56.154712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:56.160882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f736b6-9848-4364-b879-4e4398050399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:56.161493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f736b6-9848-4364-b879-4e4398050399 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:56.161776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f736b6-9848-4364-b879-4e4398050399 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:56.162089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f736b6-9848-4364-b879-4e4398050399 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:56.162730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f736b6-9848-4364-b879-4e4398050399 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:56.163068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f736b6-9848-4364-b879-4e4398050399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:56.164649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f736b6-9848-4364-b879-4e4398050399 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:56.170520 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 478/949] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:56.176451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:56.280329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a399-9b6d-4ce0-a1ad-5b93983220ed None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:56.316276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602738a6-28ff-4a98-b1f9-cc3a203683a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:56.316877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602738a6-28ff-4a98-b1f9-cc3a203683a3 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:56.317211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602738a6-28ff-4a98-b1f9-cc3a203683a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:56.317490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602738a6-28ff-4a98-b1f9-cc3a203683a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:56.318192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602738a6-28ff-4a98-b1f9-cc3a203683a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:56.319556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602738a6-28ff-4a98-b1f9-cc3a203683a3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:56.342675 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 472/950] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:32:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:32:57.207731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d896-3a45-40ea-a807-3653fec5b42d None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:57.208287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d896-3a45-40ea-a807-3653fec5b42d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:57.208287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d896-3a45-40ea-a807-3653fec5b42d None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:57.209870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 479/951] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:57.215507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690ad86-8a0d-410d-a0ef-e96340c5a8bb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:57.215910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690ad86-8a0d-410d-a0ef-e96340c5a8bb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:57.216076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690ad86-8a0d-410d-a0ef-e96340c5a8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:57.234558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f690ad86-8a0d-410d-a0ef-e96340c5a8bb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:57.235664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f690ad86-8a0d-410d-a0ef-e96340c5a8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:57.244397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:57.312846 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 473/952] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:57.402025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb769-be82-4b4d-982c-0339932f0e9b None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:57.402367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb769-be82-4b4d-982c-0339932f0e9b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:57.402579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb769-be82-4b4d-982c-0339932f0e9b None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:57.404146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 480/953] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:57.409296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befac0b-6aaa-4b11-974e-ce8c70fc4faa None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:57.409581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befac0b-6aaa-4b11-974e-ce8c70fc4faa None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:57.409894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befac0b-6aaa-4b11-974e-ce8c70fc4faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:57.428268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9befac0b-6aaa-4b11-974e-ce8c70fc4faa None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:57.429445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9befac0b-6aaa-4b11-974e-ce8c70fc4faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:57.497467 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 474/954] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:57.642430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb769-be82-4b4d-982c-0339932f0e9b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:57.758799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb769-be82-4b4d-982c-0339932f0e9b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:57.823007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:57.823679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:57.823961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:57.824237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] PATH_INFO: `/v3/services/compute` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:57.824840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:57.833874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:57.844370 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:57.845676 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:57.847841 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 481/955] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:57.852428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befac0b-6aaa-4b11-974e-ce8c70fc4faa None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:57.913654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:57.925037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:57.960786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befac0b-6aaa-4b11-974e-ce8c70fc4faa None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:58.003376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:58.004507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:58.005364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.005971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.005971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.006674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:58.007133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:58.009420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:58.019682 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 475/956] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:58.020704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:58.073017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018c2f3-5142-47b9-a96c-44bdcfd80f1f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:58.122468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:58.123170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.123453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.123723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.124406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:58.126152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:58.129122 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.server.flask.common [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 10 20:32:58.143278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82a2fa-c9ad-46e2-bb76-b9c3c3eb5736 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:58.148399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.sql.core [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 20:32:58.148399 ncdv-master-1 devstack@keystone.service[93528]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 20:32:58.148399 ncdv-master-1 devstack@keystone.service[93528]: [parameters: {'id': 'ac14f5d5d7a649798a6acf6d010fdf59', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 20:32:58.148399 ncdv-master-1 devstack@keystone.service[93528]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93528) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ac14f5d5d7a649798a6acf6d010fdf59.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ac14f5d5d7a649798a6acf6d010fdf59. Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:32:58.157230 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 10 20:32:58.158140 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application yield resource Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise self.value Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application session.commit() Sep 10 20:32:58.159783 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.flush() Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application compat.raise_( Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 20:32:58.160716 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 20:32:58.161650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:32:58.162520 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [parameters: {'id': 'ac14f5d5d7a649798a6acf6d010fdf59', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:58.163397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:58.164195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ac14f5d5d7a649798a6acf6d010fdf59. Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.165157 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 482/957] 10.222.0.33 () {60 vars in 1140 bytes} [Sat Sep 10 20:32:57 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:58.167136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:58.190599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:58.196867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf6a8a-91d2-4de3-81ec-dcee27593d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:58.197743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf6a8a-91d2-4de3-81ec-dcee27593d90 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.198260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf6a8a-91d2-4de3-81ec-dcee27593d90 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.198724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf6a8a-91d2-4de3-81ec-dcee27593d90 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.199454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf6a8a-91d2-4de3-81ec-dcee27593d90 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:58.199665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf6a8a-91d2-4de3-81ec-dcee27593d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:58.201186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf6a8a-91d2-4de3-81ec-dcee27593d90 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:58.207127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 476/958] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:58.215469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46420f-7132-49ab-84ea-10bb6703e712 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.215814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46420f-7132-49ab-84ea-10bb6703e712 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.216346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46420f-7132-49ab-84ea-10bb6703e712 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.217336 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 477/959] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:58.227336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8aca2-79e1-41c1-9780-9ac81574d4ba None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.227653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8aca2-79e1-41c1-9780-9ac81574d4ba None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.227903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8aca2-79e1-41c1-9780-9ac81574d4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.252552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c2d8aca2-79e1-41c1-9780-9ac81574d4ba None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:58.254361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c2d8aca2-79e1-41c1-9780-9ac81574d4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:58.299628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977c72-fc3f-4079-a9d2-24188e96c9fd None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:58.315941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:58.341006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:58.341767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.342569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.342569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.346261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 478/960] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:32:58.351876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:58.352726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:32:58.365101 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:58.366116 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.367731 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 483/961] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:58.372388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8aca2-79e1-41c1-9780-9ac81574d4ba None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:58.485196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8aca2-79e1-41c1-9780-9ac81574d4ba None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:58.514302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:58.525992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:58.526631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.526929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.527191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.527883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:58.528188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:58.529664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:58.550823 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 479/962] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:32:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:58.692550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:58.812873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c7842-65bd-414b-a93f-d1c3bf90ec28 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:58.848547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807b52-dbbe-4e7e-8161-4fda94222699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:58.849344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807b52-dbbe-4e7e-8161-4fda94222699 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.849712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807b52-dbbe-4e7e-8161-4fda94222699 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.850123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807b52-dbbe-4e7e-8161-4fda94222699 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.860620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76807b52-dbbe-4e7e-8161-4fda94222699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-76807b52-dbbe-4e7e-8161-4fda94222699 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:32:58.867110 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:58.868353 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:32:58.869502 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:58.871431 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 484/963] 10.222.0.33 () {56 vars in 1120 bytes} [Sat Sep 10 20:32:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:58.877231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:58.955738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f14774-88d4-4e72-8017-d7e70825edba None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.956591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f14774-88d4-4e72-8017-d7e70825edba None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.957085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f14774-88d4-4e72-8017-d7e70825edba None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.964290 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 485/964] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:32:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:32:58.973294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b6b3a-2352-4c9e-b4e7-c363f84b0140 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:58.973294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b6b3a-2352-4c9e-b4e7-c363f84b0140 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:58.973294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b6b3a-2352-4c9e-b4e7-c363f84b0140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:58.988588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47c85-8c82-4dff-9340-0b3080c0c47e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:58.992648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-758b6b3a-2352-4c9e-b4e7-c363f84b0140 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:32:58.994404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-758b6b3a-2352-4c9e-b4e7-c363f84b0140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:32:59.024388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:59.025597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:59.026428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:59.027125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:59.027125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:59.028016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:59.028352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:59.029777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:59.040729 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 480/965] 10.222.0.33 () {56 vars in 1115 bytes} [Sat Sep 10 20:32:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:59.047008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.082272 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 486/966] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:32:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:32:59.105956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:59.159424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db011bd9-a937-4773-a49d-5abd46c4ba25 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:59.193689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:59.194344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:59.194620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:59.194889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:59.207102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:59.208561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:32:59.223317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:59.224734 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:32:59.225640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:32:59.225640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:32:59.225640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:59.225640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 481/967] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:59.230114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b6b3a-2352-4c9e-b4e7-c363f84b0140 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.337937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b6b3a-2352-4c9e-b4e7-c363f84b0140 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:59.374162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:59.374831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:59.375119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:59.375394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:59.376034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:59.376338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:59.378254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:59.390517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 487/968] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:59.396211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.437658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.471181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:59.520709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:59.525027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b187b0-9028-4e0c-a394-3b0bee2cb366 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:59.568418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:59.569755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:59.570565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:59.571475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:59.574148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecc5dd-9dbe-494e-836f-34ec9aeac26d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:59.582713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:59.584927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:32:59.602236 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:59.603246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 20:32:59.604245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:32:59.612196 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 482/969] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:32:59.613900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.621723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:59.622447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:59.622728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:59.622993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:59.623564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:59.624925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:32:59.636459 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:32:59.637584 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:32:59.638594 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:32:59.641924 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 488/970] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:32:59.645196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.758839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198001-efe2-4f8d-91d1-d416600d3ba8 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:59.763598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:59.777804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd36959-e614-46de-96c2-a67c892e9f3c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:59.801615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:59.802926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:59.803430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:59.803914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:59.804893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:59.805435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:59.807352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:59.817110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:32:59.817738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:32:59.818034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:32:59.818311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:32:59.818951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:32:59.819318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:32:59.820705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:32:59.830439 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 489/971] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:32:59.832911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.840552 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 483/972] 10.222.0.33 () {56 vars in 1149 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:32:59.841430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:32:59.862029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:32:59.955012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940376e-0f26-4b88-bb02-0cfbb177bac0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:32:59.979014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6060a-7686-43fb-b835-fa7dd677fe1d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:00.007043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:00.007884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.008291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.008889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.027126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:00.028429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.028874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.029542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.030348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:00.031050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:00.032965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:00.040529 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 490/973] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:00.046081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:00.067219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:00.091185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:00.092786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:00.113531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:00.161436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f4617-10b4-4b6c-af5c-24678b48fafa None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:00.198570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:00.199461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.199986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.200396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.201344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] RBAC: Adding query filter params (interface=admin, service_id=842a16222fb74ecc9b586d59c43245b1, region_id=RegionOne) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:00.201804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:00.203430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:00.209629 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 491/974] 10.222.0.33 () {56 vars in 1249 bytes} [Sat Sep 10 20:33:00 2022] GET /identity/v3/endpoints?service_id=842a16222fb74ecc9b586d59c43245b1&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:00.215549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:00.217540 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 484/975] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:32:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:33:00.224831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:00.322906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:00.344276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16bab0-bf6b-4885-8e3b-e04d7c9fcb66 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:00.352272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93987cf-5de3-413d-a882-e981b824b066 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:00.367710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:00.389172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:00.390231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.390834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.391565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.398573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:00.399334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.399689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.400037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.400721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:00.402206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:00.406855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:00.408494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:00.413919 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 492/976] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:33:00 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:00.434204 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:00.435264 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:00.436191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:00.436191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:00.436191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:00.443256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 485/977] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:00.445108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:00.568237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9f8e7-2e17-4115-a723-ecb6f064843e None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:00.609098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf912f-3bd7-4c0e-b299-5bb1c280d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:00.609719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf912f-3bd7-4c0e-b299-5bb1c280d018 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.610040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf912f-3bd7-4c0e-b299-5bb1c280d018 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.610330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf912f-3bd7-4c0e-b299-5bb1c280d018 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.611036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf912f-3bd7-4c0e-b299-5bb1c280d018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:00.611352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf912f-3bd7-4c0e-b299-5bb1c280d018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:00.612977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf912f-3bd7-4c0e-b299-5bb1c280d018 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:00.627054 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 493/978] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:00.635271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:00.757423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6210b-abbf-411b-b5dc-c2d6209b9660 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:00.792439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:00.793070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.793353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.793626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.799411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07a4b8-89b3-4615-a6ea-7c7edf4a3f39 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.799914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07a4b8-89b3-4615-a6ea-7c7edf4a3f39 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.800309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07a4b8-89b3-4615-a6ea-7c7edf4a3f39 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.800991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:00.801942 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 494/979] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:00.802476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:00.807114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945a21d-0603-4b77-b660-f5c074b2b62c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:00.807398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945a21d-0603-4b77-b660-f5c074b2b62c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:00.807673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945a21d-0603-4b77-b660-f5c074b2b62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:33:00.818151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:00.819099 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:00.820393 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 486/980] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:00.820955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:00.827686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7945a21d-0603-4b77-b660-f5c074b2b62c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:00.828861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7945a21d-0603-4b77-b660-f5c074b2b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:00.831713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:00.872118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:00.919599 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 495/981] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:00.960529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c64172-5cae-48a4-9123-3594a11ddd94 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:00.999569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a79b65-551f-41b1-83d9-13070cfce6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:01.000515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a79b65-551f-41b1-83d9-13070cfce6c5 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:01.001005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a79b65-551f-41b1-83d9-13070cfce6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:01.001490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a79b65-551f-41b1-83d9-13070cfce6c5 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:01.002428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a79b65-551f-41b1-83d9-13070cfce6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:01.003264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a79b65-551f-41b1-83d9-13070cfce6c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:01.005181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a79b65-551f-41b1-83d9-13070cfce6c5 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:01.024200 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 487/982] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:01.025956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945a21d-0603-4b77-b660-f5c074b2b62c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:01.135558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945a21d-0603-4b77-b660-f5c074b2b62c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:01.171488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61775e11-79f9-4181-9478-755955c5bddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:01.172419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61775e11-79f9-4181-9478-755955c5bddd None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:01.172903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61775e11-79f9-4181-9478-755955c5bddd None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:01.173366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61775e11-79f9-4181-9478-755955c5bddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:01.174324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61775e11-79f9-4181-9478-755955c5bddd None admin] RBAC: Adding query filter params (role.id=2bd6237d9303489c830fba54a2be93f5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:01.175523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61775e11-79f9-4181-9478-755955c5bddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:01.177229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61775e11-79f9-4181-9478-755955c5bddd None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:01.179388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:01.185192 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 496/983] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:01 2022] GET /identity/v3/role_assignments?user.id=dcb3cf5d0ccd4ee2a1f71963dbcdfb18&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=2bd6237d9303489c830fba54a2be93f5 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:01.203924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201cea7-e07c-4f34-a407-a20560683a0f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:01.204304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201cea7-e07c-4f34-a407-a20560683a0f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:01.204578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201cea7-e07c-4f34-a407-a20560683a0f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:01.206610 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 488/984] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:01.212692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd3df-be0d-415b-a8ff-590a2c6b5f6f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:01.213314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd3df-be0d-415b-a8ff-590a2c6b5f6f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:01.214134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bd3df-be0d-415b-a8ff-590a2c6b5f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:01.235922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-716bd3df-be0d-415b-a8ff-590a2c6b5f6f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:01.237687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-716bd3df-be0d-415b-a8ff-590a2c6b5f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:01.252037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201cea7-e07c-4f34-a407-a20560683a0f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:01.324032 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 497/985] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:01.375231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:01.387817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201cea7-e07c-4f34-a407-a20560683a0f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:01.427839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:01.428363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:01.428642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:01.428912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:01.429623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:01.431073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:01.449733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 489/986] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:33:01 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:33:01.662176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bd3df-be0d-415b-a8ff-590a2c6b5f6f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:01.774248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bd3df-be0d-415b-a8ff-590a2c6b5f6f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:01.802736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:01.821147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:01.821747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:01.822071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:01.822340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:01.823032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:01.824458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:01.840579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.sql.core [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 20:33:01.840579 ncdv-master-1 devstack@keystone.service[93528]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 20:33:01.840579 ncdv-master-1 devstack@keystone.service[93528]: [parameters: {'user_id': '25cabdda014747c68a4d7a85809ce0e2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 20:33:01.840579 ncdv-master-1 devstack@keystone.service[93528]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93528) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 20:33:01.849055 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:33:01.850027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application yield resource Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise self.value Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application session.commit() Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 20:33:01.851006 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.flush() Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application compat.raise_( Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 20:33:01.851926 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 20:33:01.852861 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 20:33:01.853751 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [parameters: {'user_id': '25cabdda014747c68a4d7a85809ce0e2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:01.854646 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 10 20:33:01.855450 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 10 20:33:01.856437 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 10 20:33:01.856437 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 10 20:33:01.856437 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 20:33:01.856437 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 20:33:01.856437 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 20:33:01.856437 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:01.856437 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 498/987] 10.222.0.33 () {60 vars in 1131 bytes} [Sat Sep 10 20:33:01 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:01.865274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:01.977173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2692061-e2c8-41a6-8139-4ce86ddbdbcc None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:02.013201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15108139-f285-488b-8b95-3001d5736bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:02.013870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15108139-f285-488b-8b95-3001d5736bd9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:02.014151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15108139-f285-488b-8b95-3001d5736bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:02.014424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15108139-f285-488b-8b95-3001d5736bd9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:02.022928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15108139-f285-488b-8b95-3001d5736bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:02.024429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15108139-f285-488b-8b95-3001d5736bd9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-15108139-f285-488b-8b95-3001d5736bd9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:33:02.036181 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:02.037158 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:02.038139 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 490/988] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:33:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:02.045215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:02.155385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edb5bc-0f58-468f-90b5-568f753cb94b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:02.192129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622b0bd-6c52-47da-a062-58683070d259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:02.192733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622b0bd-6c52-47da-a062-58683070d259 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:02.193014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622b0bd-6c52-47da-a062-58683070d259 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:02.193286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622b0bd-6c52-47da-a062-58683070d259 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:02.194010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622b0bd-6c52-47da-a062-58683070d259 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:02.194309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622b0bd-6c52-47da-a062-58683070d259 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:02.195925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622b0bd-6c52-47da-a062-58683070d259 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:02.235469 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 499/989] 10.222.0.33 () {56 vars in 1101 bytes} [Sat Sep 10 20:33:02 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:03.006513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91446ff-e395-4b1b-b050-53c35c764b59 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:03.007061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91446ff-e395-4b1b-b050-53c35c764b59 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:03.007061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91446ff-e395-4b1b-b050-53c35c764b59 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:03.008516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 491/990] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:03.013625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64861-74e2-42c0-8ef8-e9c06aeaa545 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:03.014058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64861-74e2-42c0-8ef8-e9c06aeaa545 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:03.014232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca64861-74e2-42c0-8ef8-e9c06aeaa545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:03.032600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0ca64861-74e2-42c0-8ef8-e9c06aeaa545 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:03.034644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ca64861-74e2-42c0-8ef8-e9c06aeaa545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:03.078429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:03.108165 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 500/991] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:03.437383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91446ff-e395-4b1b-b050-53c35c764b59 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:03.451191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:03.545137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91446ff-e395-4b1b-b050-53c35c764b59 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:03.579374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25463239-9cf0-4b75-b6e6-c7810d1b2b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:03.580007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25463239-9cf0-4b75-b6e6-c7810d1b2b5b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:03.580292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25463239-9cf0-4b75-b6e6-c7810d1b2b5b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:03.580570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25463239-9cf0-4b75-b6e6-c7810d1b2b5b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:03.581689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25463239-9cf0-4b75-b6e6-c7810d1b2b5b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:03.583095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25463239-9cf0-4b75-b6e6-c7810d1b2b5b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-25463239-9cf0-4b75-b6e6-c7810d1b2b5b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:03.592532 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:03.593534 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:03.594798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 492/992] 10.222.0.33 () {56 vars in 1126 bytes} [Sat Sep 10 20:33:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:03.600371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca64861-74e2-42c0-8ef8-e9c06aeaa545 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:03.706413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca64861-74e2-42c0-8ef8-e9c06aeaa545 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:03.737572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dffe4f-ddea-4d54-b18a-4a42d43ccd4d None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:03.737979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dffe4f-ddea-4d54-b18a-4a42d43ccd4d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:03.738465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dffe4f-ddea-4d54-b18a-4a42d43ccd4d None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:03.739952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 493/993] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:03.744542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:03.745145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:03.745418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:03.745628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd48f4-0857-4187-a8a1-a9ab1e136978 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:03.745735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:03.745950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd48f4-0857-4187-a8a1-a9ab1e136978 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:03.746228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd48f4-0857-4187-a8a1-a9ab1e136978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:03.746321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:03.746612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:03.747956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:03.758252 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 501/994] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:33:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:03.759116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:03.765214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-17bd48f4-0857-4187-a8a1-a9ab1e136978 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:03.766423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-17bd48f4-0857-4187-a8a1-a9ab1e136978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:03.809526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:03.856435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 494/995] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:03.872283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d037e-ab11-4f48-886f-d6c48b148280 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:03.906956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d82dc0-ee4e-46e6-b5e8-7b6494c93303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:03.907557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d82dc0-ee4e-46e6-b5e8-7b6494c93303 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:03.907835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d82dc0-ee4e-46e6-b5e8-7b6494c93303 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:03.908104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d82dc0-ee4e-46e6-b5e8-7b6494c93303 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:03.908728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d82dc0-ee4e-46e6-b5e8-7b6494c93303 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:03.909179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d82dc0-ee4e-46e6-b5e8-7b6494c93303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:03.910685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d82dc0-ee4e-46e6-b5e8-7b6494c93303 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:03.920104 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 502/996] 10.222.0.33 () {56 vars in 1122 bytes} [Sat Sep 10 20:33:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:03.921096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd48f4-0857-4187-a8a1-a9ab1e136978 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:03.924613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:04.028593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd48f4-0857-4187-a8a1-a9ab1e136978 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:04.061235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfcdf8-dfab-498c-a683-389b2aa6f85f None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.061758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfcdf8-dfab-498c-a683-389b2aa6f85f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.061758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfcdf8-dfab-498c-a683-389b2aa6f85f None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.063229 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 503/997] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:04.063341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:04.064053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.064342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.064678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.065992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:04.067539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:04.068612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e81942-ce53-401d-b43a-016850352f29 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.068897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e81942-ce53-401d-b43a-016850352f29 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.069166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e81942-ce53-401d-b43a-016850352f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.090886 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 495/998] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:33:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:33:04.091090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-93e81942-ce53-401d-b43a-016850352f29 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:04.091090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-93e81942-ce53-401d-b43a-016850352f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:04.174086 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 504/999] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:04.175868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:04.246426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:04.292720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1a05b-0cc4-4f97-a3fe-aac605c256bd None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:04.330376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126476b-6496-494b-9107-c275546abf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:04.331811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126476b-6496-494b-9107-c275546abf26 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.332062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126476b-6496-494b-9107-c275546abf26 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.332287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126476b-6496-494b-9107-c275546abf26 None admin] PATH_INFO: `/v3/services/image` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.332871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126476b-6496-494b-9107-c275546abf26 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:04.334289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126476b-6496-494b-9107-c275546abf26 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-0126476b-6496-494b-9107-c275546abf26 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:04.343702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:04.344905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:04.346085 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 496/1000] 10.222.0.33 () {56 vars in 1105 bytes} [Sat Sep 10 20:33:04 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:04.352764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e81942-ce53-401d-b43a-016850352f29 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:04.381677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:04.459498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e81942-ce53-401d-b43a-016850352f29 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:04.494702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:04.495305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.495583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.495855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.496545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] RBAC: Adding query filter params (name=image) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:04.496792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:04.498210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:04.503651 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 505/1001] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:04.509062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:04.509228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126476b-6496-494b-9107-c275546abf26 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:04.605059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:04.617145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821b660-ee4e-4f66-89dd-9794c3fbeebb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:04.631045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126476b-6496-494b-9107-c275546abf26 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:04.652968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:04.654174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.654466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.654748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.655545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] RBAC: Adding query filter params (type=image) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:04.655851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:04.657271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:04.662274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:04.662624 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 506/1002] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:04.673465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:04.674158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.674478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.675278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] PATH_INFO: `/v3/services/compute` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.675877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:04.677291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:04.687145 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:04.688317 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:04.689170 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 497/1003] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:33:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:04.694521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:04.811528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe571f3-8bf9-40fa-82fe-dec06d76a68b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:04.851869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:04.852484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:04.852833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:04.853116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:04.853759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] RBAC: Adding query filter params (name=compute) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:04.854088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:04.855487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:04.866331 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 507/1004] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:33:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:04.867529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:04.979474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5971b1-b295-430a-89d0-0c8a2465e56d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:05.015376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11016726-a27e-438d-af6a-3e604433317f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:05.015983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11016726-a27e-438d-af6a-3e604433317f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:05.016267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11016726-a27e-438d-af6a-3e604433317f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:05.016543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11016726-a27e-438d-af6a-3e604433317f None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:05.017171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11016726-a27e-438d-af6a-3e604433317f None admin] RBAC: Adding query filter params (type=compute) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:05.017478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11016726-a27e-438d-af6a-3e604433317f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:05.019058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11016726-a27e-438d-af6a-3e604433317f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:05.028832 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 498/1005] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:33:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:05.029973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:05.052947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:05.135743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a06e7-6c48-4ad5-957c-326e509237df None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:05.171544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97470ce-f4d7-422c-b0c4-04959c83746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:05.172170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97470ce-f4d7-422c-b0c4-04959c83746f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:05.172449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97470ce-f4d7-422c-b0c4-04959c83746f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:05.172723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97470ce-f4d7-422c-b0c4-04959c83746f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:05.173425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97470ce-f4d7-422c-b0c4-04959c83746f None admin] RBAC: Adding query filter params (interface=public, service_id=a0652a6d995b400f9e969b1d156579ce, region_id=RegionOne) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:05.173742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97470ce-f4d7-422c-b0c4-04959c83746f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:05.175332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97470ce-f4d7-422c-b0c4-04959c83746f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:05.185117 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 508/1006] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:33:05 2022] GET /identity/v3/endpoints?service_id=a0652a6d995b400f9e969b1d156579ce&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:05.186422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11016726-a27e-438d-af6a-3e604433317f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:05.247955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:05.297179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11016726-a27e-438d-af6a-3e604433317f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:05.332233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb9012-38c2-459d-a6d4-6bd8b4bcf66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:05.332837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb9012-38c2-459d-a6d4-6bd8b4bcf66f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:05.333119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb9012-38c2-459d-a6d4-6bd8b4bcf66f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:05.333393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb9012-38c2-459d-a6d4-6bd8b4bcf66f None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:05.334034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb9012-38c2-459d-a6d4-6bd8b4bcf66f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:05.335393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb9012-38c2-459d-a6d4-6bd8b4bcf66f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:05.345914 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 499/1007] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:33:05 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:33:06.614725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3993d-b142-45b4-963a-0978979fbb7c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:06.615286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3993d-b142-45b4-963a-0978979fbb7c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:06.615286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3993d-b142-45b4-963a-0978979fbb7c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:06.616742 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 509/1008] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:06.622217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f1760-9dfb-4463-9384-60c9f9fcda71 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:06.622731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f1760-9dfb-4463-9384-60c9f9fcda71 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:06.622877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f1760-9dfb-4463-9384-60c9f9fcda71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:06.641731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7a2f1760-9dfb-4463-9384-60c9f9fcda71 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:06.643172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a2f1760-9dfb-4463-9384-60c9f9fcda71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:06.717869 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 500/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:07.047883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3993d-b142-45b4-963a-0978979fbb7c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:07.109402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84e1ba-7144-4adb-808d-8f5c5467a40a None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:07.109931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84e1ba-7144-4adb-808d-8f5c5467a40a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:07.110041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84e1ba-7144-4adb-808d-8f5c5467a40a None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:07.111305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 501/1010] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:33:07.119294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a9549-40e2-45aa-b155-6ba8ce6a25d5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:07.119572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a9549-40e2-45aa-b155-6ba8ce6a25d5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:07.119842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a9549-40e2-45aa-b155-6ba8ce6a25d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:07.138486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-199a9549-40e2-45aa-b155-6ba8ce6a25d5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:07.139660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-199a9549-40e2-45aa-b155-6ba8ce6a25d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:07.140764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:07.161010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3993d-b142-45b4-963a-0978979fbb7c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:07.195674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:07.198345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76649646-c486-472a-a419-723dc7068d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:07.199038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76649646-c486-472a-a419-723dc7068d03 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:07.199323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76649646-c486-472a-a419-723dc7068d03 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:07.199604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76649646-c486-472a-a419-723dc7068d03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:07.200812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76649646-c486-472a-a419-723dc7068d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:07.202266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76649646-c486-472a-a419-723dc7068d03 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:07.203833 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 510/1011] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:07.210653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76649646-c486-472a-a419-723dc7068d03 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:07.230244 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 502/1012] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:07 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:07.313671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76649646-c486-472a-a419-723dc7068d03 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:07.349709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07366a-46fa-409c-ab2b-2b54383f9aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:07.350377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07366a-46fa-409c-ab2b-2b54383f9aeb None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:07.350659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07366a-46fa-409c-ab2b-2b54383f9aeb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:07.350933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07366a-46fa-409c-ab2b-2b54383f9aeb None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:07.351594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07366a-46fa-409c-ab2b-2b54383f9aeb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:07.353068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07366a-46fa-409c-ab2b-2b54383f9aeb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:07.389848 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 511/1013] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:33:07 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:33:07.561933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a9549-40e2-45aa-b155-6ba8ce6a25d5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:07.676385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a9549-40e2-45aa-b155-6ba8ce6a25d5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:07.711638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac251a1-2981-443d-98d7-2a2414d733a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:07.712241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac251a1-2981-443d-98d7-2a2414d733a8 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:07.712521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac251a1-2981-443d-98d7-2a2414d733a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:07.712790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac251a1-2981-443d-98d7-2a2414d733a8 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:07.713446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac251a1-2981-443d-98d7-2a2414d733a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:07.715131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac251a1-2981-443d-98d7-2a2414d733a8 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:07.730722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 503/1014] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:33:07 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:33:07.795290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfdc16-eb04-4b7c-8377-5c02ce7ee7e3 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:07.795834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfdc16-eb04-4b7c-8377-5c02ce7ee7e3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:07.795834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfdc16-eb04-4b7c-8377-5c02ce7ee7e3 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:07.797421 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 512/1015] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:07.803303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735292-7343-4029-84a3-0f4b49d64831 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:07.803609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735292-7343-4029-84a3-0f4b49d64831 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:07.803874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735292-7343-4029-84a3-0f4b49d64831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:07.823814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-84735292-7343-4029-84a3-0f4b49d64831 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:07.824978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-84735292-7343-4029-84a3-0f4b49d64831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:07.864516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:07.908458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 504/1016] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:07 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:08.236436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfdc16-eb04-4b7c-8377-5c02ce7ee7e3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:08.240779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:08.343950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfdc16-eb04-4b7c-8377-5c02ce7ee7e3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:08.382257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:08.382886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:08.383167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:08.383442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] PATH_INFO: `/v3/services/compute` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:08.384017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:08.385406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:08.394744 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:08.395746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:08.397027 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 513/1017] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:33:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:08.401665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84735292-7343-4029-84a3-0f4b49d64831 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:08.504799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84735292-7343-4029-84a3-0f4b49d64831 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:08.538337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:08.539061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:08.539262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:08.539517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:08.540176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] RBAC: Adding query filter params (name=compute) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:08.540487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:08.541927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:08.547743 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 505/1018] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:33:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:08.554010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:08.622288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:08.658181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeb36c-fcf2-44f3-859c-142a95e1cddb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:08.692947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fabf69-d27b-4295-b204-856c237faae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:08.693556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fabf69-d27b-4295-b204-856c237faae0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:08.693857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fabf69-d27b-4295-b204-856c237faae0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:08.694145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fabf69-d27b-4295-b204-856c237faae0 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:08.694778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fabf69-d27b-4295-b204-856c237faae0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:08.695084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fabf69-d27b-4295-b204-856c237faae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:08.696452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fabf69-d27b-4295-b204-856c237faae0 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:08.701484 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 514/1019] 10.222.0.33 () {56 vars in 1112 bytes} [Sat Sep 10 20:33:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:08.707475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:08.739511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:08.810097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149862-cb94-47d3-a821-33064afbf16e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:08.855307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280896ca-9aca-4ac7-8544-2ab74fe41996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:08.855921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280896ca-9aca-4ac7-8544-2ab74fe41996 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:08.856205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280896ca-9aca-4ac7-8544-2ab74fe41996 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:08.856482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280896ca-9aca-4ac7-8544-2ab74fe41996 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:08.857150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280896ca-9aca-4ac7-8544-2ab74fe41996 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:08.858546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280896ca-9aca-4ac7-8544-2ab74fe41996 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:08.879024 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 506/1020] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:33:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:33:09.929190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2802b-5413-4f86-9075-0c4a6a1a391d None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:09.929730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2802b-5413-4f86-9075-0c4a6a1a391d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:09.929730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2802b-5413-4f86-9075-0c4a6a1a391d None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:09.931160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 515/1021] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:09.936274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc022ff-aac6-456e-9633-88c1301d9adf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:09.936675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc022ff-aac6-456e-9633-88c1301d9adf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:09.936861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc022ff-aac6-456e-9633-88c1301d9adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:09.956725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fcc022ff-aac6-456e-9633-88c1301d9adf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:09.958144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fcc022ff-aac6-456e-9633-88c1301d9adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:10.036094 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 507/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:10.197663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ceab-8698-45ad-81e8-51d2440d93c0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:10.197988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ceab-8698-45ad-81e8-51d2440d93c0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:10.198171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ceab-8698-45ad-81e8-51d2440d93c0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:10.199624 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 516/1023] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:10.204608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92105d81-c927-48d4-932f-27cd02982de8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:10.204899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92105d81-c927-48d4-932f-27cd02982de8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:10.205173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92105d81-c927-48d4-932f-27cd02982de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:10.224263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-92105d81-c927-48d4-932f-27cd02982de8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:10.225444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-92105d81-c927-48d4-932f-27cd02982de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:10.233459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:10.295191 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 508/1024] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:10.370191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1ceab-8698-45ad-81e8-51d2440d93c0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:10.475012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1ceab-8698-45ad-81e8-51d2440d93c0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:10.503872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:10.510089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:10.510721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:10.511003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:10.511278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:10.512447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:10.514028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:10.515746 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 517/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:10.521669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92105d81-c927-48d4-932f-27cd02982de8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:10.621109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:10.621732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92105d81-c927-48d4-932f-27cd02982de8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:10.658506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:10.659175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:10.659458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:10.659733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:10.660394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:10.661856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:10.679119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:10.690671 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 509/1026] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:33:10 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:33:10.733696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246557d-0997-4776-b176-b97bcfa8c5d2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:10.773079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:10.773815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:10.774189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:10.774523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] PATH_INFO: `/v3/services/image` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:10.775176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:10.776618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:10.786875 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:10.788208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:10.789319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:10.794677 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 518/1027] 10.222.0.33 () {56 vars in 1105 bytes} [Sat Sep 10 20:33:10 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:10.795638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:10.911730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e52e9-29ab-4da4-96d0-4ec81715ec8e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:10.950319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:10.950945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:10.951231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:10.951510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:10.952147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] RBAC: Adding query filter params (name=image) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:10.952456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:10.953926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:10.958983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 510/1028] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:10.965220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:11.041633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:11.084361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d26d05-d062-4e9f-a242-196fab34bc17 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:11.120797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:11.121408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:11.121687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:11.121995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:11.122613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] RBAC: Adding query filter params (type=image) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:11.122909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:11.124315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:11.130045 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 519/1029] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:11.135637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:11.198617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:11.239895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80680ff0-1fbe-42e6-9647-c12d898293f8 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:11.274776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f07e76-375d-4041-87c5-88577a9bf29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:11.275385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f07e76-375d-4041-87c5-88577a9bf29e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:11.275663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f07e76-375d-4041-87c5-88577a9bf29e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:11.275933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f07e76-375d-4041-87c5-88577a9bf29e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:11.276624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f07e76-375d-4041-87c5-88577a9bf29e None admin] RBAC: Adding query filter params (interface=public, service_id=37f202ff9cdf400b867cefd9707d580c, region_id=RegionOne) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:11.276930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f07e76-375d-4041-87c5-88577a9bf29e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:11.278442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f07e76-375d-4041-87c5-88577a9bf29e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:11.283962 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 511/1030] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:33:11 2022] GET /identity/v3/endpoints?service_id=37f202ff9cdf400b867cefd9707d580c&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:11.289611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:11.393984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8678c15-3479-41c3-9f49-324e30c7769f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:11.419078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ecb9-f558-4b59-adbe-1a8164c82b12 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:11.419386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ecb9-f558-4b59-adbe-1a8164c82b12 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:11.419643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ecb9-f558-4b59-adbe-1a8164c82b12 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:11.420898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:11.421217 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 512/1031] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:11.427139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6739f-885b-4f4a-9127-dfb24e7511f6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:11.427435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6739f-885b-4f4a-9127-dfb24e7511f6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:11.427713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6739f-885b-4f4a-9127-dfb24e7511f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:11.431923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:11.432854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:11.433352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:11.433856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:11.434629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:11.436125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:11.442903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 520/1032] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:33:11 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:33:11.451119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-93c6739f-885b-4f4a-9127-dfb24e7511f6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:11.452373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93c6739f-885b-4f4a-9127-dfb24e7511f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:11.528458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 513/1033] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:11.863256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:11.970217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5d835-6df4-4118-ac33-8d7a9950d108 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:12.008543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:12.009172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:12.009451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:12.009724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:12.020778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:12.025612 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:12.027011 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:12.028075 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 521/1034] 10.222.0.33 () {56 vars in 1120 bytes} [Sat Sep 10 20:33:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:12.034163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6739f-885b-4f4a-9127-dfb24e7511f6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:12.091238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:12.136861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6739f-885b-4f4a-9127-dfb24e7511f6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:12.171861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:12.172472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:12.172755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:12.173028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:12.173688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:12.174110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:12.175774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:12.188663 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 514/1035] 10.222.0.33 () {56 vars in 1115 bytes} [Sat Sep 10 20:33:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:12.190978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:12.245610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:12.294299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1132d-7d80-4333-8074-67efbf7eccc2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:12.328153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:12.328776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:12.329060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:12.329335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:12.340538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:12.342078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:12.356204 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:12.357160 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:12.358521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:12.358521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:12.358521 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:12.358521 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 522/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:12.363609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:12.465147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535876a-313e-4d0e-b1f5-81805971e05d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:12.481851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:12.501525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:12.502154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:12.503110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:12.503362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:12.503996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:12.504299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:12.505665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:12.517389 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 515/1037] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:12.522964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:12.648028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3825b5-6fbf-4a93-9952-30e7643a2a76 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:12.651963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:12.705029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:12.706409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:12.706986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:12.707213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:12.715444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:12.716812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:33:12.728345 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:12.729319 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:12.735387 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 523/1038] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:33:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:12.737562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:12.846386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb1510-b855-4c83-a7b5-5f7beaf7bceb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:12.884514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6013d-8d55-46d0-8ea1-08a14a9c4cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:12.885125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6013d-8d55-46d0-8ea1-08a14a9c4cb3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:12.885409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6013d-8d55-46d0-8ea1-08a14a9c4cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:12.885686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6013d-8d55-46d0-8ea1-08a14a9c4cb3 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:12.886449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6013d-8d55-46d0-8ea1-08a14a9c4cb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:12.886759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6013d-8d55-46d0-8ea1-08a14a9c4cb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:12.888184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6013d-8d55-46d0-8ea1-08a14a9c4cb3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:12.911354 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 516/1039] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:33:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:12.912799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.021190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071e1d1-ed1c-472b-9421-00baef8b06ba None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:13.055548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:13.056163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.056446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.056721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.091119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:13.131234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:13.132692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:13.216361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddb76e-b529-4fee-932f-30de0df3281e None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.216877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddb76e-b529-4fee-932f-30de0df3281e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.216985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddb76e-b529-4fee-932f-30de0df3281e None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.218388 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 517/1040] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:13.224269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5293fb-c0ee-4bca-9272-45207ac79628 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.224554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5293fb-c0ee-4bca-9272-45207ac79628 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.224829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5293fb-c0ee-4bca-9272-45207ac79628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.239958 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 524/1041] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:33:13.241370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.246033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6b5293fb-c0ee-4bca-9272-45207ac79628 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:13.247215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b5293fb-c0ee-4bca-9272-45207ac79628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:13.279631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:13.309054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:13.323994 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 518/1042] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:13.349814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c2cc-cedf-4e45-8784-258f47568ead None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:13.387137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:13.387773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.388056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.388333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.398629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:13.400136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:13.415285 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:13.416278 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:13.417208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:13.417208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:13.417208 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:13.421352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 525/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:13.422627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5293fb-c0ee-4bca-9272-45207ac79628 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.527544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5293fb-c0ee-4bca-9272-45207ac79628 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:13.564135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:13.564745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.565591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.565894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.566579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:13.566881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:13.568241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:13.584413 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 519/1044] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:13.585981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.652500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.682411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:13.700178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5472b-2b5f-4662-a996-3e037d09b402 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:13.718830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:13.739099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:13.740039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.740500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.740948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.748160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:13.750066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:33:13.762749 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:13.763654 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:13.764566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:13.765010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b09b1-e976-4d21-9587-52d297b07e0f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:13.766305 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 526/1045] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:13.772476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.805364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:13.806035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.806309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.806577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.817915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:13.823246 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:13.824210 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:13.825066 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 520/1046] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:13.830169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.887095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c83755-03a8-491e-86d5-12db2c3c5e87 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:13.923140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:13.923752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.924045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.924855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.924963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:13.925805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:13.926187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:13.927588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:13.940912 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 527/1047] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:13.941617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65736ab1-6838-4b23-bcf1-df2c3b337e96 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:13.946693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:13.977802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615e932-bbba-4655-8b2c-a391b33588f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:13.978437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615e932-bbba-4655-8b2c-a391b33588f1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.978717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615e932-bbba-4655-8b2c-a391b33588f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.978988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615e932-bbba-4655-8b2c-a391b33588f1 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.979649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615e932-bbba-4655-8b2c-a391b33588f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:13.979951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615e932-bbba-4655-8b2c-a391b33588f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:13.981304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615e932-bbba-4655-8b2c-a391b33588f1 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:13.988534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:13.990701 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 521/1048] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:13.991769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaeada-d4b6-42d5-9b59-a39576dfb179 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:13.992099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaeada-d4b6-42d5-9b59-a39576dfb179 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:13.992374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaeada-d4b6-42d5-9b59-a39576dfb179 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:13.993703 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 522/1049] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:13.998281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaeada-d4b6-42d5-9b59-a39576dfb179 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:14.054237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da7df8-a686-4ad4-90f4-36d8a4c1bd06 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:14.088697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9370b60-85db-4135-9404-b270691589bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:14.089483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9370b60-85db-4135-9404-b270691589bf None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.089760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9370b60-85db-4135-9404-b270691589bf None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.090056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9370b60-85db-4135-9404-b270691589bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.090766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9370b60-85db-4135-9404-b270691589bf None admin] RBAC: Adding query filter params (role.id=2bd6237d9303489c830fba54a2be93f5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=6307d15dca3043b987a28fc41515acc2) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:14.091067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9370b60-85db-4135-9404-b270691589bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:14.092487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9370b60-85db-4135-9404-b270691589bf None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:14.098361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 528/1050] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/role_assignments?user.id=6307d15dca3043b987a28fc41515acc2&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=2bd6237d9303489c830fba54a2be93f5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:14.099545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984dcb8-1027-492b-ad8f-6375066dd48e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.099831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984dcb8-1027-492b-ad8f-6375066dd48e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.100103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984dcb8-1027-492b-ad8f-6375066dd48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.100671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaeada-d4b6-42d5-9b59-a39576dfb179 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:14.119844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2984dcb8-1027-492b-ad8f-6375066dd48e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:14.121065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2984dcb8-1027-492b-ad8f-6375066dd48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:14.145468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:14.146151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.146439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.146709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] PATH_INFO: `/v3/users/2bff4b2220c5467188eb439fb46aaf07` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.161926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] RBAC: Authorizing `identity:get_user(user_id=2bff4b2220c5467188eb439fb46aaf07)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:14.163428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:14.169981 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 523/1051] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:13 2022] GET /identity/v3/users/2bff4b2220c5467188eb439fb46aaf07 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:14.171154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:14.180946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:14.204608 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 529/1052] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:14.285176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4baead-4b27-45c9-9e61-d335cb0613f5 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:14.322754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:14.323501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.323879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.324247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.395001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:14.396519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:14.405029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:14.502061 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 524/1053] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:14 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:14.511072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984dcb8-1027-492b-ad8f-6375066dd48e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:14.542183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:14.542367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:14.619736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984dcb8-1027-492b-ad8f-6375066dd48e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:14.657365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:14.657975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.658259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.658582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.659206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7309e-1db3-41a0-9444-5d9434e6721b None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:14.659696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] RBAC: Authorizing `identity:get_project(project_id=7317e7d77ffa4d178b9b74c92ed47212)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:14.661280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:14.667970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 530/1054] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:14 2022] GET /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:14.668158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:14.670576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:14.702333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:14.703049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.703398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.703739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] PATH_INFO: `/v3/services/image` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.704381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:14.706059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:14.715054 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:14.716426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:14.717538 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 525/1055] 10.222.0.33 () {56 vars in 1105 bytes} [Sat Sep 10 20:33:14 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:14.722238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:14.777247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:14.781912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392615f-28c2-4ae3-ab04-50f98ba7516f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:14.820345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:14.821068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.821334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.821721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.822461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=7317e7d77ffa4d178b9b74c92ed47212, user.id=2bff4b2220c5467188eb439fb46aaf07) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:14.822765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:14.824353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:14.827519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98531f1e-9bbb-4702-a351-5f6e7219fbab None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:14.830101 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 531/1056] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:14 2022] GET /identity/v3/role_assignments?user.id=2bff4b2220c5467188eb439fb46aaf07&scope.project.id=7317e7d77ffa4d178b9b74c92ed47212&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:14.878749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:14.879377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:14.879659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:14.879931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:14.880568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] RBAC: Adding query filter params (name=image) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:14.880873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:14.882290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:14.888055 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 526/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:14.894309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:15.020705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f148182-4183-4e21-8c06-314d19d2925c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:15.058666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ebe99-46dd-4b83-a715-7aaa1c314232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:15.059311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ebe99-46dd-4b83-a715-7aaa1c314232 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:15.059541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ebe99-46dd-4b83-a715-7aaa1c314232 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:15.059805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ebe99-46dd-4b83-a715-7aaa1c314232 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:15.060422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ebe99-46dd-4b83-a715-7aaa1c314232 None admin] RBAC: Adding query filter params (type=image) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:15.060720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ebe99-46dd-4b83-a715-7aaa1c314232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:15.062162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ebe99-46dd-4b83-a715-7aaa1c314232 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:15.067814 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 532/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:15.073302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:15.109691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:15.179787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d4ee-9cbd-4a85-a230-d31b83ba2759 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:15.215940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5687f-16bc-4a0b-8198-70e1f1658b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:15.216544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5687f-16bc-4a0b-8198-70e1f1658b08 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:15.216830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5687f-16bc-4a0b-8198-70e1f1658b08 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:15.217107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5687f-16bc-4a0b-8198-70e1f1658b08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:15.217773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5687f-16bc-4a0b-8198-70e1f1658b08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:15.219157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5687f-16bc-4a0b-8198-70e1f1658b08 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:15.245933 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 527/1059] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:33:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:33:16.537596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162ff7-8524-4d39-8951-628a45491cd9 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:16.538174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162ff7-8524-4d39-8951-628a45491cd9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:16.538174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162ff7-8524-4d39-8951-628a45491cd9 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:16.539539 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 533/1060] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:16.544565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0017-3d62-48e3-bcc7-2db8d7b8f8e8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:16.544947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0017-3d62-48e3-bcc7-2db8d7b8f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:16.545121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0017-3d62-48e3-bcc7-2db8d7b8f8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:16.563433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-74ae0017-3d62-48e3-bcc7-2db8d7b8f8e8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:16.564619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-74ae0017-3d62-48e3-bcc7-2db8d7b8f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:16.641193 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 528/1061] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:16.974286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1162ff7-8524-4d39-8951-628a45491cd9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.003088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:17.080804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1162ff7-8524-4d39-8951-628a45491cd9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:17.116487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d45e20-e181-4bb4-829b-a5d3943bb0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:17.117116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d45e20-e181-4bb4-829b-a5d3943bb0d0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.117399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d45e20-e181-4bb4-829b-a5d3943bb0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.117673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d45e20-e181-4bb4-829b-a5d3943bb0d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.127843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d45e20-e181-4bb4-829b-a5d3943bb0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-28d45e20-e181-4bb4-829b-a5d3943bb0d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:17.132862 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:17.133820 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:33:17.135017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:17.136135 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 534/1062] 10.222.0.33 () {56 vars in 1120 bytes} [Sat Sep 10 20:33:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:17.139951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae0017-3d62-48e3-bcc7-2db8d7b8f8e8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.157918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:17.244766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae0017-3d62-48e3-bcc7-2db8d7b8f8e8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:17.283416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:17.284566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.284850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.285106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.285775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:17.286224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:17.287657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:17.301559 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 529/1063] 10.222.0.33 () {56 vars in 1115 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:17.301745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb73b85-5bcc-4c4d-92e0-7cc757972559 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.301745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb73b85-5bcc-4c4d-92e0-7cc757972559 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.301745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb73b85-5bcc-4c4d-92e0-7cc757972559 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.301745 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 535/1064] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:33:17.302975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.304929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583e780-9c85-40d3-85b0-b5939b1341ec None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.305217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583e780-9c85-40d3-85b0-b5939b1341ec None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.305490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583e780-9c85-40d3-85b0-b5939b1341ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.323462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0583e780-9c85-40d3-85b0-b5939b1341ec None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:17.324639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0583e780-9c85-40d3-85b0-b5939b1341ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:17.389605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:17.394781 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 536/1065] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:17.414619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d486c-7687-46fb-9b6c-706473b5e110 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:17.449543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:17.450198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.450479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.450752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.462859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:17.464591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:17.479326 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:17.480296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:17.481640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:17.481640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:17.481640 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:17.481640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 530/1066] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:17.486281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583e780-9c85-40d3-85b0-b5939b1341ec None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.504832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:17.588807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583e780-9c85-40d3-85b0-b5939b1341ec None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:17.624265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9a8a7-9a5f-461c-8e5f-cc7ac96a361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:17.624871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a8a7-9a5f-461c-8e5f-cc7ac96a361a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.625151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a8a7-9a5f-461c-8e5f-cc7ac96a361a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.625423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a8a7-9a5f-461c-8e5f-cc7ac96a361a None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.626192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9a8a7-9a5f-461c-8e5f-cc7ac96a361a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:17.626497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9a8a7-9a5f-461c-8e5f-cc7ac96a361a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:17.628533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9a8a7-9a5f-461c-8e5f-cc7ac96a361a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:17.640156 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 537/1067] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:17.645618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.687140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92615e-9a7b-41d1-94a4-f7fe19574866 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.687452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92615e-9a7b-41d1-94a4-f7fe19574866 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.687709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92615e-9a7b-41d1-94a4-f7fe19574866 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.689190 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 538/1068] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:17.694665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca43c5-d2f2-4a71-b00a-01349287e44c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.694955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca43c5-d2f2-4a71-b00a-01349287e44c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.695235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca43c5-d2f2-4a71-b00a-01349287e44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.713371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-10ca43c5-d2f2-4a71-b00a-01349287e44c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:17.714845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10ca43c5-d2f2-4a71-b00a-01349287e44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:17.748473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9a24-92e0-45c2-bfec-62da8903b97f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:17.778246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:17.783340 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 539/1069] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 20:33:17.784470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca43c5-d2f2-4a71-b00a-01349287e44c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.791325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:17.792123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:17.792370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.792664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.792936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.802891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:17.804341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:33:17.816218 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:17.817184 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:33:17.818191 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:17.822888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 531/1070] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:17.823709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.907719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca43c5-d2f2-4a71-b00a-01349287e44c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:17.927136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599c56b-43a1-4fb8-8eb7-456a8e0ee246 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:17.944767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:17.945450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.945736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.946038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.956868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:17.961985 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:17.962965 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:17.964504 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 540/1071] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:17.968826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:17.969446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:17.969729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:17.970057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:17.970159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:17.970779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:17.971121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:17.972551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:17.992227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 532/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:17.997688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.042986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.052478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.081990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dfdbb-8461-411b-bad8-8319c64ced58 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.110534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65130b-6a06-499d-b276-eb0faf37b922 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.116343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.116974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.117253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.117523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.118214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:18.118516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.119914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.129866 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 541/1073] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:18.129866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.151060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.151670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.151965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.152234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.163189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.164615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:18.179548 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:18.180726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:18.181723 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:18.181723 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:18.181723 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:18.181723 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 533/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:18.181723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.233182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dc7c3-e6c3-413e-b5fb-3a0b2323de11 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.258719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.267350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.268462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.268741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.269012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.279224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.282776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92805194-fcee-4608-8364-b5bd3fb333d8 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.317097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.317726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.318047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.318324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] PATH_INFO: `/v3/users/2bff4b2220c5467188eb439fb46aaf07` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.319452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] RBAC: Authorizing `identity:get_user(user_id=2bff4b2220c5467188eb439fb46aaf07)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.320884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.322473 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 534/1075] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/users/2bff4b2220c5467188eb439fb46aaf07 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:18.323231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.336232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.337673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.423753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056b514-8dec-4b4c-9ea1-c50c07c1667f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.442317 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 542/1076] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:18.443158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.461367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.462015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.462325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.462577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.463209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:18.463514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.465124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.473173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.476803 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 535/1077] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:18.477637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.504823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.551064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edd2c3-1636-4978-9327-adaface9a312 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.575861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f444a-9137-4cc1-b8d1-834eda28d119 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.585010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.585638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.585970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.586236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.587344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] RBAC: Authorizing `identity:get_project(project_id=7317e7d77ffa4d178b9b74c92ed47212)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.588804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.590734 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 543/1078] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:18.591490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.612768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.613792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.614176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.614518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.624511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.626126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:18.640911 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:18.642170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:18.644059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:18.644059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:18.644059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:18.644059 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 536/1079] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:18.644059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.689067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.696106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba81f54-27f1-47fb-b853-315d3d5f9235 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.723598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:18.730387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.731006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.731284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.731559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.738214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.739646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.743295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c32a9-1ee2-4515-bd14-0ad6f8e11715 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:33:18.750478 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:18.751558 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 544/1080] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:18.753292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.782694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.783339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.783732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.784224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212/users/2bff4b2220c5467188eb439fb46aaf07/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.786209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7317e7d77ffa4d178b9b74c92ed47212, user_id=2bff4b2220c5467188eb439fb46aaf07, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.787624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.800517 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 537/1081] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:33:18 2022] PUT /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212/users/2bff4b2220c5467188eb439fb46aaf07/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:33:18.801305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:18.864433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124e0ea-f6c3-4411-8a9b-2454f63280e0 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.907460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.909028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.909546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.910298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.911069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:18.911386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.912789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:18.937856 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 545/1082] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:18.938225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426ed1d-3124-4cbe-8308-956f26c6bed9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:18.986100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c2c18-262c-4d20-acf8-d84454c49c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:18.987403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2c18-262c-4d20-acf8-d84454c49c8f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:18.987753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2c18-262c-4d20-acf8-d84454c49c8f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:18.988106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2c18-262c-4d20-acf8-d84454c49c8f None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:18.988885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c2c18-262c-4d20-acf8-d84454c49c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:18.989264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c2c18-262c-4d20-acf8-d84454c49c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:18.990896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c2c18-262c-4d20-acf8-d84454c49c8f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:19.003879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:19.010006 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 538/1083] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:33:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:19.015501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:19.038802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:19.129963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619126b-f4c8-471a-a251-4d978f67e854 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:19.167161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af74ee3-b034-481a-856d-6c8b3b107b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:19.167886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74ee3-b034-481a-856d-6c8b3b107b14 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:19.168174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74ee3-b034-481a-856d-6c8b3b107b14 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:19.168450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74ee3-b034-481a-856d-6c8b3b107b14 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/6307d15dca3043b987a28fc41515acc2/roles/2bd6237d9303489c830fba54a2be93f5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:19.170625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af74ee3-b034-481a-856d-6c8b3b107b14 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=6307d15dca3043b987a28fc41515acc2, role_id=2bd6237d9303489c830fba54a2be93f5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:19.172132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af74ee3-b034-481a-856d-6c8b3b107b14 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:19.187527 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 546/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:33:19 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/6307d15dca3043b987a28fc41515acc2/roles/2bd6237d9303489c830fba54a2be93f5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:33:21.256040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3fc59-8a1f-404a-80cf-991b0e47a0a5 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:21.256548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3fc59-8a1f-404a-80cf-991b0e47a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:21.256548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3fc59-8a1f-404a-80cf-991b0e47a0a5 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:21.257809 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 539/1085] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:21.263156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b0c22-8e0b-4483-bc2b-c4a1b65d9134 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:21.263556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b0c22-8e0b-4483-bc2b-c4a1b65d9134 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:21.263710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b0c22-8e0b-4483-bc2b-c4a1b65d9134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:21.282420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ad9b0c22-8e0b-4483-bc2b-c4a1b65d9134 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:21.284293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad9b0c22-8e0b-4483-bc2b-c4a1b65d9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:21.351625 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 547/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:21.608891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995ff2-ee4c-4dbb-9b86-7dc7fc75d89f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:21.609205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995ff2-ee4c-4dbb-9b86-7dc7fc75d89f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:21.609483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995ff2-ee4c-4dbb-9b86-7dc7fc75d89f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:21.610838 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 540/1087] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:21.616440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26613-6597-4432-a19a-f3bd89a00fc9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:21.616879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26613-6597-4432-a19a-f3bd89a00fc9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:21.617162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26613-6597-4432-a19a-f3bd89a00fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:21.633371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:21.640470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1bf26613-6597-4432-a19a-f3bd89a00fc9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:21.642314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1bf26613-6597-4432-a19a-f3bd89a00fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:21.678196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f995ff2-ee4c-4dbb-9b86-7dc7fc75d89f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:21.711298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 548/1088] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:21.784987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f995ff2-ee4c-4dbb-9b86-7dc7fc75d89f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:21.819375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:21.820182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:21.820459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:21.820752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:21.832746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:21.839702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:21.840633 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:21.841495 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 541/1089] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:33:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:21.847932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26613-6597-4432-a19a-f3bd89a00fc9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:21.947301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26613-6597-4432-a19a-f3bd89a00fc9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:21.981288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:21.981917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:21.982198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:21.982467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:21.983122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:21.983422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:21.984784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:21.988146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:21.995178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 549/1090] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:33:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:22.001118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.039450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.060383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:22.103791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e4dc-7f93-4fc0-85f5-9fdc7b40f1a7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.137564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.138219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.138495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.138764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] PATH_INFO: `/v3/users/2bff4b2220c5467188eb439fb46aaf07` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.139867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] RBAC: Authorizing `identity:get_user(user_id=2bff4b2220c5467188eb439fb46aaf07)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.141283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.141731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22751157-5d2a-4deb-a803-f87b233884eb None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.143371 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 542/1091] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/users/2bff4b2220c5467188eb439fb46aaf07 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:22.149059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.176663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.177292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.177569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.177869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.188247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:22.193561 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:22.194552 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:22.195407 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 550/1092] 10.222.0.33 () {56 vars in 1120 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:22.200839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.248689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e67871-f6ef-4156-a7d7-681d3f19b6e4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.259130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:22.275368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:22.283619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.284214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.284488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.284764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.301794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac76f0-a6d6-43ed-aad4-104bc8be016b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.335273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.335872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.336154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.336427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.337079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:22.337385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.338777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.346989 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 551/1093] 10.222.0.33 () {56 vars in 1115 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:22.352740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.353060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.354532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.451540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e65723-91b3-4a4b-aca2-6a3679d84afc None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.454687 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 543/1094] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 20:33:22.461264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.472544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:22.487402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.488031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.488164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:22.488309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.488583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.499222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.500661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:22.515639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:22.516720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:22.517641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:22.517641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:22.517641 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:22.517641 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 552/1095] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:22.523033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.561065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90f116-f3a0-4ecf-8c9e-65b6807abc2a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.597504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.598168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.598451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.598724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.599840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] RBAC: Authorizing `identity:get_project(project_id=7317e7d77ffa4d178b9b74c92ed47212)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.601260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.602902 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 544/1096] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:22.608895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.624408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff7d3f-3501-46df-b334-e1c76074c79d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.656993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.657594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.657912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.658190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.659496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:22.659688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.661089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.673566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 553/1097] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:22.680021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.704739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:22.710012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:22.712055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3e7f-56b9-41c9-81b3-8b129d4d7569 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.746013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.746610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.746895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.747169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.747886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=7317e7d77ffa4d178b9b74c92ed47212, user.id=2bff4b2220c5467188eb439fb46aaf07) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:22.748195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.749612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.755555 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 545/1098] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/role_assignments?user.id=2bff4b2220c5467188eb439fb46aaf07&scope.project.id=7317e7d77ffa4d178b9b74c92ed47212&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:22.782613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae271e-69ec-4136-b20d-4af6cca92984 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.819577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:22.820221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:22.820499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:22.820770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:22.828949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:22.831053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:33:22.844020 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:22.845045 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:22.846624 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 554/1099] 10.222.0.33 () {56 vars in 1093 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:33:22.851856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:22.959553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a48f49-e3eb-4557-a695-f9f3d0b56420 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:22.999854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:23.000462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:23.000746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:23.001021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:23.001744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:23.002164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:23.003656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:23.021528 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 546/1100] 10.222.0.33 () {56 vars in 1133 bytes} [Sat Sep 10 20:33:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:23.027738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:23.119551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:23.129307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ea84f-98c2-4cd7-b0f8-9c9addbdefb3 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:23.162861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:23.163461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:23.163741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:23.164012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:23.230936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:23.232302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:23.320554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:23.331557 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 555/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:23.337343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:23.356471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:23.438415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cb122-4455-40ea-a8de-f0cb1c68245d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:23.472377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:23.472997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:23.473281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:23.473557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:23.483491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:23.484861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:23.499193 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:23.500147 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:33:23.501050 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:23.501050 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:33:23.501050 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:23.501050 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 547/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:23.506080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:23.604177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ceab5-4b31-4bfd-a078-048630df63ba None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:23.637479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:23.638109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:23.638393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:23.638668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:23.639300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:23.639602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:23.640960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:23.652454 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 556/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:23.658156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:23.727870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:23.759690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3505a91-83de-4bae-a92d-01bd85397857 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:23.791509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:23.792127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:23.792408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:23.792683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:23.799638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:23.801116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:33:23.812101 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:23.813175 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:23.814422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 548/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:33:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:23.819615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:23.873425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:23.920751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19046-e838-44b3-b25b-3ac6f931ee50 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:23.954209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5c22b-659e-4321-899d-e48d0f1f359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:23.954813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5c22b-659e-4321-899d-e48d0f1f359d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:23.955091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5c22b-659e-4321-899d-e48d0f1f359d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:23.955720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5c22b-659e-4321-899d-e48d0f1f359d None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:23.956409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5c22b-659e-4321-899d-e48d0f1f359d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:23.956714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5c22b-659e-4321-899d-e48d0f1f359d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:23.958205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5c22b-659e-4321-899d-e48d0f1f359d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:23.970628 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 557/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:33:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:23.977237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:24.077547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b3bed-5c24-46c7-99ce-6f80be548fe8 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:24.105609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:24.111912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379109c2-fe0a-4b89-b657-9fc66ee73c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:24.112508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379109c2-fe0a-4b89-b657-9fc66ee73c49 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:24.112788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379109c2-fe0a-4b89-b657-9fc66ee73c49 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:24.113062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379109c2-fe0a-4b89-b657-9fc66ee73c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:24.113780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379109c2-fe0a-4b89-b657-9fc66ee73c49 None admin] RBAC: Adding query filter params (role.id=2bd6237d9303489c830fba54a2be93f5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=6307d15dca3043b987a28fc41515acc2) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:24.114123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379109c2-fe0a-4b89-b657-9fc66ee73c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:24.115577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379109c2-fe0a-4b89-b657-9fc66ee73c49 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:24.121235 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 549/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:23 2022] GET /identity/v3/role_assignments?user.id=6307d15dca3043b987a28fc41515acc2&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=2bd6237d9303489c830fba54a2be93f5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:25.226702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1122e-b913-4794-841a-7b3310296313 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:25.227220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1122e-b913-4794-841a-7b3310296313 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:25.227220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1122e-b913-4794-841a-7b3310296313 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:25.228486 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 558/1107] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:25.233631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950a9fc-c656-488d-9cbc-725da53d846b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:25.234058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950a9fc-c656-488d-9cbc-725da53d846b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:25.234211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950a9fc-c656-488d-9cbc-725da53d846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:25.252143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5950a9fc-c656-488d-9cbc-725da53d846b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:25.253347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5950a9fc-c656-488d-9cbc-725da53d846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:25.318078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 550/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:25.647424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1122e-b913-4794-841a-7b3310296313 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:25.748217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1122e-b913-4794-841a-7b3310296313 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:25.761537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:25.782685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ace4a6-2213-425e-ac5d-0d5c4c513c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:25.783357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ace4a6-2213-425e-ac5d-0d5c4c513c3a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:25.784167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ace4a6-2213-425e-ac5d-0d5c4c513c3a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:25.784409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ace4a6-2213-425e-ac5d-0d5c4c513c3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:25.785491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ace4a6-2213-425e-ac5d-0d5c4c513c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:25.786951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ace4a6-2213-425e-ac5d-0d5c4c513c3a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:25.788504 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 559/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:25.794311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950a9fc-c656-488d-9cbc-725da53d846b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:25.890006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950a9fc-c656-488d-9cbc-725da53d846b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:25.913965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:25.924884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503e90a-ce79-4dd9-93de-b2ab8c310abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:25.925473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503e90a-ce79-4dd9-93de-b2ab8c310abc None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:25.925751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503e90a-ce79-4dd9-93de-b2ab8c310abc None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:25.926049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503e90a-ce79-4dd9-93de-b2ab8c310abc None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:25.926682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503e90a-ce79-4dd9-93de-b2ab8c310abc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:25.928091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503e90a-ce79-4dd9-93de-b2ab8c310abc None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:25.949138 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 551/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:33:25 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:33:28.388000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329f45-7152-4d9c-bacd-b9f7a54900b9 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:28.388531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329f45-7152-4d9c-bacd-b9f7a54900b9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:28.388531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329f45-7152-4d9c-bacd-b9f7a54900b9 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:28.389894 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 560/1111] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:28.395811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e3f3-700f-4eee-8410-f0c2b899f27b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:28.396265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e3f3-700f-4eee-8410-f0c2b899f27b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:28.396367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e3f3-700f-4eee-8410-f0c2b899f27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:28.414063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7805e3f3-700f-4eee-8410-f0c2b899f27b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:28.415245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7805e3f3-700f-4eee-8410-f0c2b899f27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:28.480041 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 552/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:28.811923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329f45-7152-4d9c-bacd-b9f7a54900b9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:28.911829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329f45-7152-4d9c-bacd-b9f7a54900b9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:28.944087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:28.944700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:28.944982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:28.945256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] PATH_INFO: `/v3/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:28.946661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] RBAC: Authorizing `identity:get_role(role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:28.948032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:28.949328 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 561/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:28 2022] GET /identity/v3/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:28.955774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e3f3-700f-4eee-8410-f0c2b899f27b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:29.051425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e3f3-700f-4eee-8410-f0c2b899f27b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:29.061553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:29.085899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:29.086479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:29.086761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:29.087037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] PATH_INFO: `/v3/users/67b9a5ac6c864050be70366f6793199b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:29.099864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] RBAC: Authorizing `identity:get_user(user_id=67b9a5ac6c864050be70366f6793199b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:29.101292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:29.102931 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 553/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:28 2022] GET /identity/v3/users/67b9a5ac6c864050be70366f6793199b => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:29.108718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:29.156458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:29.206900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1c77-fd3e-4d13-92e5-79d85b0330a1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:29.239529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:29.240127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:29.240408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:29.240682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:29.305001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:29.306480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:29.354993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:29.404927 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 562/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:29.411131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:29.508041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795659fd-01c7-4dde-ac2e-ffa53f268bac None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:29.540732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc3223-baee-41df-9b32-37d26c054eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:29.541338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc3223-baee-41df-9b32-37d26c054eda None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:29.541616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc3223-baee-41df-9b32-37d26c054eda None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:29.541914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc3223-baee-41df-9b32-37d26c054eda None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:29.542997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc3223-baee-41df-9b32-37d26c054eda None admin] RBAC: Authorizing `identity:get_project(project_id=7317e7d77ffa4d178b9b74c92ed47212)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:29.544397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc3223-baee-41df-9b32-37d26c054eda None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:29.545970 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 554/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:29 2022] GET /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:29.552750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:29.648687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6d847-3d06-49f3-8040-90c667b96a6d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:29.682705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7554d5-cb06-4d98-bca9-49201a938127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:29.683377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7554d5-cb06-4d98-bca9-49201a938127 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:29.683660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7554d5-cb06-4d98-bca9-49201a938127 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:29.683933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7554d5-cb06-4d98-bca9-49201a938127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:29.684647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7554d5-cb06-4d98-bca9-49201a938127 None admin] RBAC: Adding query filter params (role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=7317e7d77ffa4d178b9b74c92ed47212, user.id=67b9a5ac6c864050be70366f6793199b) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:29.684951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7554d5-cb06-4d98-bca9-49201a938127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:29.686401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7554d5-cb06-4d98-bca9-49201a938127 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:29.692596 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 563/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:29 2022] GET /identity/v3/role_assignments?user.id=67b9a5ac6c864050be70366f6793199b&scope.project.id=7317e7d77ffa4d178b9b74c92ed47212&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:32.069354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf89a0-8c14-482c-88cc-0458024aad4f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:32.069898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf89a0-8c14-482c-88cc-0458024aad4f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:32.069898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf89a0-8c14-482c-88cc-0458024aad4f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:32.071213 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 555/1118] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:32.076466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb969588-726b-49eb-bf03-231825fb5f9d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:32.076907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb969588-726b-49eb-bf03-231825fb5f9d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:32.077005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb969588-726b-49eb-bf03-231825fb5f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:32.095063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cb969588-726b-49eb-bf03-231825fb5f9d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:32.096848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cb969588-726b-49eb-bf03-231825fb5f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:32.103942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:32.162095 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 564/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:32.494560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf89a0-8c14-482c-88cc-0458024aad4f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:32.531260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:32.596515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf89a0-8c14-482c-88cc-0458024aad4f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:32.630083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:32.630704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:32.630989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:32.631264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] PATH_INFO: `/v3/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:32.632648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] RBAC: Authorizing `identity:get_role(role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:32.634073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:32.635360 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 556/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:32 2022] GET /identity/v3/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:32.641294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb969588-726b-49eb-bf03-231825fb5f9d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:32.739181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb969588-726b-49eb-bf03-231825fb5f9d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:32.772636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:32.773205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:32.773484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:32.773755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] PATH_INFO: `/v3/users/67b9a5ac6c864050be70366f6793199b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:32.774977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] RBAC: Authorizing `identity:get_user(user_id=67b9a5ac6c864050be70366f6793199b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:32.776391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:32.777949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 565/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:32 2022] GET /identity/v3/users/67b9a5ac6c864050be70366f6793199b => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:32.783732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:32.881099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56ef01-ff18-41db-a737-4a4fc45b8eed None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:32.883340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:32.913851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0e202-35af-4033-b33c-f2c19a7d399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:32.914472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0e202-35af-4033-b33c-f2c19a7d399c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:32.914748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0e202-35af-4033-b33c-f2c19a7d399c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:32.915017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0e202-35af-4033-b33c-f2c19a7d399c None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:32.916099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0e202-35af-4033-b33c-f2c19a7d399c None admin] RBAC: Authorizing `identity:get_project(project_id=7317e7d77ffa4d178b9b74c92ed47212)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:32.917612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0e202-35af-4033-b33c-f2c19a7d399c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:32.919207 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 557/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:32 2022] GET /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:32.925714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:32.937454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:33.023722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98b41a-6aa9-4e13-82b0-af80a495721c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:33.056580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d48ed4-3d63-4965-9bdd-a63c8c9e98cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:33.057217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48ed4-3d63-4965-9bdd-a63c8c9e98cb None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:33.057494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48ed4-3d63-4965-9bdd-a63c8c9e98cb None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:33.057764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48ed4-3d63-4965-9bdd-a63c8c9e98cb None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212/users/67b9a5ac6c864050be70366f6793199b/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:33.059867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d48ed4-3d63-4965-9bdd-a63c8c9e98cb None admin] RBAC: Authorizing `identity:create_grant(project_id=7317e7d77ffa4d178b9b74c92ed47212, user_id=67b9a5ac6c864050be70366f6793199b, role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:33.061282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d48ed4-3d63-4965-9bdd-a63c8c9e98cb None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:33.074962 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 566/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:33:32 2022] PUT /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212/users/67b9a5ac6c864050be70366f6793199b/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:33:35.465414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128b840-5b42-43a0-9231-20d3f6833c70 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:35.466002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128b840-5b42-43a0-9231-20d3f6833c70 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:35.466107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128b840-5b42-43a0-9231-20d3f6833c70 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:35.467335 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 558/1124] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:35.472599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3ba6d-36f5-4709-a3a5-167039484fd9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:35.472992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3ba6d-36f5-4709-a3a5-167039484fd9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:35.473145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3ba6d-36f5-4709-a3a5-167039484fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:35.491511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c9c3ba6d-36f5-4709-a3a5-167039484fd9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:35.492717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c9c3ba6d-36f5-4709-a3a5-167039484fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:35.553221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:35.558263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 567/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:35.885918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128b840-5b42-43a0-9231-20d3f6833c70 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:35.987175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128b840-5b42-43a0-9231-20d3f6833c70 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:36.024251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:36.024870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:36.025149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:36.025421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] PATH_INFO: `/v3/roles/50bddbfb052241ab806d8d0f18e5a0c6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:36.026821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] RBAC: Authorizing `identity:get_role(role_id=50bddbfb052241ab806d8d0f18e5a0c6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:36.028179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:36.029469 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 559/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:35 2022] GET /identity/v3/roles/50bddbfb052241ab806d8d0f18e5a0c6 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:36.035574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3ba6d-36f5-4709-a3a5-167039484fd9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:36.132320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3ba6d-36f5-4709-a3a5-167039484fd9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:36.165681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:36.166271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:36.166556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:36.166831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] PATH_INFO: `/v3/users/67b9a5ac6c864050be70366f6793199b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:36.167938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] RBAC: Authorizing `identity:get_user(user_id=67b9a5ac6c864050be70366f6793199b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:36.169364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:36.170947 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 568/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:36 2022] GET /identity/v3/users/67b9a5ac6c864050be70366f6793199b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:36.177245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:36.218376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:36.275761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1d90d-c760-41e4-9504-ff6c9b827f35 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:36.310065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:36.310669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:36.310947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:36.311218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:36.376500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:36.378111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:36.416611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:36.474699 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 560/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:36.480686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:36.546470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:36.578540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded646f-55ea-4ae0-a828-dce16a490318 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:36.613158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022e5bf-9959-4575-91de-338c14cb174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:36.613963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e5bf-9959-4575-91de-338c14cb174a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:36.614247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e5bf-9959-4575-91de-338c14cb174a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:36.614619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e5bf-9959-4575-91de-338c14cb174a None admin] PATH_INFO: `/v3/projects/7317e7d77ffa4d178b9b74c92ed47212` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:36.616073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022e5bf-9959-4575-91de-338c14cb174a None admin] RBAC: Authorizing `identity:get_project(project_id=7317e7d77ffa4d178b9b74c92ed47212)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:36.617489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022e5bf-9959-4575-91de-338c14cb174a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:36.619112 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 569/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:36 2022] GET /identity/v3/projects/7317e7d77ffa4d178b9b74c92ed47212 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:36.625414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:36.722253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bb72e-270a-45f4-b1f7-c4a309a784cb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:36.754308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9adeb-0745-411c-a029-c079459e7dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:36.754956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9adeb-0745-411c-a029-c079459e7dcf None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:36.755248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9adeb-0745-411c-a029-c079459e7dcf None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:36.755520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9adeb-0745-411c-a029-c079459e7dcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:36.756221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9adeb-0745-411c-a029-c079459e7dcf None admin] RBAC: Adding query filter params (role.id=50bddbfb052241ab806d8d0f18e5a0c6, scope.project.id=7317e7d77ffa4d178b9b74c92ed47212, user.id=67b9a5ac6c864050be70366f6793199b) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:36.756524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9adeb-0745-411c-a029-c079459e7dcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:36.757960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9adeb-0745-411c-a029-c079459e7dcf None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:36.763369 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 561/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:36 2022] GET /identity/v3/role_assignments?user.id=67b9a5ac6c864050be70366f6793199b&scope.project.id=7317e7d77ffa4d178b9b74c92ed47212&role.id=50bddbfb052241ab806d8d0f18e5a0c6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:39.396208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896f625-8b29-44e2-9c27-05034382564c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:39.396771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896f625-8b29-44e2-9c27-05034382564c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:39.396771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896f625-8b29-44e2-9c27-05034382564c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:39.398100 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 570/1131] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:39.403323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfec90-c75c-4be4-a510-3ca02854e106 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:39.403710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfec90-c75c-4be4-a510-3ca02854e106 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:39.403891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfec90-c75c-4be4-a510-3ca02854e106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:39.411944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:39.423478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-03cfec90-c75c-4be4-a510-3ca02854e106 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:39.425001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-03cfec90-c75c-4be4-a510-3ca02854e106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:39.489188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 562/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:39.826494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896f625-8b29-44e2-9c27-05034382564c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:39.925676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896f625-8b29-44e2-9c27-05034382564c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:39.958313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f7627-1eb5-4a55-a4ea-23af1a994c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:39.958919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f7627-1eb5-4a55-a4ea-23af1a994c8e None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:39.959202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f7627-1eb5-4a55-a4ea-23af1a994c8e None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:39.959476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f7627-1eb5-4a55-a4ea-23af1a994c8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:39.960600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f7627-1eb5-4a55-a4ea-23af1a994c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:39.962067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f7627-1eb5-4a55-a4ea-23af1a994c8e None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:39.963630 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 571/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:39.969806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfec90-c75c-4be4-a510-3ca02854e106 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:40.067990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfec90-c75c-4be4-a510-3ca02854e106 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:40.100857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f79db-2381-4d29-b553-74d184c123d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:40.101384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f79db-2381-4d29-b553-74d184c123d3 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:40.101662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f79db-2381-4d29-b553-74d184c123d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:40.101966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f79db-2381-4d29-b553-74d184c123d3 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:40.102609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f79db-2381-4d29-b553-74d184c123d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:40.104021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f79db-2381-4d29-b553-74d184c123d3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:40.108376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:40.130526 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 563/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:33:39 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:33:42.561896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ebf11-ec37-40d6-8f92-43f73ababa63 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:42.562459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ebf11-ec37-40d6-8f92-43f73ababa63 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:42.562459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ebf11-ec37-40d6-8f92-43f73ababa63 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:42.563774 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 572/1135] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:42.569583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73adc2a-fb04-448e-9bd8-03ec201ce948 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:42.570057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73adc2a-fb04-448e-9bd8-03ec201ce948 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:42.570186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73adc2a-fb04-448e-9bd8-03ec201ce948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:42.587963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e73adc2a-fb04-448e-9bd8-03ec201ce948 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:42.589137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e73adc2a-fb04-448e-9bd8-03ec201ce948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:42.654530 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 564/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:42.986746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ebf11-ec37-40d6-8f92-43f73ababa63 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:42.990632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:43.089725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ebf11-ec37-40d6-8f92-43f73ababa63 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:43.124165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f13d0-c6a4-4c3e-9faa-c1e7235affcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:43.124789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f13d0-c6a4-4c3e-9faa-c1e7235affcc None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:43.125068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f13d0-c6a4-4c3e-9faa-c1e7235affcc None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:43.125344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f13d0-c6a4-4c3e-9faa-c1e7235affcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:43.126431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f13d0-c6a4-4c3e-9faa-c1e7235affcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:43.128001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f13d0-c6a4-4c3e-9faa-c1e7235affcc None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:43.129554 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 573/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:33:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:43.135367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73adc2a-fb04-448e-9bd8-03ec201ce948 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:43.230558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73adc2a-fb04-448e-9bd8-03ec201ce948 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:43.246061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:43.263784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10572a-ad3c-450d-bcfc-7c3c894d2116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:43.264371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10572a-ad3c-450d-bcfc-7c3c894d2116 None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:43.264649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10572a-ad3c-450d-bcfc-7c3c894d2116 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:43.264920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10572a-ad3c-450d-bcfc-7c3c894d2116 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:43.265563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10572a-ad3c-450d-bcfc-7c3c894d2116 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:43.267015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10572a-ad3c-450d-bcfc-7c3c894d2116 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:43.289057 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 565/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:33:43 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:33:45.719661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043f305-8cfb-479f-ba6f-286abb6d54ae None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:45.720703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043f305-8cfb-479f-ba6f-286abb6d54ae None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:45.720703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043f305-8cfb-479f-ba6f-286abb6d54ae None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:45.722477 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 574/1139] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:45.727731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f185b0-514a-4151-9471-29cad6ddc45d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:45.728138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f185b0-514a-4151-9471-29cad6ddc45d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:45.728286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f185b0-514a-4151-9471-29cad6ddc45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:45.746843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-93f185b0-514a-4151-9471-29cad6ddc45d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:45.748033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93f185b0-514a-4151-9471-29cad6ddc45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:45.813115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 566/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:46.139017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043f305-8cfb-479f-ba6f-286abb6d54ae None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:46.208479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:46.241938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043f305-8cfb-479f-ba6f-286abb6d54ae None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:46.276594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:46.277221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:46.277500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:46.277768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:46.288082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:46.292913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:46.293974 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:46.294913 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:46.298149 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 575/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:46.300707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f185b0-514a-4151-9471-29cad6ddc45d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:46.401957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f185b0-514a-4151-9471-29cad6ddc45d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:46.403588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:46.438218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:46.439696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:46.440064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:46.440337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:46.441026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:46.441324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:46.442731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:46.451605 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 567/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:33:46.458285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:46.555459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2bf68-0135-4c22-9df2-8f4a825059af None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:46.583897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:46.589020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:46.589924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:46.590210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:46.590486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] PATH_INFO: `/v3/users/a178468f906d4f3f928fb9db3453e188` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:46.603952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] RBAC: Authorizing `identity:get_user(user_id=a178468f906d4f3f928fb9db3453e188)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:46.605388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:46.607004 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 576/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:46 2022] GET /identity/v3/users/a178468f906d4f3f928fb9db3453e188 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:46.614052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:46.711340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ed1c7-9153-467c-a7e9-fd6470730059 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:46.743881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:46.744474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:46.744732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:46.745002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:46.776155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:46.812861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:46.814411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:46.909056 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 568/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:46.915220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:47.012360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cd789-81dc-4f3e-88ec-c7f0537b414f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:47.047648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c512aa-ee72-4afe-9ef8-8fc1cec4bc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:47.048458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c512aa-ee72-4afe-9ef8-8fc1cec4bc72 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:47.048962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c512aa-ee72-4afe-9ef8-8fc1cec4bc72 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:47.049261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c512aa-ee72-4afe-9ef8-8fc1cec4bc72 None admin] PATH_INFO: `/v3/projects/2c5612d0798e42de8f668235f7279e04` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:47.050464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c512aa-ee72-4afe-9ef8-8fc1cec4bc72 None admin] RBAC: Authorizing `identity:get_project(project_id=2c5612d0798e42de8f668235f7279e04)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:47.051892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c512aa-ee72-4afe-9ef8-8fc1cec4bc72 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:47.053492 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 577/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:46 2022] GET /identity/v3/projects/2c5612d0798e42de8f668235f7279e04 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:47.059801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:47.121587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:47.161581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed982d1-da61-4119-bf0c-ae0be861cf5a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:47.194342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbedddb-942a-47e3-82d8-067ab71d9eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:47.194931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbedddb-942a-47e3-82d8-067ab71d9eaf None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:47.195210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbedddb-942a-47e3-82d8-067ab71d9eaf None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:47.195481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbedddb-942a-47e3-82d8-067ab71d9eaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:47.196190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbedddb-942a-47e3-82d8-067ab71d9eaf None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=2c5612d0798e42de8f668235f7279e04, user.id=a178468f906d4f3f928fb9db3453e188) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:47.196498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbedddb-942a-47e3-82d8-067ab71d9eaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:47.197928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbedddb-942a-47e3-82d8-067ab71d9eaf None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:47.203075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 569/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:47 2022] GET /identity/v3/role_assignments?user.id=a178468f906d4f3f928fb9db3453e188&scope.project.id=2c5612d0798e42de8f668235f7279e04&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:49.588663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4e9a-7481-452c-b9c1-67f62c47b648 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:49.589218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4e9a-7481-452c-b9c1-67f62c47b648 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:49.589218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4e9a-7481-452c-b9c1-67f62c47b648 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:49.590829 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 578/1147] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:49.596246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c1ae-1c01-459c-baef-dc38a06c1884 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:49.596648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c1ae-1c01-459c-baef-dc38a06c1884 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:49.596830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c1ae-1c01-459c-baef-dc38a06c1884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:49.614664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1d01c1ae-1c01-459c-baef-dc38a06c1884 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:49.615992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d01c1ae-1c01-459c-baef-dc38a06c1884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:49.681106 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 570/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:50.009308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4e9a-7481-452c-b9c1-67f62c47b648 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:50.068165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:50.109983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4e9a-7481-452c-b9c1-67f62c47b648 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:50.143026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:50.143634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:50.143911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:50.144177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:50.153652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:50.158668 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:50.159597 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:50.160484 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 579/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:50.165779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c1ae-1c01-459c-baef-dc38a06c1884 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:50.222669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:50.262556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c1ae-1c01-459c-baef-dc38a06c1884 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:50.294880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:50.295467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:50.295746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:50.296017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:50.296669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:50.296971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:50.298474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:50.306161 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 571/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:50.311901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:50.409716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4d57c-fabe-4a75-b550-89d3f5376730 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:50.426248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:50.443184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:50.443798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:50.444077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:50.444350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] PATH_INFO: `/v3/users/a178468f906d4f3f928fb9db3453e188` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:50.445448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] RBAC: Authorizing `identity:get_user(user_id=a178468f906d4f3f928fb9db3453e188)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:50.446906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:50.448457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 580/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:50 2022] GET /identity/v3/users/a178468f906d4f3f928fb9db3453e188 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:50.454218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:50.551344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e447da3-2c66-4531-8f62-96f47ffb3c53 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:50.569910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:50.584393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310e269-498f-4284-b630-0b02f4720124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:50.585008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e269-498f-4284-b630-0b02f4720124 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:50.585286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e269-498f-4284-b630-0b02f4720124 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:50.585560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310e269-498f-4284-b630-0b02f4720124 None admin] PATH_INFO: `/v3/projects/2c5612d0798e42de8f668235f7279e04` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:50.586708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310e269-498f-4284-b630-0b02f4720124 None admin] RBAC: Authorizing `identity:get_project(project_id=2c5612d0798e42de8f668235f7279e04)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:50.588111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310e269-498f-4284-b630-0b02f4720124 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:50.589977 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 572/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:50 2022] GET /identity/v3/projects/2c5612d0798e42de8f668235f7279e04 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:50.596681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:50.692336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e4d6b-196e-44c0-8039-dca55c725a27 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:50.726625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56119b-2a90-420d-b57a-cd62d7f3bc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:50.727255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56119b-2a90-420d-b57a-cd62d7f3bc1c None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:50.727531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56119b-2a90-420d-b57a-cd62d7f3bc1c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:50.727798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56119b-2a90-420d-b57a-cd62d7f3bc1c None admin] PATH_INFO: `/v3/projects/2c5612d0798e42de8f668235f7279e04/users/a178468f906d4f3f928fb9db3453e188/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:50.729656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56119b-2a90-420d-b57a-cd62d7f3bc1c None admin] RBAC: Authorizing `identity:create_grant(project_id=2c5612d0798e42de8f668235f7279e04, user_id=a178468f906d4f3f928fb9db3453e188, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:50.731099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56119b-2a90-420d-b57a-cd62d7f3bc1c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:50.744089 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 581/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:33:50 2022] PUT /identity/v3/projects/2c5612d0798e42de8f668235f7279e04/users/a178468f906d4f3f928fb9db3453e188/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:33:53.165370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f3bdd-4b86-4511-a195-480266a7dc05 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:53.165940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f3bdd-4b86-4511-a195-480266a7dc05 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:53.165940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f3bdd-4b86-4511-a195-480266a7dc05 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:53.167235 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 573/1154] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:53.172645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73633-76b7-4903-85b7-71d81fe77364 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:53.173023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73633-76b7-4903-85b7-71d81fe77364 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:53.173267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e73633-76b7-4903-85b7-71d81fe77364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:53.191770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-11e73633-76b7-4903-85b7-71d81fe77364 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:53.192950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-11e73633-76b7-4903-85b7-71d81fe77364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:53.233947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:53.258875 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 582/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:53.592445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f3bdd-4b86-4511-a195-480266a7dc05 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:53.692225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f3bdd-4b86-4511-a195-480266a7dc05 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:53.724519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:53.725135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:53.725417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:53.725698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:53.735453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:33:53.741067 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:53.742178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:33:53.743360 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 574/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:53.748278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73633-76b7-4903-85b7-71d81fe77364 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:53.847254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e73633-76b7-4903-85b7-71d81fe77364 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:53.880078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:53.880674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:53.880957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:53.881229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:53.881906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:53.882209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:53.883572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:53.891316 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 583/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:53.897331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:53.945760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:53.995865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c55e1-ce24-43ff-87cd-6e3accddd7f1 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:54.028602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:54.029219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:54.029501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:54.029778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] PATH_INFO: `/v3/users/a178468f906d4f3f928fb9db3453e188` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:54.030920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] RBAC: Authorizing `identity:get_user(user_id=a178468f906d4f3f928fb9db3453e188)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:54.032346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:54.034454 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 575/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:33:53 2022] GET /identity/v3/users/a178468f906d4f3f928fb9db3453e188 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:54.040312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:54.080086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:54.140073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5113cd0-1964-468b-b1d7-ef6cb3aed63c None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:54.172860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:54.174112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:54.174397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:54.174661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:54.239861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:54.241297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:54.277873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:54.336682 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 584/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:54.342951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:54.439007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2603f-81f4-4a64-9bb0-df3c92584709 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:54.455639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:54.474297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733649a-8c39-4738-85a8-57473ea368b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:54.474921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733649a-8c39-4738-85a8-57473ea368b6 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:54.475205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733649a-8c39-4738-85a8-57473ea368b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:54.475481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733649a-8c39-4738-85a8-57473ea368b6 None admin] PATH_INFO: `/v3/projects/2c5612d0798e42de8f668235f7279e04` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:54.476588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733649a-8c39-4738-85a8-57473ea368b6 None admin] RBAC: Authorizing `identity:get_project(project_id=2c5612d0798e42de8f668235f7279e04)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:54.478057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733649a-8c39-4738-85a8-57473ea368b6 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:54.479660 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 576/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:33:54 2022] GET /identity/v3/projects/2c5612d0798e42de8f668235f7279e04 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:54.486518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:54.583046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f64fb-9927-4cb3-9974-95a6db54a2a8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:54.616759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772fb94-5c78-4984-a447-bb8ab6701174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:54.617353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772fb94-5c78-4984-a447-bb8ab6701174 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:54.617637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772fb94-5c78-4984-a447-bb8ab6701174 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:54.617943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772fb94-5c78-4984-a447-bb8ab6701174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:54.618661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772fb94-5c78-4984-a447-bb8ab6701174 None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=2c5612d0798e42de8f668235f7279e04, user.id=a178468f906d4f3f928fb9db3453e188) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:54.618968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772fb94-5c78-4984-a447-bb8ab6701174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:54.620383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772fb94-5c78-4984-a447-bb8ab6701174 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:54.626029 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 585/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:33:54 2022] GET /identity/v3/role_assignments?user.id=a178468f906d4f3f928fb9db3453e188&scope.project.id=2c5612d0798e42de8f668235f7279e04&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:33:57.066685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4799f5f-a907-4d85-9186-4170e89f9b2e None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:57.067192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4799f5f-a907-4d85-9186-4170e89f9b2e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:57.067301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4799f5f-a907-4d85-9186-4170e89f9b2e None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:57.068537 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 577/1162] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:33:57.073606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8f88-256d-4631-bf04-5240212e9c4d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:57.074001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8f88-256d-4631-bf04-5240212e9c4d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:57.074180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8f88-256d-4631-bf04-5240212e9c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:57.080599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:57.093234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-26fb8f88-256d-4631-bf04-5240212e9c4d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:33:57.094685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-26fb8f88-256d-4631-bf04-5240212e9c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:33:57.160315 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 586/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:33:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:33:57.486766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4799f5f-a907-4d85-9186-4170e89f9b2e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:57.586248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4799f5f-a907-4d85-9186-4170e89f9b2e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:57.619951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:57.620542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:57.620820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:57.621097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:57.669744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:57.685693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:57.687156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:57.780610 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 578/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:33:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:33:57.787060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb8f88-256d-4631-bf04-5240212e9c4d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:57.888311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb8f88-256d-4631-bf04-5240212e9c4d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:57.917136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:57.922246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f824f0e-418c-4a69-ae66-12c1cefe6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:57.922873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f0e-418c-4a69-ae66-12c1cefe6e71 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:57.923734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f0e-418c-4a69-ae66-12c1cefe6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:57.923970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f0e-418c-4a69-ae66-12c1cefe6e71 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:57.930630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824f0e-418c-4a69-ae66-12c1cefe6e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:57.932062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824f0e-418c-4a69-ae66-12c1cefe6e71 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-2f824f0e-418c-4a69-ae66-12c1cefe6e71 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:33:57.943363 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:33:57.944557 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:33:57.945628 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 587/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:33:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:33:57.950527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:33:58.033329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:33:58.049606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb207de-a001-44a2-b342-bac56513bb68 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:33:58.082219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4006c-4d68-4769-95bf-c99ad3a12bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:33:58.082812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4006c-4d68-4769-95bf-c99ad3a12bf4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:33:58.083088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4006c-4d68-4769-95bf-c99ad3a12bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:33:58.083357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4006c-4d68-4769-95bf-c99ad3a12bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:33:58.083975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4006c-4d68-4769-95bf-c99ad3a12bf4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:33:58.084274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4006c-4d68-4769-95bf-c99ad3a12bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:33:58.085620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4006c-4d68-4769-95bf-c99ad3a12bf4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:33:58.092726 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 579/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Sat Sep 10 20:33:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:00.490327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7682c-967b-456c-98e1-7394dfaf099e None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:00.490953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7682c-967b-456c-98e1-7394dfaf099e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:00.490953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7682c-967b-456c-98e1-7394dfaf099e None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:00.492196 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 588/1167] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:34:00.497481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40bb9c-9355-49fd-bc78-f0b5a10d7f59 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:00.497897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40bb9c-9355-49fd-bc78-f0b5a10d7f59 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:00.498073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40bb9c-9355-49fd-bc78-f0b5a10d7f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:00.516549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fe40bb9c-9355-49fd-bc78-f0b5a10d7f59 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:34:00.517995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fe40bb9c-9355-49fd-bc78-f0b5a10d7f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:34:00.582634 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 580/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:34:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:34:00.916251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7682c-967b-456c-98e1-7394dfaf099e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:01.017452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7682c-967b-456c-98e1-7394dfaf099e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:01.050407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23022aa-3a6e-4f7e-9add-fa158cf9c828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:01.050996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23022aa-3a6e-4f7e-9add-fa158cf9c828 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:01.051276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23022aa-3a6e-4f7e-9add-fa158cf9c828 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:01.051547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23022aa-3a6e-4f7e-9add-fa158cf9c828 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:01.052213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23022aa-3a6e-4f7e-9add-fa158cf9c828 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:01.053564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23022aa-3a6e-4f7e-9add-fa158cf9c828 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:01.069218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 589/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Sat Sep 10 20:34:00 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:34:03.523498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf45e8-f554-45c3-b679-7cedefa66cb7 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:03.524072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf45e8-f554-45c3-b679-7cedefa66cb7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:03.524072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf45e8-f554-45c3-b679-7cedefa66cb7 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:03.525360 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 581/1170] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:34:03.530699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0454f8c-2581-475b-8f79-8f048ba21c63 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:03.531116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0454f8c-2581-475b-8f79-8f048ba21c63 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:03.531261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0454f8c-2581-475b-8f79-8f048ba21c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:03.549660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a0454f8c-2581-475b-8f79-8f048ba21c63 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:34:03.550877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a0454f8c-2581-475b-8f79-8f048ba21c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:34:03.596552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:03.617677 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 590/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:34:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:34:03.944854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf45e8-f554-45c3-b679-7cedefa66cb7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:04.023115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:04.044736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf45e8-f554-45c3-b679-7cedefa66cb7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:04.077766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf67e40-1bc6-45f7-9ed0-83451fac3d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:04.078412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf67e40-1bc6-45f7-9ed0-83451fac3d3a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:04.078696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf67e40-1bc6-45f7-9ed0-83451fac3d3a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:04.078971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf67e40-1bc6-45f7-9ed0-83451fac3d3a None admin] PATH_INFO: `/v3/domains/a8bb2f117fd94eb0b9432a30de8fc43b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:04.089158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf67e40-1bc6-45f7-9ed0-83451fac3d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8bb2f117fd94eb0b9432a30de8fc43b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:04.090908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf67e40-1bc6-45f7-9ed0-83451fac3d3a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:04.092440 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 582/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Sat Sep 10 20:34:03 2022] GET /identity/v3/domains/a8bb2f117fd94eb0b9432a30de8fc43b => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:04.098573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0454f8c-2581-475b-8f79-8f048ba21c63 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:04.195224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0454f8c-2581-475b-8f79-8f048ba21c63 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:04.227794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83a1af-53a3-4e9a-abc7-fe4e49f4c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:04.228391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83a1af-53a3-4e9a-abc7-fe4e49f4c154 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:04.228675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83a1af-53a3-4e9a-abc7-fe4e49f4c154 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:04.228952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83a1af-53a3-4e9a-abc7-fe4e49f4c154 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:04.229601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83a1af-53a3-4e9a-abc7-fe4e49f4c154 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:04.231061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83a1af-53a3-4e9a-abc7-fe4e49f4c154 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:04.258124 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 591/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:34:04 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:34:06.686260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd40c4-8a79-487d-96c0-316ed6872faf None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:06.686818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd40c4-8a79-487d-96c0-316ed6872faf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:06.686818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd40c4-8a79-487d-96c0-316ed6872faf None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:06.688104 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 583/1174] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:34:06.693272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0896b2c-fa99-48a7-b20a-4aa987ec891e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:06.693665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0896b2c-fa99-48a7-b20a-4aa987ec891e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:06.693858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0896b2c-fa99-48a7-b20a-4aa987ec891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:06.711817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a0896b2c-fa99-48a7-b20a-4aa987ec891e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:34:06.713008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a0896b2c-fa99-48a7-b20a-4aa987ec891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:34:06.740174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:06.779446 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 592/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:34:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:34:07.106654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd40c4-8a79-487d-96c0-316ed6872faf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:07.205402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd40c4-8a79-487d-96c0-316ed6872faf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:07.239357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaabc05-5b2c-4414-bc3c-ceb6ae75c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:07.239902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaabc05-5b2c-4414-bc3c-ceb6ae75c700 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:07.240179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaabc05-5b2c-4414-bc3c-ceb6ae75c700 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:07.240452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaabc05-5b2c-4414-bc3c-ceb6ae75c700 None admin] PATH_INFO: `/v3/domains/a8bb2f117fd94eb0b9432a30de8fc43b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:07.241539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaabc05-5b2c-4414-bc3c-ceb6ae75c700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a8bb2f117fd94eb0b9432a30de8fc43b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:07.242986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaabc05-5b2c-4414-bc3c-ceb6ae75c700 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:07.244509 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 584/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Sat Sep 10 20:34:07 2022] GET /identity/v3/domains/a8bb2f117fd94eb0b9432a30de8fc43b => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:07.250323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0896b2c-fa99-48a7-b20a-4aa987ec891e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:07.347631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0896b2c-fa99-48a7-b20a-4aa987ec891e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:07.380867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903f30c-8297-4096-a031-0bbc1160b2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:07.381463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903f30c-8297-4096-a031-0bbc1160b2f5 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:07.381745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903f30c-8297-4096-a031-0bbc1160b2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:07.382050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903f30c-8297-4096-a031-0bbc1160b2f5 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:07.382689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903f30c-8297-4096-a031-0bbc1160b2f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:07.384113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903f30c-8297-4096-a031-0bbc1160b2f5 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:07.404515 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 593/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:34:07 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:34:09.858227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800095f9-6d56-43c7-aba2-2623949b87bb None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:09.858762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800095f9-6d56-43c7-aba2-2623949b87bb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:09.858762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800095f9-6d56-43c7-aba2-2623949b87bb None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:09.860068 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 585/1178] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:34:09.866019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b1a16-43b3-4e5d-9eb3-02aa33ea66cc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:09.866449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b1a16-43b3-4e5d-9eb3-02aa33ea66cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:09.866647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b1a16-43b3-4e5d-9eb3-02aa33ea66cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:09.887228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-317b1a16-43b3-4e5d-9eb3-02aa33ea66cc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:34:09.888508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-317b1a16-43b3-4e5d-9eb3-02aa33ea66cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:34:09.902970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:09.956022 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 594/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:34:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:34:10.282073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800095f9-6d56-43c7-aba2-2623949b87bb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:10.291677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:10.382407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800095f9-6d56-43c7-aba2-2623949b87bb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:10.414764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:10.415375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:10.415654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:10.415924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:10.425726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:34:10.430509 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:34:10.431684 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:34:10.432639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 586/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:34:10.437617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b1a16-43b3-4e5d-9eb3-02aa33ea66cc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:10.535843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b1a16-43b3-4e5d-9eb3-02aa33ea66cc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:10.568095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:10.568683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:10.568960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:10.569227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:10.569902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:34:10.570201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:10.571546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:10.579722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 595/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:10.585701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:10.649727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:10.683638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fb169-8dc1-4c6f-be17-92ad376b2194 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:10.717219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:10.717857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:10.718138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:10.718413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] PATH_INFO: `/v3/users/788f2118a56d4b4bb86ca686d25f4c56` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:10.731172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] RBAC: Authorizing `identity:get_user(user_id=788f2118a56d4b4bb86ca686d25f4c56)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:10.732597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:10.734173 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 587/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:34:10 2022] GET /identity/v3/users/788f2118a56d4b4bb86ca686d25f4c56 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:10.740492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:10.755925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:10.837948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b047a14-d6ff-4309-b09f-a6a2efb3c427 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:10.870938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:10.871524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:10.871793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:10.872060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:10.936823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:10.938301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:10.951743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:11.034426 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 596/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:34:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:34:11.040479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:11.137073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44f21e-3be0-4526-8823-5847b2d0ca6c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:11.159879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:11.170558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdfd29-5415-4b7d-9147-a618bf752548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:11.171177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdfd29-5415-4b7d-9147-a618bf752548 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:11.171461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdfd29-5415-4b7d-9147-a618bf752548 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:11.171738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdfd29-5415-4b7d-9147-a618bf752548 None admin] PATH_INFO: `/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:11.173390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdfd29-5415-4b7d-9147-a618bf752548 None admin] RBAC: Authorizing `identity:get_project(project_id=49ced64ea0fc4e0080ff9400ca2278fa)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:11.174863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdfd29-5415-4b7d-9147-a618bf752548 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:11.176462 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 588/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:34:11 2022] GET /identity/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:11.182557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:11.277020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5f9d1-d80c-4367-81c9-a9fdacaa5606 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:11.289909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:11.310777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a1ae-80dd-409d-b80f-963600b6657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:11.311364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a1ae-80dd-409d-b80f-963600b6657d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:11.311638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a1ae-80dd-409d-b80f-963600b6657d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:11.311908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a1ae-80dd-409d-b80f-963600b6657d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:11.312613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a1ae-80dd-409d-b80f-963600b6657d None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=49ced64ea0fc4e0080ff9400ca2278fa, user.id=788f2118a56d4b4bb86ca686d25f4c56) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:34:11.312917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a1ae-80dd-409d-b80f-963600b6657d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:11.314604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a1ae-80dd-409d-b80f-963600b6657d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:11.319751 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 597/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:34:11 2022] GET /identity/v3/role_assignments?user.id=788f2118a56d4b4bb86ca686d25f4c56&scope.project.id=49ced64ea0fc4e0080ff9400ca2278fa&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:13.700150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08aa9-2346-437a-9d23-5e8b119dba91 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:13.700705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08aa9-2346-437a-9d23-5e8b119dba91 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:13.700705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08aa9-2346-437a-9d23-5e8b119dba91 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:13.702048 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 589/1186] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:34:13.707863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea98d85-68d9-425f-9829-01e66676e53c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:13.708259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea98d85-68d9-425f-9829-01e66676e53c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:13.708414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea98d85-68d9-425f-9829-01e66676e53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:13.726556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1ea98d85-68d9-425f-9829-01e66676e53c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:34:13.727730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ea98d85-68d9-425f-9829-01e66676e53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:34:13.792392 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 598/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:34:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:34:14.128527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a08aa9-2346-437a-9d23-5e8b119dba91 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:14.228140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a08aa9-2346-437a-9d23-5e8b119dba91 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:14.261001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:14.261613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:14.261926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:14.262207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:14.272642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:34:14.277412 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:34:14.278560 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:34:14.279454 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 590/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:34:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:34:14.284650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea98d85-68d9-425f-9829-01e66676e53c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:14.382132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea98d85-68d9-425f-9829-01e66676e53c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:14.389882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:14.416931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:14.417528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:14.417809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:14.418118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:14.418775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:34:14.419079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:14.420444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:14.428404 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 599/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:14.434190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:14.487965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:14.533672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533bb2c-d54c-459f-9a5a-bc75466d1fb8 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:14.567363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:14.567982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:14.568265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:14.568537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] PATH_INFO: `/v3/users/788f2118a56d4b4bb86ca686d25f4c56` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:14.569645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] RBAC: Authorizing `identity:get_user(user_id=788f2118a56d4b4bb86ca686d25f4c56)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:14.571119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:14.572666 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 591/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:34:14 2022] GET /identity/v3/users/788f2118a56d4b4bb86ca686d25f4c56 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:14.578503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:14.675571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc82733-0f0b-4b81-bbcd-b6592fe4e7ba None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:14.708271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7748fe-4590-4d9a-87d1-6761b54c2f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:14.708881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7748fe-4590-4d9a-87d1-6761b54c2f0d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:14.709143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7748fe-4590-4d9a-87d1-6761b54c2f0d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:14.709418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7748fe-4590-4d9a-87d1-6761b54c2f0d None admin] PATH_INFO: `/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:14.710532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7748fe-4590-4d9a-87d1-6761b54c2f0d None admin] RBAC: Authorizing `identity:get_project(project_id=49ced64ea0fc4e0080ff9400ca2278fa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:14.711951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7748fe-4590-4d9a-87d1-6761b54c2f0d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:14.713547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 600/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:34:14 2022] GET /identity/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:14.720122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:14.817765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8992b2-9e3c-45be-8e57-64cbc97d8adc None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:14.836605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:14.851540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053f47d-cf00-4de8-a1b9-c9c9ca24558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:14.852174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053f47d-cf00-4de8-a1b9-c9c9ca24558b None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:14.852450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053f47d-cf00-4de8-a1b9-c9c9ca24558b None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:14.852721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053f47d-cf00-4de8-a1b9-c9c9ca24558b None admin] PATH_INFO: `/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa/users/788f2118a56d4b4bb86ca686d25f4c56/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:14.854592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053f47d-cf00-4de8-a1b9-c9c9ca24558b None admin] RBAC: Authorizing `identity:create_grant(project_id=49ced64ea0fc4e0080ff9400ca2278fa, user_id=788f2118a56d4b4bb86ca686d25f4c56, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:14.856010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053f47d-cf00-4de8-a1b9-c9c9ca24558b None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:14.868481 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 592/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:34:14 2022] PUT /identity/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa/users/788f2118a56d4b4bb86ca686d25f4c56/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:34:17.235722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19daa3-fe27-4fbd-bbae-05de4fb8a124 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:17.236222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19daa3-fe27-4fbd-bbae-05de4fb8a124 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:17.236222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19daa3-fe27-4fbd-bbae-05de4fb8a124 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:17.237563 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 601/1193] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:34:17.242897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7a5c0-f486-4944-ad8b-b609a64d6912 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:17.243295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7a5c0-f486-4944-ad8b-b609a64d6912 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:17.243450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7a5c0-f486-4944-ad8b-b609a64d6912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:17.261723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-52a7a5c0-f486-4944-ad8b-b609a64d6912 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:34:17.262932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-52a7a5c0-f486-4944-ad8b-b609a64d6912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:34:17.326465 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 593/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:34:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:34:17.650766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19daa3-fe27-4fbd-bbae-05de4fb8a124 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:17.686892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:17.751688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19daa3-fe27-4fbd-bbae-05de4fb8a124 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:17.784493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:17.785096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:17.785398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:17.785670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:17.795658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:34:17.800720 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:34:17.801811 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:34:17.802717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 602/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:34:17.807809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7a5c0-f486-4944-ad8b-b609a64d6912 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:17.904055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7a5c0-f486-4944-ad8b-b609a64d6912 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:17.914674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:17.944839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:17.945438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:17.945720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:17.946024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:17.946680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:34:17.946983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:17.948337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:17.956577 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 594/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:34:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:17.962733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:18.056381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:18.062827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acf359-dd4b-433c-af86-d87b44c976b4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:18.095922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:18.096532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:18.096812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:18.097082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] PATH_INFO: `/v3/users/788f2118a56d4b4bb86ca686d25f4c56` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:18.098198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] RBAC: Authorizing `identity:get_user(user_id=788f2118a56d4b4bb86ca686d25f4c56)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:18.099608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:18.101150 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 603/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Sat Sep 10 20:34:17 2022] GET /identity/v3/users/788f2118a56d4b4bb86ca686d25f4c56 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:18.106920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:18.204642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6d79f-9be0-4ab2-8d6e-857b2ec011b0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:18.237098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:18.237679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:18.237991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:18.238267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:18.277053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:18.303638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:18.305055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:18.399697 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 595/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:34:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:34:18.406083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:18.504911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62371b63-2430-48ef-a8ed-b3d053b65468 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:18.537029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530814a-e7e5-4fd6-a2c8-3ed92691d687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:18.537643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530814a-e7e5-4fd6-a2c8-3ed92691d687 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:18.537979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530814a-e7e5-4fd6-a2c8-3ed92691d687 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:18.538258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530814a-e7e5-4fd6-a2c8-3ed92691d687 None admin] PATH_INFO: `/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:18.539372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530814a-e7e5-4fd6-a2c8-3ed92691d687 None admin] RBAC: Authorizing `identity:get_project(project_id=49ced64ea0fc4e0080ff9400ca2278fa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:18.540784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530814a-e7e5-4fd6-a2c8-3ed92691d687 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:18.542410 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 604/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Sat Sep 10 20:34:18 2022] GET /identity/v3/projects/49ced64ea0fc4e0080ff9400ca2278fa => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:18.549270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:34:18.617231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:18.645118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d0dc8-8ae3-4351-aada-b43a0a01a4d9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:34:18.678618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d5bc3-ef73-4f48-bfef-b1e5d9c9e15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:34:18.679202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d5bc3-ef73-4f48-bfef-b1e5d9c9e15a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:34:18.679481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d5bc3-ef73-4f48-bfef-b1e5d9c9e15a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:34:18.679751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d5bc3-ef73-4f48-bfef-b1e5d9c9e15a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:34:18.680454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d5bc3-ef73-4f48-bfef-b1e5d9c9e15a None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=49ced64ea0fc4e0080ff9400ca2278fa, user.id=788f2118a56d4b4bb86ca686d25f4c56) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:34:18.680755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d5bc3-ef73-4f48-bfef-b1e5d9c9e15a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:34:18.682189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d5bc3-ef73-4f48-bfef-b1e5d9c9e15a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:34:18.687652 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 596/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:34:18 2022] GET /identity/v3/role_assignments?user.id=788f2118a56d4b4bb86ca686d25f4c56&scope.project.id=49ced64ea0fc4e0080ff9400ca2278fa&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:34:28.532982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:34:28.685132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=32 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:22.695221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b569b-e9a3-4ded-b49e-e4a9d955f8e6 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:22.695846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b569b-e9a3-4ded-b49e-e4a9d955f8e6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:22.695846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b569b-e9a3-4ded-b49e-e4a9d955f8e6 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:22.697371 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 605/1201] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:22.704867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f7841-36d9-43aa-8e05-5cc8bcc69784 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:22.705436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f7841-36d9-43aa-8e05-5cc8bcc69784 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:22.705538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f7841-36d9-43aa-8e05-5cc8bcc69784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:22.732448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-931f7841-36d9-43aa-8e05-5cc8bcc69784 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:22.734798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-931f7841-36d9-43aa-8e05-5cc8bcc69784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:22.829859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 597/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:35:23.242578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b569b-e9a3-4ded-b49e-e4a9d955f8e6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:23.410192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b569b-e9a3-4ded-b49e-e4a9d955f8e6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:23.467236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:23.467236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:23.467236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:23.467236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:23.484459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:23.487156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:23.506782 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:23.507943 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:23.508961 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:23.508961 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:23.508961 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:23.512762 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 606/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:23.515790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f7841-36d9-43aa-8e05-5cc8bcc69784 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:23.666782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f7841-36d9-43aa-8e05-5cc8bcc69784 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:23.710078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:23.718670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76541ae-b64e-455d-a468-8bc76f2e3a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:23.719492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76541ae-b64e-455d-a468-8bc76f2e3a27 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:23.719780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76541ae-b64e-455d-a468-8bc76f2e3a27 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:23.721684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76541ae-b64e-455d-a468-8bc76f2e3a27 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:23.721684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76541ae-b64e-455d-a468-8bc76f2e3a27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:23.721684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76541ae-b64e-455d-a468-8bc76f2e3a27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:23.723444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76541ae-b64e-455d-a468-8bc76f2e3a27 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:23.736302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 598/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:23.744056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:23.822388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:23.901979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d545b4a-9aab-4d90-a0b1-78d94987c6ad None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:23.947436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c46c18-1265-44b2-90c7-2877f72f594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:23.948624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46c18-1265-44b2-90c7-2877f72f594c None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:23.948904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46c18-1265-44b2-90c7-2877f72f594c None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:23.949171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46c18-1265-44b2-90c7-2877f72f594c None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:23.949996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c46c18-1265-44b2-90c7-2877f72f594c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:23.952651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c46c18-1265-44b2-90c7-2877f72f594c None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:23.984851 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 607/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Sat Sep 10 20:35:23 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:35:26.527628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49942e3-afbd-48a5-a3bd-df8155bf31bb None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:26.528183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49942e3-afbd-48a5-a3bd-df8155bf31bb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:26.528183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49942e3-afbd-48a5-a3bd-df8155bf31bb None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:26.529528 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 599/1206] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:26.536251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4445-25d1-4f94-bf71-5c9e9d850914 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:26.536751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4445-25d1-4f94-bf71-5c9e9d850914 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:26.536858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a4445-25d1-4f94-bf71-5c9e9d850914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:26.557095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-626a4445-25d1-4f94-bf71-5c9e9d850914 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:26.558760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-626a4445-25d1-4f94-bf71-5c9e9d850914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:26.646201 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 608/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 10 20:35:26.991793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49942e3-afbd-48a5-a3bd-df8155bf31bb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:27.105849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49942e3-afbd-48a5-a3bd-df8155bf31bb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:27.144627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:27.145258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:27.145542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:27.145818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:27.159602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:35:27.164506 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:27.165812 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:27.167028 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 600/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:35:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:27.172519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a4445-25d1-4f94-bf71-5c9e9d850914 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:27.217512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:27.290242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a4445-25d1-4f94-bf71-5c9e9d850914 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:27.326180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:27.327713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:27.328213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:27.328736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:27.329611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:27.330166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:27.331748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:27.341216 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 609/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:35:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:27.348809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:27.404143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:27.473864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858913ed-2f47-4981-9721-3685ec8f0fcb None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:27.516220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:27.516888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:27.517170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:27.517463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:27.529039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:27.532256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:27.549516 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:27.550785 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:27.551726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:27.551726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:27.551726 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:27.551726 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 601/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:27.559361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:27.690355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:27.693429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9034c1e-2ea7-4c56-a414-f719dcf738a2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:27.735901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:27.736524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:27.736783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:27.737084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:27.737691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:27.738018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:27.739436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:27.753136 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 610/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:27.759006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:27.881906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe1d40-f62d-4eaf-b815-7781cea237fe None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:27.883879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:27.924756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:27.925486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:27.925767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:27.926081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:27.935191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:27.937267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:35:27.952811 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:27.954159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:27.955394 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 602/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:27.957284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:28.085548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fdb0c-dc8f-47f6-b966-61f48bcc86c4 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:28.124093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:28.124710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:28.124997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:28.125365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:28.126042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:28.126350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:28.127769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:28.147223 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 611/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Sat Sep 10 20:35:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:28.152917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:28.279142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e571210-d32c-40f8-92f8-4c6e07eae666 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:28.318574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:28.319224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:28.319560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:28.319999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:28.355712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:28.403332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:28.404845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:28.522569 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 603/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:35:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:35:28.528266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:28.552601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:28.651519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cd363-d794-4cee-a36f-8a79657155c7 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:28.694711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:28.695281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:28.695567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:28.695846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:28.706411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:28.707861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:28.724119 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:28.725305 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:28.726457 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:28.726457 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:28.726457 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:28.726457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 612/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:28.731986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:28.801358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:28.846946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089667fa-3106-4356-8fc4-6fc3e49bf52a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:28.887540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:28.888168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:28.888517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:28.888814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:28.889649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:28.889996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:28.891412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:28.905563 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 604/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:35:28.911690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:29.000741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:29.034395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1480f7-07a2-4871-a9f5-7bac3a31bc24 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:29.068724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:29.069354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:29.069635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:29.069938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:29.077310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:29.078954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:35:29.091398 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:29.092908 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:29.093854 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 613/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:29.101409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:29.219370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac6c12-9bb0-469a-aa36-e98ce1084df3 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:29.246261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:29.262263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c1e16-f1cc-4481-8e7a-408397ff2a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:29.262910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1e16-f1cc-4481-8e7a-408397ff2a2e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:29.263304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1e16-f1cc-4481-8e7a-408397ff2a2e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:29.263583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c1e16-f1cc-4481-8e7a-408397ff2a2e None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:29.265572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c1e16-f1cc-4481-8e7a-408397ff2a2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:29.266126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c1e16-f1cc-4481-8e7a-408397ff2a2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:29.267810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c1e16-f1cc-4481-8e7a-408397ff2a2e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:29.282956 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 605/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:35:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:29.289664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:29.415349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11596ef-157c-457f-b559-bc9b0232e7b1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:29.448261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:29.455859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d371df8-d3de-4fba-9e54-e95d2ff37476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:29.456549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371df8-d3de-4fba-9e54-e95d2ff37476 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:29.456857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371df8-d3de-4fba-9e54-e95d2ff37476 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:29.457974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371df8-d3de-4fba-9e54-e95d2ff37476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:29.458754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d371df8-d3de-4fba-9e54-e95d2ff37476 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=b7ea3c7b376b40cfa12b65cd62046345) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:29.459121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d371df8-d3de-4fba-9e54-e95d2ff37476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:29.460461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d371df8-d3de-4fba-9e54-e95d2ff37476 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:29.466627 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 614/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:35:29 2022] GET /identity/v3/role_assignments?user.id=b7ea3c7b376b40cfa12b65cd62046345&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:31.917667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba71d3-9498-4c25-a7e7-6e2a5d4cdcb6 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:31.918252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba71d3-9498-4c25-a7e7-6e2a5d4cdcb6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:31.918252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba71d3-9498-4c25-a7e7-6e2a5d4cdcb6 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:31.919985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 606/1220] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:31.925740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab95b2-9cec-405c-be5c-92df74d3178f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:31.926211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab95b2-9cec-405c-be5c-92df74d3178f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:31.926323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab95b2-9cec-405c-be5c-92df74d3178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:31.945949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-06ab95b2-9cec-405c-be5c-92df74d3178f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:31.947141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-06ab95b2-9cec-405c-be5c-92df74d3178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:32.032178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 615/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:35:32.390459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba71d3-9498-4c25-a7e7-6e2a5d4cdcb6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:32.510872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba71d3-9498-4c25-a7e7-6e2a5d4cdcb6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:32.546333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:32.546954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:32.547236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:32.547510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:32.557583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:35:32.563095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:32.564192 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:32.565518 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 607/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:32.570480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab95b2-9cec-405c-be5c-92df74d3178f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:32.680681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab95b2-9cec-405c-be5c-92df74d3178f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:32.707166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:32.720461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:32.721069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:32.721350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:32.721622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:32.722313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] RBAC: Adding query filter params (name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:32.722613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:32.723981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:32.734165 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 616/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:32.740387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:32.789866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:32.849646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dffa4e-01c0-4cb4-8a39-4ef8c7cc8442 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:32.891173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:32.891872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:32.892265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:32.892637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:32.905427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:32.907044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:32.924772 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:32.926531 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:32.927663 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:32.927663 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:32.927663 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:32.930805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 608/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:32.931728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:33.046006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50116700-bd7a-41bb-9b0b-92206af9a152 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:33.085130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:33.085859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:33.086152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:33.086430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:33.087068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:33.087372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:33.088845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:33.101843 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 617/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:33.108523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:33.211180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:33.218627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7e73d-3f21-49e7-a6b2-3ec6e581939a None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:33.257482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:33.258228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:33.258513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:33.258800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:33.267336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:33.268785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:35:33.280039 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:33.286064 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:33.287252 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 609/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:33.288006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:33.337756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:33.396072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86a6a-664a-4881-b578-dd8b8a0f1e7a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:33.432722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:33.433323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:33.433621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:33.433944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:33.434650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:33.434972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:33.436630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:33.455660 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 618/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Sat Sep 10 20:35:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:33.461381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:33.568141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff4d39-fac1-4ab9-a87a-924283c21a29 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:33.604304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:33.604937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:33.605213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:33.605481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:33.617376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:33.618942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:33.634188 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:33.635747 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:33.636897 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:33.636897 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:33.636897 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:33.636897 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 610/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:35:33.641747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:33.761636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3975-d993-459f-b544-70a654129b0b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:33.764999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:33.801841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:33.802529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:33.802824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:33.803137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:33.803849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:33.804155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:33.805554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:33.818548 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 619/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:33.825680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:33.836659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:33.933631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854f1e1-2b1e-4faf-9f38-cd98eff67532 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:33.970970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:33.971621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:33.971893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:33.972170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:33.979418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:33.980869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:35:33.992905 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:33.993862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:33.994791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 611/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:35:34.000936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:34.105095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156f447-4cae-48df-8886-95ded9dc637b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:34.142304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cbc54-b069-4c0e-bdcf-c8be0a8a33de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:34.142972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cbc54-b069-4c0e-bdcf-c8be0a8a33de None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:34.143255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cbc54-b069-4c0e-bdcf-c8be0a8a33de None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:34.143530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cbc54-b069-4c0e-bdcf-c8be0a8a33de None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:34.144249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cbc54-b069-4c0e-bdcf-c8be0a8a33de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:34.144555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cbc54-b069-4c0e-bdcf-c8be0a8a33de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:34.146008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cbc54-b069-4c0e-bdcf-c8be0a8a33de None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:34.159251 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 620/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:34.166011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:34.236481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:34.272396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56127e5-9b7f-4433-bf29-162b47fc25c9 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:34.309111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51ea39-1a1e-4165-9a66-4bcf54e266e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:34.309772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51ea39-1a1e-4165-9a66-4bcf54e266e3 None admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:34.310085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51ea39-1a1e-4165-9a66-4bcf54e266e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:34.310361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51ea39-1a1e-4165-9a66-4bcf54e266e3 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/b7ea3c7b376b40cfa12b65cd62046345/roles/34d4065b512e49b883776816da214951` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:34.326351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51ea39-1a1e-4165-9a66-4bcf54e266e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=b7ea3c7b376b40cfa12b65cd62046345, role_id=34d4065b512e49b883776816da214951)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:34.327796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51ea39-1a1e-4165-9a66-4bcf54e266e3 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:34.341634 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 612/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:35:34 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/b7ea3c7b376b40cfa12b65cd62046345/roles/34d4065b512e49b883776816da214951 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:35:36.823916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c42ad-0e26-4e81-bd8a-c059af19d813 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:36.824481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c42ad-0e26-4e81-bd8a-c059af19d813 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:36.824481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c42ad-0e26-4e81-bd8a-c059af19d813 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:36.825779 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 621/1233] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:35:36.831292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4743-8f3f-45d4-aad8-77aad99b716d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:36.831765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4743-8f3f-45d4-aad8-77aad99b716d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:36.831904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4743-8f3f-45d4-aad8-77aad99b716d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:36.851194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cadf4743-8f3f-45d4-aad8-77aad99b716d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:36.852413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cadf4743-8f3f-45d4-aad8-77aad99b716d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:36.923912 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 613/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:35:37.253693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c42ad-0e26-4e81-bd8a-c059af19d813 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:37.285378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:37.377546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c42ad-0e26-4e81-bd8a-c059af19d813 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:37.416199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:37.416831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:37.417114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:37.417418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:37.429608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:35:37.435263 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:37.436889 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:37.437766 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 622/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Sat Sep 10 20:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:35:37.444458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4743-8f3f-45d4-aad8-77aad99b716d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:37.497928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:37.564522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4743-8f3f-45d4-aad8-77aad99b716d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:37.604276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:37.604954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:37.605245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:37.605517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:37.606272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] RBAC: Adding query filter params (name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:37.606629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:37.608963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:37.617759 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 614/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Sat Sep 10 20:35:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:37.625289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:37.731191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:37.749816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55491536-f92c-441b-8fd9-cfaa5570244a None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:37.790989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:37.791766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:37.792083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:37.792514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:37.804954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:37.806444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:37.822477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:37.823503 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:37.824870 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:37.824870 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:37.824870 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:37.824870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 623/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:37.829721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:37.943391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c5f9-cedd-43a7-9205-4cc1200a301e None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:37.953552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:37.983006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:37.983713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:37.983996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:37.984281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:37.985122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:37.985447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:37.986990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:38.004782 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 615/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:38.006302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:38.126153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1bbc7-03d3-4595-a571-a038112b81c0 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:38.164490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:38.165127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:38.165409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:38.165681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:38.168674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:38.174623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:38.176111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:35:38.188247 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:38.189277 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:38.190350 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 624/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:38.195295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:38.304716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0f7d8-f778-4def-abb7-a361a89ccb29 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:38.344257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:38.344919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:38.345201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:38.345471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:38.346257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:38.346546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:38.347978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:38.368630 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 616/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Sat Sep 10 20:35:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:38.375569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:38.492186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056ff44-e654-4e65-b4e1-55dd82af862b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:38.528270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:38.528873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:38.529155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:38.529462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:38.607347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:38.616263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:38.617751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:38.742171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 625/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:35:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:35:38.748773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:38.782049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:38.864719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0e9cb-e82f-4c86-b63d-fa3c4b9f9102 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:38.902966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:38.903694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:38.904005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:38.904306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:38.915192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:38.916648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:38.935354 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:38.937176 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:38.938139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:38.938139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:38.938139 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:38.938139 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 617/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 20:35:38.944118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:39.062470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:39.070333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a18f4f-992a-45e7-846e-c5b71bbf5cc1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:39.114903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:39.115810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:39.116301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:39.116726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:39.117685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:39.118260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:39.120045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:39.135270 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 626/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:35:39.141130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:39.233162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:39.257919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd1c4-3495-4eef-b9d9-67194b203890 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:39.295980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:39.296793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:39.296983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:39.297295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:39.304603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:39.306100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:35:39.317884 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:39.319074 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:39.320287 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:39.324256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 618/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:39.325070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:39.443581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b027c0-779d-411c-af88-1959aa168370 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:39.484486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb45d89-7a7c-48f8-90d4-f2de1a7efe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:39.485094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb45d89-7a7c-48f8-90d4-f2de1a7efe9a None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:39.485377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb45d89-7a7c-48f8-90d4-f2de1a7efe9a None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:39.485650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb45d89-7a7c-48f8-90d4-f2de1a7efe9a None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:39.486388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb45d89-7a7c-48f8-90d4-f2de1a7efe9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:39.486695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb45d89-7a7c-48f8-90d4-f2de1a7efe9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:39.488622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb45d89-7a7c-48f8-90d4-f2de1a7efe9a None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:39.500822 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 627/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:39.506932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:39.619341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760f79-5d08-47bb-bf5f-3a246266aff4 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:39.663059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a078e3-992f-4db6-9a20-3667f03bf403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:39.663663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a078e3-992f-4db6-9a20-3667f03bf403 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:39.663942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a078e3-992f-4db6-9a20-3667f03bf403 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:39.664271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a078e3-992f-4db6-9a20-3667f03bf403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:39.665139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a078e3-992f-4db6-9a20-3667f03bf403 None admin] RBAC: Adding query filter params (role.id=34d4065b512e49b883776816da214951, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=b7ea3c7b376b40cfa12b65cd62046345) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:39.665429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a078e3-992f-4db6-9a20-3667f03bf403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:39.667087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a078e3-992f-4db6-9a20-3667f03bf403 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:39.678938 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 619/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:35:39 2022] GET /identity/v3/role_assignments?user.id=b7ea3c7b376b40cfa12b65cd62046345&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=34d4065b512e49b883776816da214951 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:42.349109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc706d-dbc3-4656-9b54-16671effc9ef None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:42.349657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc706d-dbc3-4656-9b54-16671effc9ef None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:42.349657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc706d-dbc3-4656-9b54-16671effc9ef None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:42.350998 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 628/1247] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:42.356190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39d728-5fa4-42b7-a614-6a4d23ae9dbd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:42.356603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39d728-5fa4-42b7-a614-6a4d23ae9dbd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:42.356738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39d728-5fa4-42b7-a614-6a4d23ae9dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:42.363018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:42.375542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5e39d728-5fa4-42b7-a614-6a4d23ae9dbd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:42.376723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5e39d728-5fa4-42b7-a614-6a4d23ae9dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:42.446209 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 620/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:35:42.780365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc706d-dbc3-4656-9b54-16671effc9ef None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:42.798117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:42.880007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc706d-dbc3-4656-9b54-16671effc9ef None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:42.913818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:42.914459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:42.914739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:42.915009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:42.924807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:35:42.929592 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:42.930602 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:42.931545 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 629/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:35:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:42.937218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39d728-5fa4-42b7-a614-6a4d23ae9dbd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:43.036122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39d728-5fa4-42b7-a614-6a4d23ae9dbd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:43.065616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:43.071085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:43.071682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:43.071961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:43.072231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:43.072877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:43.073177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:43.074556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:43.082509 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 621/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:35:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:43.088390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:43.163980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:43.187814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e2915-5a39-45e9-9ee8-33faa021071b None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:43.220894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:43.221513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:43.221793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:43.222096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:43.231677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:43.233631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:43.248880 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:43.250040 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:43.251145 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:43.251145 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:43.251145 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:43.251145 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 630/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:43.255757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:43.353442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb57595-6849-4ede-8e55-ae2eb6c425a0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:43.385963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:43.386545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:43.386824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:43.387099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:43.387729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:43.388039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:43.390078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:43.402064 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 622/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:43.407770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:43.505426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c8f44-ae39-4dd7-9a9d-5a2e9308bffe None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:43.531125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:43.539233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:43.539848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:43.540127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:43.540395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:43.548128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:43.549550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:35:43.560545 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:43.561635 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:43.562687 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 631/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:43.567486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:43.622293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:43.670999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e5dee-5073-4f25-a842-71ec70c1ea3d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:43.705116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:43.705704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:43.706013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:43.706282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:43.706997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:43.707297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:43.708696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:43.726317 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 623/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Sat Sep 10 20:35:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:43.732180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:43.829588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f70016-25f0-45ce-83e3-165fc2015008 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:43.863432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:43.864021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:43.864305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:43.864581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:43.917807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:43.930849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:43.932294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:44.029233 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 632/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:35:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:35:44.035096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:44.132983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4ccc7-3438-4741-a063-8c8e029b132c None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:44.146096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:44.166895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:44.167514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:44.167794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:44.168113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:44.177871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:44.179301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:44.193934 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:44.195080 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:44.195990 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:44.195990 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:44.195990 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:44.195990 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 624/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:44.200852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:44.288097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:44.300354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ca402-9c3f-4d14-be57-64966862e8c2 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:44.334050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:44.334702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:44.334976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:44.335243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:44.335866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:44.336163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:44.337529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:44.348899 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 633/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:44.354727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:44.451777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c09c0-aeae-4e0f-ac0d-a4ddf22ed076 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:44.484295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:44.484908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:44.485191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:44.485469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:44.491996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:44.493442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:35:44.504944 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:44.506089 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:44.507167 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 625/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:44.511908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:44.614198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584df3e-949a-431c-8ea7-fbaaddb86b2f None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:44.651283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa449a7-bd90-4a43-a058-9533aaa5d257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:44.651744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa449a7-bd90-4a43-a058-9533aaa5d257 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:44.652049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa449a7-bd90-4a43-a058-9533aaa5d257 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:44.652386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa449a7-bd90-4a43-a058-9533aaa5d257 None admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:44.653166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa449a7-bd90-4a43-a058-9533aaa5d257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:44.653438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa449a7-bd90-4a43-a058-9533aaa5d257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:44.655404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa449a7-bd90-4a43-a058-9533aaa5d257 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:44.669144 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 634/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:44.676005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:44.701754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:44.774351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a3a9c-a0f7-40fd-889f-3ee8beffaeaa None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:44.806907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e901696-8329-43b6-a43e-8edb179ca2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:44.807500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901696-8329-43b6-a43e-8edb179ca2dd None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:44.807783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901696-8329-43b6-a43e-8edb179ca2dd None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:44.808060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901696-8329-43b6-a43e-8edb179ca2dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:44.808777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901696-8329-43b6-a43e-8edb179ca2dd None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=b7ea3c7b376b40cfa12b65cd62046345) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:44.809092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901696-8329-43b6-a43e-8edb179ca2dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:44.811165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901696-8329-43b6-a43e-8edb179ca2dd None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:44.816316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 626/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:35:44 2022] GET /identity/v3/role_assignments?user.id=b7ea3c7b376b40cfa12b65cd62046345&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:47.181370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095c8d0-82d0-4f04-8ea6-14ec5291f962 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:47.182031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095c8d0-82d0-4f04-8ea6-14ec5291f962 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:47.182031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095c8d0-82d0-4f04-8ea6-14ec5291f962 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:47.183282 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 635/1261] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:47.188888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c2908-dc7e-472d-af2c-1ce55ed13687 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:47.189321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c2908-dc7e-472d-af2c-1ce55ed13687 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:47.189426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c2908-dc7e-472d-af2c-1ce55ed13687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:47.209013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b82c2908-dc7e-472d-af2c-1ce55ed13687 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:47.210216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b82c2908-dc7e-472d-af2c-1ce55ed13687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:47.275465 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 627/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:35:47.601790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095c8d0-82d0-4f04-8ea6-14ec5291f962 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:47.612712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:47.708485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095c8d0-82d0-4f04-8ea6-14ec5291f962 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:47.742210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:47.742822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:47.743127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:47.743398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:47.754029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:35:47.758779 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:47.759885 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:47.760765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 636/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:35:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:47.766619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c2908-dc7e-472d-af2c-1ce55ed13687 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:47.784875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:47.862746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c2908-dc7e-472d-af2c-1ce55ed13687 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:47.895694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:47.896285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:47.896564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:47.896836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:47.897491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:47.897798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:47.899176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:47.907199 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 628/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:35:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:47.913027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:47.978927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:48.012815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed29b4-13fa-4051-bf7b-6ac7646243bf None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:48.046414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:48.047030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:48.047309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:48.047580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:48.057191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:48.058672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:48.074303 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:48.075289 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:48.077227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:48.077227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:48.077227 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:48.077227 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 637/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:48.082009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:48.160966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:48.179649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd14d-aa09-48ce-a190-fa4180b8758d None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:48.213596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:48.214244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:48.214481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:48.214753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:48.215635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:48.215932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:48.217589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:48.228736 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 629/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:48.235250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:48.332227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe95352-3574-443f-aeba-c4f7090ceeda None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:48.351101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:48.365706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:48.366360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:48.366645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:48.366924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:48.374905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:48.376338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:35:48.387226 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:48.393295 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:48.394451 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 638/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:48.395274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:48.499280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32cc48-e262-4547-bfea-3af0b4541c4f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:48.532111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:48.532701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:48.532982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:48.533255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:48.533998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:48.534305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:48.535720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:48.543966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:48.552958 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 630/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Sat Sep 10 20:35:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:48.558658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:48.661221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e357b-2338-4a80-a75f-0f196b8f8ab1 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:48.694278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:48.694897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:48.695177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:48.695452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:48.706144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:48.707595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:48.721566 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:48.722711 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:48.723795 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:48.723795 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:48.723795 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:48.723795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 639/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:48.728446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:48.824526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9b4fb-6330-4543-bd53-b8b719aa2e13 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:48.858736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:48.859330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:48.859612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:48.859887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:48.860519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:48.860823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:48.862206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:48.873136 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 631/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:48.878661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:48.909785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:48.978089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fbc66-4bf7-4909-a629-6b95e4129b19 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:49.011667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:49.012306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:49.012584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:49.012854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:49.019195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:49.020613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:35:49.031873 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:49.032781 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:49.033729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 640/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:49.038787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:49.097331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:49.138740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed953d-1884-402a-aabf-518613c71d07 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:49.171374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a3585-380c-4218-a506-cbd655bdc469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:49.171966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a3585-380c-4218-a506-cbd655bdc469 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:49.172244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a3585-380c-4218-a506-cbd655bdc469 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:49.172515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a3585-380c-4218-a506-cbd655bdc469 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:49.173185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a3585-380c-4218-a506-cbd655bdc469 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:49.173485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a3585-380c-4218-a506-cbd655bdc469 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:49.174948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a3585-380c-4218-a506-cbd655bdc469 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:49.187125 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 632/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:49.193070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:49.281718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:49.290839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc112dc1-b13b-438f-b961-7ae6e045b3b8 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:49.323870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03063b-6c0e-4b47-8586-e20ec30d3999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:49.324512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03063b-6c0e-4b47-8586-e20ec30d3999 None admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:49.324793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03063b-6c0e-4b47-8586-e20ec30d3999 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:49.325067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03063b-6c0e-4b47-8586-e20ec30d3999 None admin] PATH_INFO: `/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/b7ea3c7b376b40cfa12b65cd62046345/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:49.327059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03063b-6c0e-4b47-8586-e20ec30d3999 None admin] RBAC: Authorizing `identity:create_grant(project_id=a01cb3a7943240f6984664abc52ce14e, user_id=b7ea3c7b376b40cfa12b65cd62046345, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:49.328479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03063b-6c0e-4b47-8586-e20ec30d3999 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:49.343151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 641/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Sat Sep 10 20:35:49 2022] PUT /identity/v3/projects/a01cb3a7943240f6984664abc52ce14e/users/b7ea3c7b376b40cfa12b65cd62046345/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:35:51.699918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2a2a0-5513-4a53-ac50-aaa5048c686c None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:51.700450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2a2a0-5513-4a53-ac50-aaa5048c686c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:51.700450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2a2a0-5513-4a53-ac50-aaa5048c686c None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:51.701799 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 633/1274] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:51.707065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa06431-b0c7-4e93-ba9b-f0645fe0013c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:51.707494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa06431-b0c7-4e93-ba9b-f0645fe0013c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:51.707626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa06431-b0c7-4e93-ba9b-f0645fe0013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:51.726453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9fa06431-b0c7-4e93-ba9b-f0645fe0013c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:51.727635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9fa06431-b0c7-4e93-ba9b-f0645fe0013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:51.792521 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 642/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:35:52.116678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2a2a0-5513-4a53-ac50-aaa5048c686c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:52.215557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2a2a0-5513-4a53-ac50-aaa5048c686c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:52.232257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:52.249021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:52.249620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:52.249924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:52.250198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:52.259807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 20:35:52.264786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:52.265916 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:52.266782 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 634/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Sat Sep 10 20:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:52.271758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa06431-b0c7-4e93-ba9b-f0645fe0013c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:52.350276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:52.369959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa06431-b0c7-4e93-ba9b-f0645fe0013c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:52.402865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:52.403465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:52.404278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:52.404551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:52.405208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:52.405510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:52.406922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:52.414676 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 643/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Sat Sep 10 20:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:52.421015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:52.522960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1901a-0fa5-4d74-9f19-bb227c2a9c36 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:52.555907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:52.556521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:52.556801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:52.557073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:52.566914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:52.568341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:52.582417 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:52.583545 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:52.584666 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:52.584666 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:52.584666 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:52.584666 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 635/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:52.589232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:52.698212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe604f7-fe75-46b4-a9de-cca6eb6fbb7d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:52.735313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:52.735953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:52.736178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:52.736450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:52.737077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:52.737376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:52.738763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:52.750288 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 644/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:52.755968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:52.787127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:52.854472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3efb-6934-4a39-bff4-707169a689e7 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:52.886531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:52.887140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:52.887421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:52.887694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:52.895324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:52.896758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 20:35:52.908333 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:52.909575 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:52.910857 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 636/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:52.915128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:52.918836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:53.012919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f23927-e440-4879-971a-1d0029cd0410 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:53.046237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:53.046770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:53.047052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:53.047325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:53.048037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:53.048340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:53.049746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:53.067149 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 645/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Sat Sep 10 20:35:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:53.072801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:53.162436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:53.172868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d9ba-ee10-4bee-9127-6c0e93a7aa34 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:53.205926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:53.206524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:53.206805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:53.207076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:53.272391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:53.273850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:53.359794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:53.370326 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 637/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Sat Sep 10 20:35:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 20:35:53.376145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:53.435780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:53.474894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427bd7d-b7ec-4fdd-832c-db4cc23d2513 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:53.507858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:53.508479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:53.508761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:53.509035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:53.519202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:53.520666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 20:35:53.535455 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:53.536571 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 20:35:53.537518 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 20:35:53.537518 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 20:35:53.537518 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:53.537518 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 646/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:53.542227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:53.639768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840da64-3da9-46d9-8369-40be4940b3ea None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:53.674347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:53.675371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:53.675880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:53.676358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:53.677470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:53.678037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:53.680575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:53.691830 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 638/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Sat Sep 10 20:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:53.697457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:53.796299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf3ae-d83b-423c-9984-e6d680a1f0ac None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:53.809981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:53.830595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:53.831219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:53.831499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:53.831769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:53.838225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:53.839667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 20:35:53.851542 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:53.852581 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 20:35:53.853746 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 647/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Sat Sep 10 20:35:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:53.858487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:53.907641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:53.957200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787aed2-9a07-4ba9-93b2-c1f5ceed0669 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:53.989936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9f45b-703d-4e38-80de-aa9843d8d934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:53.990539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9f45b-703d-4e38-80de-aa9843d8d934 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:53.990823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9f45b-703d-4e38-80de-aa9843d8d934 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:53.991101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9f45b-703d-4e38-80de-aa9843d8d934 None admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:53.991783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9f45b-703d-4e38-80de-aa9843d8d934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:53.992092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9f45b-703d-4e38-80de-aa9843d8d934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:53.993509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9f45b-703d-4e38-80de-aa9843d8d934 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:54.005738 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 639/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Sat Sep 10 20:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:54.011695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:54.111485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c13f2-d499-4881-922e-4aa892bcd709 None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:54.144055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca88f3-3e6e-4443-bc0d-f34be804780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:54.144645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca88f3-3e6e-4443-bc0d-f34be804780b None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:54.144926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca88f3-3e6e-4443-bc0d-f34be804780b None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:54.145196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca88f3-3e6e-4443-bc0d-f34be804780b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:54.145938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca88f3-3e6e-4443-bc0d-f34be804780b None admin] RBAC: Adding query filter params (role.id=d819f0a222ec4024ac762c673f345cf5, scope.project.id=a01cb3a7943240f6984664abc52ce14e, user.id=b7ea3c7b376b40cfa12b65cd62046345) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:54.146246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca88f3-3e6e-4443-bc0d-f34be804780b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:54.147683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca88f3-3e6e-4443-bc0d-f34be804780b None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:54.153234 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 648/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Sat Sep 10 20:35:54 2022] GET /identity/v3/role_assignments?user.id=b7ea3c7b376b40cfa12b65cd62046345&scope.project.id=a01cb3a7943240f6984664abc52ce14e&role.id=d819f0a222ec4024ac762c673f345cf5 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:56.550384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84359c0b-3cf0-4d51-920a-487497483fa8 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:56.550943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84359c0b-3cf0-4d51-920a-487497483fa8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:56.550943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84359c0b-3cf0-4d51-920a-487497483fa8 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:56.552281 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 640/1288] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:56.557379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01a502-8194-4497-82f5-821c0f723557 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:56.557795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01a502-8194-4497-82f5-821c0f723557 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:56.557960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01a502-8194-4497-82f5-821c0f723557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:56.576248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4b01a502-8194-4497-82f5-821c0f723557 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:56.577415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4b01a502-8194-4497-82f5-821c0f723557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:56.605896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:56.644263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 649/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:35:56.969475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84359c0b-3cf0-4d51-920a-487497483fa8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:57.069918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84359c0b-3cf0-4d51-920a-487497483fa8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:57.080153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:57.103629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:57.104227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:57.104512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:57.104787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:57.105361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:57.107321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:35:57.116037 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:35:57.117366 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:35:57.118296 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 641/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:35:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:35:57.123307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01a502-8194-4497-82f5-821c0f723557 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:57.219923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01a502-8194-4497-82f5-821c0f723557 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:57.252056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08b757-515a-4982-8a34-7e5156db63b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:57.252677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08b757-515a-4982-8a34-7e5156db63b3 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:57.252956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08b757-515a-4982-8a34-7e5156db63b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:57.253226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08b757-515a-4982-8a34-7e5156db63b3 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:57.253864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08b757-515a-4982-8a34-7e5156db63b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:57.254194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08b757-515a-4982-8a34-7e5156db63b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:57.255550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08b757-515a-4982-8a34-7e5156db63b3 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:57.260220 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 650/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Sat Sep 10 20:35:57 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:57.266738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:35:57.364816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b46145-ca34-4ceb-a46b-c0a7e3fd0f16 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:35:57.397400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fad1f-d9c3-4380-935b-a3fcfa24e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:35:57.398021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fad1f-d9c3-4380-935b-a3fcfa24e718 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:57.398304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fad1f-d9c3-4380-935b-a3fcfa24e718 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:57.398579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fad1f-d9c3-4380-935b-a3fcfa24e718 None admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:57.399198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fad1f-d9c3-4380-935b-a3fcfa24e718 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:35:57.399499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fad1f-d9c3-4380-935b-a3fcfa24e718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:35:57.400847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fad1f-d9c3-4380-935b-a3fcfa24e718 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:35:57.405850 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 642/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Sat Sep 10 20:35:57 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:35:59.774167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0374e-5704-4fb0-ae32-8d50116fbc27 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:59.774733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0374e-5704-4fb0-ae32-8d50116fbc27 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:59.774733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0374e-5704-4fb0-ae32-8d50116fbc27 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:59.776031 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 651/1293] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:35:59.781296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7084a-03ee-4d5d-aec8-5243e5db5fc1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:35:59.781715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7084a-03ee-4d5d-aec8-5243e5db5fc1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:35:59.781860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7084a-03ee-4d5d-aec8-5243e5db5fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:35:59.800144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-28d7084a-03ee-4d5d-aec8-5243e5db5fc1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:35:59.801310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-28d7084a-03ee-4d5d-aec8-5243e5db5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:35:59.831272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:35:59.868799 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 643/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:35:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:36:00.194704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0374e-5704-4fb0-ae32-8d50116fbc27 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:00.226138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:36:00.294499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0374e-5704-4fb0-ae32-8d50116fbc27 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:00.327113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128570-7c9b-4c23-9207-a2cc20af63d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:00.327693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128570-7c9b-4c23-9207-a2cc20af63d9 None admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:00.327979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128570-7c9b-4c23-9207-a2cc20af63d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:00.328252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128570-7c9b-4c23-9207-a2cc20af63d9 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:00.328888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128570-7c9b-4c23-9207-a2cc20af63d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:00.330274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128570-7c9b-4c23-9207-a2cc20af63d9 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:00.345173 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 652/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Sat Sep 10 20:36:00 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:36:02.741188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5faf-7f1e-4d35-bb36-e306e014d683 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:02.741718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5faf-7f1e-4d35-bb36-e306e014d683 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:02.741718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5faf-7f1e-4d35-bb36-e306e014d683 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:02.743098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 644/1296] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:36:02.748867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3c395-069b-4ecb-afdb-3788ebfcc679 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:02.749276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3c395-069b-4ecb-afdb-3788ebfcc679 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:02.749383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3c395-069b-4ecb-afdb-3788ebfcc679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:02.767154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c8e3c395-069b-4ecb-afdb-3788ebfcc679 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:36:02.768330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8e3c395-069b-4ecb-afdb-3788ebfcc679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:36:02.839073 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 653/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:36:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:36:03.165168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d5faf-7f1e-4d35-bb36-e306e014d683 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:03.267311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d5faf-7f1e-4d35-bb36-e306e014d683 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:03.299512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:03.300125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:03.300403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:03.300672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] PATH_INFO: `/v3/services/placement` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:03.301231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:03.302600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:36:03.311095 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:36:03.312334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:36:03.313458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 645/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:36:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:36:03.318377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3c395-069b-4ecb-afdb-3788ebfcc679 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:03.336636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:36:03.417751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3c395-069b-4ecb-afdb-3788ebfcc679 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:03.452639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:03.453316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:03.453598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:03.453893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:03.454513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] RBAC: Adding query filter params (name=placement) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:36:03.454810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:03.456159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:03.461074 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 654/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Sat Sep 10 20:36:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:36:03.467769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:03.498742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:36:03.565076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800582e-c8c2-453f-8c58-f3d179f2623f None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:03.598025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e27a63-5a48-40f2-bc7e-8c85a9ec8d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:03.598620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e27a63-5a48-40f2-bc7e-8c85a9ec8d12 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:03.598903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e27a63-5a48-40f2-bc7e-8c85a9ec8d12 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:03.599177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e27a63-5a48-40f2-bc7e-8c85a9ec8d12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:03.599865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e27a63-5a48-40f2-bc7e-8c85a9ec8d12 None admin] RBAC: Adding query filter params (interface=public, service_id=1e3df29936a54b899ca92527f890fdfc, region_id=RegionOne) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:36:03.600173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e27a63-5a48-40f2-bc7e-8c85a9ec8d12 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:03.601522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e27a63-5a48-40f2-bc7e-8c85a9ec8d12 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:03.606331 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 646/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Sat Sep 10 20:36:03 2022] GET /identity/v3/endpoints?service_id=1e3df29936a54b899ca92527f890fdfc&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:36:03.612012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:03.691652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:36:03.709427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd97be7-e17f-46ab-ae0e-0d5cde70ac9d None admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:03.743619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824074f0-6132-48d0-ace6-162c616d6c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:03.744208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824074f0-6132-48d0-ace6-162c616d6c50 None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:03.744487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824074f0-6132-48d0-ace6-162c616d6c50 None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:03.744759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824074f0-6132-48d0-ace6-162c616d6c50 None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:03.745354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824074f0-6132-48d0-ace6-162c616d6c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:03.746735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824074f0-6132-48d0-ace6-162c616d6c50 None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:03.753601 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 655/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Sat Sep 10 20:36:03 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:36:06.141304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520fa7-3a22-41dd-be6f-0cb28338a8a0 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:06.141869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520fa7-3a22-41dd-be6f-0cb28338a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:06.141869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85520fa7-3a22-41dd-be6f-0cb28338a8a0 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:06.143184 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 647/1302] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:36:06.148381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74cb86-904b-4a81-8adc-b48d82e6433e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:06.148786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74cb86-904b-4a81-8adc-b48d82e6433e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:06.148922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74cb86-904b-4a81-8adc-b48d82e6433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:06.167232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ef74cb86-904b-4a81-8adc-b48d82e6433e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:36:06.168391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ef74cb86-904b-4a81-8adc-b48d82e6433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:36:06.234360 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 656/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:36:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 20:36:06.565306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520fa7-3a22-41dd-be6f-0cb28338a8a0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:06.658572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:36:06.665571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85520fa7-3a22-41dd-be6f-0cb28338a8a0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:06.698251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:06.698862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:06.699144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:06.699417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:06.699987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:06.701488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 20:36:06.709952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 20:36:06.711270 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 20:36:06.712419 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 648/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Sat Sep 10 20:36:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 20:36:06.717925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74cb86-904b-4a81-8adc-b48d82e6433e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:06.791461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:36:06.814565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74cb86-904b-4a81-8adc-b48d82e6433e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:06.847639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabd45c-c5ce-466a-99a9-dc19c4b6bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:06.848177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabd45c-c5ce-466a-99a9-dc19c4b6bcee None admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:06.848459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabd45c-c5ce-466a-99a9-dc19c4b6bcee None admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:06.848734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabd45c-c5ce-466a-99a9-dc19c4b6bcee None admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:06.849356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabd45c-c5ce-466a-99a9-dc19c4b6bcee None admin] RBAC: Adding query filter params (name=placement) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:36:06.849658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabd45c-c5ce-466a-99a9-dc19c4b6bcee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:06.851058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabd45c-c5ce-466a-99a9-dc19c4b6bcee None admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:06.855983 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 657/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Sat Sep 10 20:36:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:36:06.862108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:36:06.959781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a9e63-ffc0-45b9-b8b0-35cc6db231f0 None admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:36:06.993244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c415b-0e50-4334-9f4a-9bac8496608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:36:06.993868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c415b-0e50-4334-9f4a-9bac8496608a None admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:36:06.994158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c415b-0e50-4334-9f4a-9bac8496608a None admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:36:06.994433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c415b-0e50-4334-9f4a-9bac8496608a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:36:06.995084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c415b-0e50-4334-9f4a-9bac8496608a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:36:06.996439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c415b-0e50-4334-9f4a-9bac8496608a None admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:36:07.016109 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 649/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:36:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:36:16.853073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=30 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:36:17.011883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:20.016660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19b31-5b99-484b-86e3-da99348a04fb None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:20.017199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19b31-5b99-484b-86e3-da99348a04fb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:20.017199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19b31-5b99-484b-86e3-da99348a04fb None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:20.024372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911de5a0-75c8-4953-be56-0b2ebd13400b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:20.024853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911de5a0-75c8-4953-be56-0b2ebd13400b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:20.024943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911de5a0-75c8-4953-be56-0b2ebd13400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:20.027981 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 658/1307] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:20.113489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-911de5a0-75c8-4953-be56-0b2ebd13400b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:20.114733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-911de5a0-75c8-4953-be56-0b2ebd13400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:20.232571 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 650/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:20.586883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a8d74-b69c-4b5f-bbb8-b52982f46bbb None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:20.587164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a8d74-b69c-4b5f-bbb8-b52982f46bbb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:20.587665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a8d74-b69c-4b5f-bbb8-b52982f46bbb None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:20.589479 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 659/1309] 10.222.0.33 () {54 vars in 965 bytes} [Sat Sep 10 20:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:20.595853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358c7e1-da8b-4502-87da-638210f89226 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:20.596215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358c7e1-da8b-4502-87da-638210f89226 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:20.596586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358c7e1-da8b-4502-87da-638210f89226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:20.683081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7358c7e1-da8b-4502-87da-638210f89226 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:20.684431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7358c7e1-da8b-4502-87da-638210f89226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:20.705930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:20.768448 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 651/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Sat Sep 10 20:37:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:20.777730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a8d74-b69c-4b5f-bbb8-b52982f46bbb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:20.779514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-864a8d74-b69c-4b5f-bbb8-b52982f46bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:20.896040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a8d74-b69c-4b5f-bbb8-b52982f46bbb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:20.991830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a92bb-e3b3-487b-b60d-d85b1ed7e1de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:20.992443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a92bb-e3b3-487b-b60d-d85b1ed7e1de service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:20.992726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a92bb-e3b3-487b-b60d-d85b1ed7e1de service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:20.993000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a92bb-e3b3-487b-b60d-d85b1ed7e1de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:20.994887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb2a92bb-e3b3-487b-b60d-d85b1ed7e1de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:21.053195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:21.123201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a92bb-e3b3-487b-b60d-d85b1ed7e1de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:21.125014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a92bb-e3b3-487b-b60d-d85b1ed7e1de service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:21.149385 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 660/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:23.900788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882788d6-ae30-40a4-8125-e3f09ad574eb None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:23.901328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882788d6-ae30-40a4-8125-e3f09ad574eb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:23.901654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882788d6-ae30-40a4-8125-e3f09ad574eb None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:23.903348 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 652/1312] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:23.908978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aa7ce-1897-4685-89a3-fb2032ca70c8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:23.909518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aa7ce-1897-4685-89a3-fb2032ca70c8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:23.909518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aa7ce-1897-4685-89a3-fb2032ca70c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:23.989007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-497aa7ce-1897-4685-89a3-fb2032ca70c8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:23.990265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-497aa7ce-1897-4685-89a3-fb2032ca70c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:24.101868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 661/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:24.445279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882788d6-ae30-40a4-8125-e3f09ad574eb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:24.446933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-882788d6-ae30-40a4-8125-e3f09ad574eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:24.554807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882788d6-ae30-40a4-8125-e3f09ad574eb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:24.635407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:24.637494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82ee0d-5436-4bd0-8a87-d608908da71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:24.638456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ee0d-5436-4bd0-8a87-d608908da71b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:24.639142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ee0d-5436-4bd0-8a87-d608908da71b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:24.639667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ee0d-5436-4bd0-8a87-d608908da71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:24.641854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ee82ee0d-5436-4bd0-8a87-d608908da71b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:24.756398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82ee0d-5436-4bd0-8a87-d608908da71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:24.757851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82ee0d-5436-4bd0-8a87-d608908da71b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:24.777362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 653/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:27.523858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30819-d16b-4489-b6e5-48df641191bd None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:27.524402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30819-d16b-4489-b6e5-48df641191bd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:27.524402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30819-d16b-4489-b6e5-48df641191bd None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:27.525971 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 662/1315] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:27.531378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9418d-e2e2-4cc1-bb00-e9fc471ac009 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:27.531893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9418d-e2e2-4cc1-bb00-e9fc471ac009 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:27.532001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9418d-e2e2-4cc1-bb00-e9fc471ac009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:27.598237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0dc9418d-e2e2-4cc1-bb00-e9fc471ac009 None None] MFA Rules not processed for user `8116bc11609448a7a4b58a423a52d68d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:27.599460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0dc9418d-e2e2-4cc1-bb00-e9fc471ac009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:27.653296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:27.697358 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 654/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:28.052756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217928a-d050-499d-900a-4bcee0a3ec17 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:28.053042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217928a-d050-499d-900a-4bcee0a3ec17 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:28.053316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217928a-d050-499d-900a-4bcee0a3ec17 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:28.054689 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 663/1317] 10.222.0.33 () {54 vars in 965 bytes} [Sat Sep 10 20:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:28.061203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fc907-bbe3-46a5-8a7a-cdce4a964e28 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:28.061492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fc907-bbe3-46a5-8a7a-cdce4a964e28 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:28.061766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fc907-bbe3-46a5-8a7a-cdce4a964e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:28.137909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c19fc907-bbe3-46a5-8a7a-cdce4a964e28 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:28.139044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c19fc907-bbe3-46a5-8a7a-cdce4a964e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:28.214429 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 655/1318] 10.222.0.33 () {58 vars in 1062 bytes} [Sat Sep 10 20:37:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:28.221895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217928a-d050-499d-900a-4bcee0a3ec17 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:28.223439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3217928a-d050-499d-900a-4bcee0a3ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:28.233888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:28.328286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217928a-d050-499d-900a-4bcee0a3ec17 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:28.401224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec21daf-85d0-482d-92cf-7b65192f87f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:28.401751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec21daf-85d0-482d-92cf-7b65192f87f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:28.402058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec21daf-85d0-482d-92cf-7b65192f87f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:28.402327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec21daf-85d0-482d-92cf-7b65192f87f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:28.403886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ec21daf-85d0-482d-92cf-7b65192f87f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:28.465249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:28.511853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec21daf-85d0-482d-92cf-7b65192f87f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:28.513283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec21daf-85d0-482d-92cf-7b65192f87f6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:28.533904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 664/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:31.830829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213faa1-e94c-469c-b5cc-ceb2ba982de5 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:31.831210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213faa1-e94c-469c-b5cc-ceb2ba982de5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:31.831389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213faa1-e94c-469c-b5cc-ceb2ba982de5 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:31.832727 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 656/1320] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:31.838384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a5ade-56b7-45cb-8fa6-7d7255d5b575 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:31.838957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a5ade-56b7-45cb-8fa6-7d7255d5b575 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:31.838957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a5ade-56b7-45cb-8fa6-7d7255d5b575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:31.905699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-773a5ade-56b7-45cb-8fa6-7d7255d5b575 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:31.906940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-773a5ade-56b7-45cb-8fa6-7d7255d5b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:32.002862 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 665/1321] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:32.349362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213faa1-e94c-469c-b5cc-ceb2ba982de5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:32.350975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9213faa1-e94c-469c-b5cc-ceb2ba982de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:32.403071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:32.456447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213faa1-e94c-469c-b5cc-ceb2ba982de5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:32.530526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42773d0a-14fa-4e6b-9286-c860e91c99cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:32.531217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42773d0a-14fa-4e6b-9286-c860e91c99cf service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:32.531403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42773d0a-14fa-4e6b-9286-c860e91c99cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:32.531672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42773d0a-14fa-4e6b-9286-c860e91c99cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:32.533237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-42773d0a-14fa-4e6b-9286-c860e91c99cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:32.628576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:32.642374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42773d0a-14fa-4e6b-9286-c860e91c99cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:32.643796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42773d0a-14fa-4e6b-9286-c860e91c99cf service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:32.661396 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 657/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:35.763111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f31fa-9751-42a4-a5a7-5de5a991ba89 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:35.763778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f31fa-9751-42a4-a5a7-5de5a991ba89 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:35.763778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f31fa-9751-42a4-a5a7-5de5a991ba89 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:35.764995 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 666/1323] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:35.770331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ecfd7-f30a-40ca-8dba-6905bc84c11a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:35.770767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ecfd7-f30a-40ca-8dba-6905bc84c11a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:35.770865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ecfd7-f30a-40ca-8dba-6905bc84c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:35.838015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ab1ecfd7-f30a-40ca-8dba-6905bc84c11a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:35.839234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab1ecfd7-f30a-40ca-8dba-6905bc84c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:35.939985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 658/1324] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:36.283290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f31fa-9751-42a4-a5a7-5de5a991ba89 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:36.284827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b68f31fa-9751-42a4-a5a7-5de5a991ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:36.305347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:36.391475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f31fa-9751-42a4-a5a7-5de5a991ba89 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:36.466960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ce121-2cb8-4844-b13d-70dc0a5e440e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:36.467563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ce121-2cb8-4844-b13d-70dc0a5e440e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:36.467840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ce121-2cb8-4844-b13d-70dc0a5e440e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:36.468112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ce121-2cb8-4844-b13d-70dc0a5e440e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:36.469673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d41ce121-2cb8-4844-b13d-70dc0a5e440e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:36.544430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:36.582792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ce121-2cb8-4844-b13d-70dc0a5e440e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:36.584229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ce121-2cb8-4844-b13d-70dc0a5e440e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:36.604994 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 667/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:39.126738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc205bc4-0f09-43f1-a887-9c2ab2116505 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:39.127297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc205bc4-0f09-43f1-a887-9c2ab2116505 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:39.127297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc205bc4-0f09-43f1-a887-9c2ab2116505 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:39.128754 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 659/1326] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:39.133941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce60ee-cc1f-43fd-b999-634cf5e4d1cd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:39.134533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce60ee-cc1f-43fd-b999-634cf5e4d1cd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:39.134907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce60ee-cc1f-43fd-b999-634cf5e4d1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:39.201941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9fce60ee-cc1f-43fd-b999-634cf5e4d1cd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:39.203128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9fce60ee-cc1f-43fd-b999-634cf5e4d1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:39.299592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 668/1327] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:39.641723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc205bc4-0f09-43f1-a887-9c2ab2116505 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:39.643361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc205bc4-0f09-43f1-a887-9c2ab2116505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:39.705356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:39.749301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc205bc4-0f09-43f1-a887-9c2ab2116505 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:39.822940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b7c7f-c1b8-44b5-87db-da1f70a8a69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:39.823544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7c7f-c1b8-44b5-87db-da1f70a8a69f service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:39.823826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7c7f-c1b8-44b5-87db-da1f70a8a69f service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:39.824099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7c7f-c1b8-44b5-87db-da1f70a8a69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:39.825695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-191b7c7f-c1b8-44b5-87db-da1f70a8a69f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:39.937350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:39.940427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b7c7f-c1b8-44b5-87db-da1f70a8a69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:39.941935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b7c7f-c1b8-44b5-87db-da1f70a8a69f service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:39.967125 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 660/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:42.600083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de390d-4406-4b7e-b491-9207e3c2a9cc None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:42.600698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de390d-4406-4b7e-b491-9207e3c2a9cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:42.600698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de390d-4406-4b7e-b491-9207e3c2a9cc None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:42.601967 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 669/1329] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:42.608878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b0c1-ae46-4934-9618-60ae85ad3560 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:42.609380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b0c1-ae46-4934-9618-60ae85ad3560 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:42.609380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b0c1-ae46-4934-9618-60ae85ad3560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:42.677819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e770b0c1-ae46-4934-9618-60ae85ad3560 None None] MFA Rules not processed for user `8116bc11609448a7a4b58a423a52d68d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:42.679067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e770b0c1-ae46-4934-9618-60ae85ad3560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:42.777437 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 661/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:37:43.124634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de390d-4406-4b7e-b491-9207e3c2a9cc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:43.126238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f0de390d-4406-4b7e-b491-9207e3c2a9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:43.158254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:43.233012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de390d-4406-4b7e-b491-9207e3c2a9cc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:43.312812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9656a-7ff6-419c-91cc-dcba13fd1b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:43.313410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9656a-7ff6-419c-91cc-dcba13fd1b15 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:43.313687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9656a-7ff6-419c-91cc-dcba13fd1b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:43.313990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9656a-7ff6-419c-91cc-dcba13fd1b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:43.315556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4b9656a-7ff6-419c-91cc-dcba13fd1b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:43.397243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:43.435293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9656a-7ff6-419c-91cc-dcba13fd1b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:43.436753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9656a-7ff6-419c-91cc-dcba13fd1b15 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:43.456595 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 670/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:46.107907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161be8a-334c-4649-8c03-07666dd4d6d0 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:46.108408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161be8a-334c-4649-8c03-07666dd4d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:46.108408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161be8a-334c-4649-8c03-07666dd4d6d0 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:46.110016 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 662/1332] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:46.115192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f022165-93ed-4d73-be30-8b6a0f625268 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:46.115973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f022165-93ed-4d73-be30-8b6a0f625268 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:46.116252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f022165-93ed-4d73-be30-8b6a0f625268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:46.191335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0f022165-93ed-4d73-be30-8b6a0f625268 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:46.192526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f022165-93ed-4d73-be30-8b6a0f625268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:46.292144 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 671/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:37:46.633002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161be8a-334c-4649-8c03-07666dd4d6d0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:46.635260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6161be8a-334c-4649-8c03-07666dd4d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:46.708534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:46.745930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161be8a-334c-4649-8c03-07666dd4d6d0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:46.818519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1fe77-423e-49ff-8711-02e1263ccc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:46.819120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1fe77-423e-49ff-8711-02e1263ccc42 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:46.819397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1fe77-423e-49ff-8711-02e1263ccc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:46.819669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1fe77-423e-49ff-8711-02e1263ccc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:46.821288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0bf1fe77-423e-49ff-8711-02e1263ccc42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:46.931379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1fe77-423e-49ff-8711-02e1263ccc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:46.932832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1fe77-423e-49ff-8711-02e1263ccc42 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:46.943010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:46.953134 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 663/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:49.429630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845851a-6579-47d5-99e6-d874c01ecdf0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:49.430189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845851a-6579-47d5-99e6-d874c01ecdf0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:49.430324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845851a-6579-47d5-99e6-d874c01ecdf0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:49.432195 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 672/1335] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:49.437441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b74e92-22f9-4f1a-8721-b63c53f4d66e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:49.437872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b74e92-22f9-4f1a-8721-b63c53f4d66e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:49.438036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b74e92-22f9-4f1a-8721-b63c53f4d66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:49.503783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-46b74e92-22f9-4f1a-8721-b63c53f4d66e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:49.504979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-46b74e92-22f9-4f1a-8721-b63c53f4d66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:49.599592 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 664/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:49.948693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845851a-6579-47d5-99e6-d874c01ecdf0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:49.950263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0845851a-6579-47d5-99e6-d874c01ecdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:49.992064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:50.056178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845851a-6579-47d5-99e6-d874c01ecdf0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:50.129680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982524e-34cd-466e-9c2f-3e37a798500c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:50.130235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982524e-34cd-466e-9c2f-3e37a798500c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:50.130514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982524e-34cd-466e-9c2f-3e37a798500c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:50.130787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982524e-34cd-466e-9c2f-3e37a798500c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:50.132355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7982524e-34cd-466e-9c2f-3e37a798500c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:50.214474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:50.239950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982524e-34cd-466e-9c2f-3e37a798500c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:50.241393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982524e-34cd-466e-9c2f-3e37a798500c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:50.260187 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 673/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:52.987720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbc5d1-25d3-4f1f-843e-86cdde52b9e1 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:52.988239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbc5d1-25d3-4f1f-843e-86cdde52b9e1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:52.988352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbc5d1-25d3-4f1f-843e-86cdde52b9e1 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:52.989582 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 665/1338] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:52.995848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8575-045d-4c92-b1c1-e0c1a9ea9f60 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:52.996406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8575-045d-4c92-b1c1-e0c1a9ea9f60 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:52.996406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8575-045d-4c92-b1c1-e0c1a9ea9f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:53.064793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9dac8575-045d-4c92-b1c1-e0c1a9ea9f60 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:53.066026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9dac8575-045d-4c92-b1c1-e0c1a9ea9f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:53.163135 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 674/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:37:53.501393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbc5d1-25d3-4f1f-843e-86cdde52b9e1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:53.502990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-73dbc5d1-25d3-4f1f-843e-86cdde52b9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:53.579404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:53.607698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbc5d1-25d3-4f1f-843e-86cdde52b9e1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:53.682215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576ed9-8cff-49fc-8343-24a50efdf6cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:53.682809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576ed9-8cff-49fc-8343-24a50efdf6cb service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:53.683085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576ed9-8cff-49fc-8343-24a50efdf6cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:53.683352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576ed9-8cff-49fc-8343-24a50efdf6cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:53.684933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2a576ed9-8cff-49fc-8343-24a50efdf6cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:53.793476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a576ed9-8cff-49fc-8343-24a50efdf6cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:53.794844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a576ed9-8cff-49fc-8343-24a50efdf6cb service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:53.812657 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 666/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:37:58.222450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3754d-6de3-456e-9210-0fe212b11525 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:58.223010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3754d-6de3-456e-9210-0fe212b11525 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:58.223010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3754d-6de3-456e-9210-0fe212b11525 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:58.224330 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 675/1341] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:37:58.230380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0a26-0a9b-46ca-b347-b1f1fc8a1705 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:58.230801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0a26-0a9b-46ca-b347-b1f1fc8a1705 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:58.230990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c0a26-0a9b-46ca-b347-b1f1fc8a1705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:58.236351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:58.298146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-973c0a26-0a9b-46ca-b347-b1f1fc8a1705 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:37:58.299366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-973c0a26-0a9b-46ca-b347-b1f1fc8a1705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:58.397968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 667/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:37:58.748675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3754d-6de3-456e-9210-0fe212b11525 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:37:58.750379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74f3754d-6de3-456e-9210-0fe212b11525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:58.800608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:58.858184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3754d-6de3-456e-9210-0fe212b11525 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:37:58.931855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c422c2-2726-493a-b8d5-147ba8ffc593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:37:58.932450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c422c2-2726-493a-b8d5-147ba8ffc593 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:37:58.932730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c422c2-2726-493a-b8d5-147ba8ffc593 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:37:58.933002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c422c2-2726-493a-b8d5-147ba8ffc593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:37:58.934622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c7c422c2-2726-493a-b8d5-147ba8ffc593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:37:59.033902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:37:59.051846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c422c2-2726-493a-b8d5-147ba8ffc593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:37:59.053569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c422c2-2726-493a-b8d5-147ba8ffc593 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:37:59.073350 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 676/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:02.135883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b581-70e7-4b83-adbe-4a0a41e920e5 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:02.136412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b581-70e7-4b83-adbe-4a0a41e920e5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:02.136412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b581-70e7-4b83-adbe-4a0a41e920e5 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:02.137713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 668/1344] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:02.143058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765c8cb-b436-44f0-9221-05c749ee5877 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:02.143468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765c8cb-b436-44f0-9221-05c749ee5877 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:02.143579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765c8cb-b436-44f0-9221-05c749ee5877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:02.209951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f765c8cb-b436-44f0-9221-05c749ee5877 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:02.211147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f765c8cb-b436-44f0-9221-05c749ee5877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:02.310960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 677/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:02.647404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b581-70e7-4b83-adbe-4a0a41e920e5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:02.649458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ab4b581-70e7-4b83-adbe-4a0a41e920e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:02.746058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:02.764360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b581-70e7-4b83-adbe-4a0a41e920e5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:02.842767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc55d5-0223-4d90-a16a-8e04a648492e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:02.843368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc55d5-0223-4d90-a16a-8e04a648492e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:02.843646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc55d5-0223-4d90-a16a-8e04a648492e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:02.843915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc55d5-0223-4d90-a16a-8e04a648492e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:02.845503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9dfc55d5-0223-4d90-a16a-8e04a648492e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:02.954912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc55d5-0223-4d90-a16a-8e04a648492e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:02.956251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc55d5-0223-4d90-a16a-8e04a648492e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:02.977930 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 669/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:07.409464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4151ca-0959-4657-aab7-1f2877462cbc None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:07.410286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4151ca-0959-4657-aab7-1f2877462cbc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:07.410519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4151ca-0959-4657-aab7-1f2877462cbc None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:07.412217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 678/1347] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:07.419225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba4629-0fd5-4671-a001-d4b6245410a2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:07.419791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba4629-0fd5-4671-a001-d4b6245410a2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:07.419791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba4629-0fd5-4671-a001-d4b6245410a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:07.437051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:07.500172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c6ba4629-0fd5-4671-a001-d4b6245410a2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:07.502178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6ba4629-0fd5-4671-a001-d4b6245410a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:07.625925 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 670/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:07.970945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4151ca-0959-4657-aab7-1f2877462cbc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:07.972501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e4151ca-0959-4657-aab7-1f2877462cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:08.032206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:08.084323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4151ca-0959-4657-aab7-1f2877462cbc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:08.172207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25235c21-6ed4-4660-bd34-6987a6bc9e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:08.172812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25235c21-6ed4-4660-bd34-6987a6bc9e34 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:08.173133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25235c21-6ed4-4660-bd34-6987a6bc9e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:08.173412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25235c21-6ed4-4660-bd34-6987a6bc9e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:08.175282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-25235c21-6ed4-4660-bd34-6987a6bc9e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:08.307309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:08.313481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25235c21-6ed4-4660-bd34-6987a6bc9e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:08.316027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25235c21-6ed4-4660-bd34-6987a6bc9e34 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:08.338424 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 679/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:13.944209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea63208-e7c9-4991-85af-df5225ae1e13 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:13.945506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea63208-e7c9-4991-85af-df5225ae1e13 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:13.946441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea63208-e7c9-4991-85af-df5225ae1e13 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:13.948615 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 671/1350] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:38:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:13.954568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ea549-6570-487e-a323-0a7c0819cea5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:13.955429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ea549-6570-487e-a323-0a7c0819cea5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:13.956024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ea549-6570-487e-a323-0a7c0819cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:14.067233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-413ea549-6570-487e-a323-0a7c0819cea5 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:14.069506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-413ea549-6570-487e-a323-0a7c0819cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:14.241940 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 680/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:38:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:17.623915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=59 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:18.520584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7df0c-1b9b-4f9f-b5e1-b0e7fd86268d None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:18.520941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7df0c-1b9b-4f9f-b5e1-b0e7fd86268d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:18.521159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7df0c-1b9b-4f9f-b5e1-b0e7fd86268d None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:18.522809 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 672/1352] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:18.527992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66309c3-a008-42ed-bf11-fc1006359c10 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:18.528475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66309c3-a008-42ed-bf11-fc1006359c10 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:18.528839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66309c3-a008-42ed-bf11-fc1006359c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:18.590988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:18.616231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c66309c3-a008-42ed-bf11-fc1006359c10 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:18.618015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c66309c3-a008-42ed-bf11-fc1006359c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:18.745044 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 681/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:38:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:20.338490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728779de-9f0b-44d6-ab44-9ab1703bfc8c None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:20.339014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728779de-9f0b-44d6-ab44-9ab1703bfc8c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:20.339126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728779de-9f0b-44d6-ab44-9ab1703bfc8c None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:20.340669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 673/1354] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:20.345807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f57d-1e3d-4f98-8b0b-72c421ff11f8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:20.346312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f57d-1e3d-4f98-8b0b-72c421ff11f8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:20.346561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f57d-1e3d-4f98-8b0b-72c421ff11f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:20.431110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a791f57d-1e3d-4f98-8b0b-72c421ff11f8 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:20.432311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a791f57d-1e3d-4f98-8b0b-72c421ff11f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:20.474042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:20.540005 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 682/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:38:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:38:24.371447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925c51-7ed9-4944-bc23-a74240d38ca1 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:24.372057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925c51-7ed9-4944-bc23-a74240d38ca1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:24.372166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925c51-7ed9-4944-bc23-a74240d38ca1 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:24.373531 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 674/1356] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:24.378871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3e21-42b7-4524-89a5-42026f810cf4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:24.379282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3e21-42b7-4524-89a5-42026f810cf4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:24.379428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3e21-42b7-4524-89a5-42026f810cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:24.449277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c41a3e21-42b7-4524-89a5-42026f810cf4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:24.450519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c41a3e21-42b7-4524-89a5-42026f810cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:24.557419 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 683/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:24.917959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4387d-76d7-451b-973b-06cbf32771a5 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:24.918245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4387d-76d7-451b-973b-06cbf32771a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:24.918717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4387d-76d7-451b-973b-06cbf32771a5 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:24.920512 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 675/1358] 10.222.0.33 () {54 vars in 962 bytes} [Sat Sep 10 20:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:24.928576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563a09f-ad41-4b10-bb8d-5a6fa273647d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:24.929181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563a09f-ad41-4b10-bb8d-5a6fa273647d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:24.930178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563a09f-ad41-4b10-bb8d-5a6fa273647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:24.954846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:25.035079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d563a09f-ad41-4b10-bb8d-5a6fa273647d None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:25.036411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d563a09f-ad41-4b10-bb8d-5a6fa273647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:25.174288 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 684/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Sat Sep 10 20:38:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:38:25.175367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4387d-76d7-451b-973b-06cbf32771a5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:25.176971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-33e4387d-76d7-451b-973b-06cbf32771a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:25.332197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4387d-76d7-451b-973b-06cbf32771a5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:25.407565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:25.449058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f79b51-b5c1-435a-bb8a-1d425d7c58f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:25.449581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f79b51-b5c1-435a-bb8a-1d425d7c58f6 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:25.449948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f79b51-b5c1-435a-bb8a-1d425d7c58f6 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:25.450163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f79b51-b5c1-435a-bb8a-1d425d7c58f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:25.451732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e4f79b51-b5c1-435a-bb8a-1d425d7c58f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:25.545420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f79b51-b5c1-435a-bb8a-1d425d7c58f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:25.547044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f79b51-b5c1-435a-bb8a-1d425d7c58f6 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:25.565545 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 676/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:38:28.231663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2fde1-fd3f-430d-885b-788ce613a5be None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:28.232236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2fde1-fd3f-430d-885b-788ce613a5be None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:28.232236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2fde1-fd3f-430d-885b-788ce613a5be None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:28.233654 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 685/1361] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:28.239955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ef479-0324-41fa-86b2-7141ebcefa24 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:28.240543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ef479-0324-41fa-86b2-7141ebcefa24 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:28.240543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ef479-0324-41fa-86b2-7141ebcefa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:28.323698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-745ef479-0324-41fa-86b2-7141ebcefa24 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:28.324839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-745ef479-0324-41fa-86b2-7141ebcefa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:28.365003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:28.562299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 677/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:28.897252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2fde1-fd3f-430d-885b-788ce613a5be None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:28.899482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2dc2fde1-fd3f-430d-885b-788ce613a5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:28.969332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:29.058675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2fde1-fd3f-430d-885b-788ce613a5be None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:29.186024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a4282-a2ff-4b5f-bacd-80ddbfb94628 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:29.186640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a4282-a2ff-4b5f-bacd-80ddbfb94628 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:29.186931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a4282-a2ff-4b5f-bacd-80ddbfb94628 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:29.187202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a4282-a2ff-4b5f-bacd-80ddbfb94628 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:29.188813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ff8a4282-a2ff-4b5f-bacd-80ddbfb94628 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:29.219887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:29.293181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a4282-a2ff-4b5f-bacd-80ddbfb94628 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:29.294879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a4282-a2ff-4b5f-bacd-80ddbfb94628 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:29.313786 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 686/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:32.088759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6d8cf-8ddf-4d20-bd64-279e7399f279 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:32.089592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6d8cf-8ddf-4d20-bd64-279e7399f279 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:32.090338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6d8cf-8ddf-4d20-bd64-279e7399f279 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:32.091836 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 678/1364] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:32.097250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b3ff4-e035-4c0f-9ab5-2555bda9dcc2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:32.097659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b3ff4-e035-4c0f-9ab5-2555bda9dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:32.098059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b3ff4-e035-4c0f-9ab5-2555bda9dcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:32.177087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ad7b3ff4-e035-4c0f-9ab5-2555bda9dcc2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:32.178501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad7b3ff4-e035-4c0f-9ab5-2555bda9dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:32.302465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:32.302747 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 687/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:32.695591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468eec5-7e38-42de-82b6-c5ea9bbc09cf None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:32.696186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468eec5-7e38-42de-82b6-c5ea9bbc09cf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:32.696725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468eec5-7e38-42de-82b6-c5ea9bbc09cf None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:32.698900 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 679/1366] 10.222.0.33 () {54 vars in 962 bytes} [Sat Sep 10 20:38:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:32.705359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68336a28-c740-4a34-b94b-40ee169cfd22 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:32.705359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68336a28-c740-4a34-b94b-40ee169cfd22 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:32.706187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68336a28-c740-4a34-b94b-40ee169cfd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:32.812751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-68336a28-c740-4a34-b94b-40ee169cfd22 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:32.814062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-68336a28-c740-4a34-b94b-40ee169cfd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:32.992855 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 688/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Sat Sep 10 20:38:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:33.002379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468eec5-7e38-42de-82b6-c5ea9bbc09cf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:33.004172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6468eec5-7e38-42de-82b6-c5ea9bbc09cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:33.264255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468eec5-7e38-42de-82b6-c5ea9bbc09cf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:33.332800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:33.439531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dda298-3e37-49e0-8856-7e74f34a7527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:33.440142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dda298-3e37-49e0-8856-7e74f34a7527 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:33.440422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dda298-3e37-49e0-8856-7e74f34a7527 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:33.440691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dda298-3e37-49e0-8856-7e74f34a7527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:33.442310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c4dda298-3e37-49e0-8856-7e74f34a7527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:33.543902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dda298-3e37-49e0-8856-7e74f34a7527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:33.545715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dda298-3e37-49e0-8856-7e74f34a7527 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:33.568091 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 680/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:36.525741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453423d8-04e4-4537-b3ad-bd4f175d13c1 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:36.526315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453423d8-04e4-4537-b3ad-bd4f175d13c1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:36.526413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453423d8-04e4-4537-b3ad-bd4f175d13c1 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:36.527991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 689/1369] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:36.533517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b302489-0474-4a85-9a27-7e0a732b3c2b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:36.534038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b302489-0474-4a85-9a27-7e0a732b3c2b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:36.534134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b302489-0474-4a85-9a27-7e0a732b3c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:36.572935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:36.623438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8b302489-0474-4a85-9a27-7e0a732b3c2b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:36.624694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8b302489-0474-4a85-9a27-7e0a732b3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:36.751128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 681/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:37.059705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cdfb-3e08-4ce5-b5d0-ef28684155f4 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:37.060046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cdfb-3e08-4ce5-b5d0-ef28684155f4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:37.060262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cdfb-3e08-4ce5-b5d0-ef28684155f4 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:37.061775 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 690/1371] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:37.067468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc5e43-1e7c-45f3-b715-1a2c0dc36119 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:37.067764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc5e43-1e7c-45f3-b715-1a2c0dc36119 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:37.068044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc5e43-1e7c-45f3-b715-1a2c0dc36119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:37.115444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cdfb-3e08-4ce5-b5d0-ef28684155f4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:37.117907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7895cdfb-3e08-4ce5-b5d0-ef28684155f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:37.144689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-18bc5e43-1e7c-45f3-b715-1a2c0dc36119 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:37.145908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-18bc5e43-1e7c-45f3-b715-1a2c0dc36119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:37.170962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:37.208024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:37.267569 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 682/1372] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:37.294240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cdfb-3e08-4ce5-b5d0-ef28684155f4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:37.426299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c872bad-e65d-417c-8154-06572625d090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:37.426812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c872bad-e65d-417c-8154-06572625d090 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:37.427087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c872bad-e65d-417c-8154-06572625d090 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:37.427373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c872bad-e65d-417c-8154-06572625d090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:37.430244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c872bad-e65d-417c-8154-06572625d090 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:37.440974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:37.536442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c872bad-e65d-417c-8154-06572625d090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:37.538222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c872bad-e65d-417c-8154-06572625d090 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:37.557343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 691/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:37.820849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fcda1-67cb-4179-98f6-0f16c5911a24 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:37.821436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fcda1-67cb-4179-98f6-0f16c5911a24 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:37.821883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fcda1-67cb-4179-98f6-0f16c5911a24 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:37.823530 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 683/1374] 10.222.0.33 () {54 vars in 964 bytes} [Sat Sep 10 20:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:37.829277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8e663-0e51-4763-9787-31ef5f2f2c8d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:37.829556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8e663-0e51-4763-9787-31ef5f2f2c8d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:37.829871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8e663-0e51-4763-9787-31ef5f2f2c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:37.914580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d8e8e663-0e51-4763-9787-31ef5f2f2c8d None None] MFA Rules not processed for user `5fe5d3512475454080c8528db2208241`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:37.915922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d8e8e663-0e51-4763-9787-31ef5f2f2c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:38.000222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:38.007986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 692/1375] 10.222.0.33 () {58 vars in 1061 bytes} [Sat Sep 10 20:38:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:38.011323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fcda1-67cb-4179-98f6-0f16c5911a24 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:38.012887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c70fcda1-67cb-4179-98f6-0f16c5911a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:38.127723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fcda1-67cb-4179-98f6-0f16c5911a24 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:38.208962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6682abf-cac1-4914-8b9c-ca5dd15e930e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:38.209583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6682abf-cac1-4914-8b9c-ca5dd15e930e service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:38.209921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6682abf-cac1-4914-8b9c-ca5dd15e930e service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:38.210788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6682abf-cac1-4914-8b9c-ca5dd15e930e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:38.212481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b6682abf-cac1-4914-8b9c-ca5dd15e930e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:38.223210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:38.338742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6682abf-cac1-4914-8b9c-ca5dd15e930e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:38.340671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6682abf-cac1-4914-8b9c-ca5dd15e930e service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:38.363008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 684/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:40.250231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf60b-8617-4b56-be49-c69ec544e2ff None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:40.250867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf60b-8617-4b56-be49-c69ec544e2ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:40.250867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cf60b-8617-4b56-be49-c69ec544e2ff None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:40.252124 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 693/1377] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:40.258623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f56336-613b-4680-9980-990c74432828 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:40.259230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f56336-613b-4680-9980-990c74432828 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:40.259230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f56336-613b-4680-9980-990c74432828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:40.333401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d1f56336-613b-4680-9980-990c74432828 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:40.334690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d1f56336-613b-4680-9980-990c74432828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:40.398622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:40.439041 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 685/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:40.788080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf60b-8617-4b56-be49-c69ec544e2ff None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:40.790414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-154cf60b-8617-4b56-be49-c69ec544e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:40.949955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cf60b-8617-4b56-be49-c69ec544e2ff None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:41.030651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:41.078511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc4cb3-b8a0-4589-8f7c-59484f8d8c75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:41.078511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc4cb3-b8a0-4589-8f7c-59484f8d8c75 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:41.078511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc4cb3-b8a0-4589-8f7c-59484f8d8c75 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:41.078511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc4cb3-b8a0-4589-8f7c-59484f8d8c75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:41.078908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-78fc4cb3-b8a0-4589-8f7c-59484f8d8c75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:41.180728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc4cb3-b8a0-4589-8f7c-59484f8d8c75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:41.182365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc4cb3-b8a0-4589-8f7c-59484f8d8c75 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:41.202467 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 694/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:43.843606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0461341-8dd2-456b-b48c-82a25266896c None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:43.844229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0461341-8dd2-456b-b48c-82a25266896c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:43.844229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0461341-8dd2-456b-b48c-82a25266896c None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:43.845639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 686/1380] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:43.851359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc2d3-d197-4380-a41d-73a0314355e7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:43.851791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc2d3-d197-4380-a41d-73a0314355e7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:43.851980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc2d3-d197-4380-a41d-73a0314355e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:43.929114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a35cc2d3-d197-4380-a41d-73a0314355e7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:43.930513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a35cc2d3-d197-4380-a41d-73a0314355e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:43.979237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:44.046528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 695/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:44.397990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0461341-8dd2-456b-b48c-82a25266896c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:44.399890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b0461341-8dd2-456b-b48c-82a25266896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:44.575366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0461341-8dd2-456b-b48c-82a25266896c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:44.626305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:44.708905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36b735-0995-4d05-baec-9e7312632138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:44.709717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36b735-0995-4d05-baec-9e7312632138 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:44.710165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36b735-0995-4d05-baec-9e7312632138 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:44.710659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36b735-0995-4d05-baec-9e7312632138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:44.712514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3f36b735-0995-4d05-baec-9e7312632138 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:44.825965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36b735-0995-4d05-baec-9e7312632138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:44.827566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36b735-0995-4d05-baec-9e7312632138 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:44.846847 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 687/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:47.421282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da13a3d-582d-426f-b2ab-03acf4c94be9 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:47.421911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da13a3d-582d-426f-b2ab-03acf4c94be9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:47.422286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da13a3d-582d-426f-b2ab-03acf4c94be9 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:47.424213 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 696/1383] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:47.431891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcd22f-69a6-4c43-9de4-974ea1209878 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:47.432982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcd22f-69a6-4c43-9de4-974ea1209878 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:47.433402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcd22f-69a6-4c43-9de4-974ea1209878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:47.520064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f3dcd22f-69a6-4c43-9de4-974ea1209878 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:47.521285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f3dcd22f-69a6-4c43-9de4-974ea1209878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:47.524572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:47.655269 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 688/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:48.007466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da13a3d-582d-426f-b2ab-03acf4c94be9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:48.009352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3da13a3d-582d-426f-b2ab-03acf4c94be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:48.176193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da13a3d-582d-426f-b2ab-03acf4c94be9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:48.205893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:48.300448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369f66b-af04-478a-b6a4-5b53e4378d45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:48.301069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369f66b-af04-478a-b6a4-5b53e4378d45 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:48.301358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369f66b-af04-478a-b6a4-5b53e4378d45 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:48.301639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369f66b-af04-478a-b6a4-5b53e4378d45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:48.303944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a369f66b-af04-478a-b6a4-5b53e4378d45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:48.408493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369f66b-af04-478a-b6a4-5b53e4378d45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:48.410155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369f66b-af04-478a-b6a4-5b53e4378d45 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:48.429701 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 697/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:50.895641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf78193-7af7-4e87-9999-2823aa0e1085 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:50.896288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf78193-7af7-4e87-9999-2823aa0e1085 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:50.896288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf78193-7af7-4e87-9999-2823aa0e1085 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:50.901949 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 689/1386] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:50.903643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e052bf-3e96-4da4-9588-86514974c41f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:50.904140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e052bf-3e96-4da4-9588-86514974c41f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:50.904298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e052bf-3e96-4da4-9588-86514974c41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:50.942090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:50.984377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-00e052bf-3e96-4da4-9588-86514974c41f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:50.985802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-00e052bf-3e96-4da4-9588-86514974c41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:51.094711 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 698/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:51.459127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf78193-7af7-4e87-9999-2823aa0e1085 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:51.461476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ddf78193-7af7-4e87-9999-2823aa0e1085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:51.603518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:51.621158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf78193-7af7-4e87-9999-2823aa0e1085 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:51.753652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f2a41-0b95-4fb0-89cc-5019208e089c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:51.754307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2a41-0b95-4fb0-89cc-5019208e089c service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:51.754589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2a41-0b95-4fb0-89cc-5019208e089c service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:51.754864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2a41-0b95-4fb0-89cc-5019208e089c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:51.756528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f81f2a41-0b95-4fb0-89cc-5019208e089c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:51.864073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f2a41-0b95-4fb0-89cc-5019208e089c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:51.865741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f2a41-0b95-4fb0-89cc-5019208e089c service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:51.873661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:51.888382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 690/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:54.592761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65f93a-a294-49ab-a1db-bb4ac2dc7796 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:54.593401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65f93a-a294-49ab-a1db-bb4ac2dc7796 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:54.593401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65f93a-a294-49ab-a1db-bb4ac2dc7796 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:54.594692 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 699/1389] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:54.600720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e97b8a-a079-47d9-ba54-547c9de0bc79 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:54.601429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e97b8a-a079-47d9-ba54-547c9de0bc79 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:54.601429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e97b8a-a079-47d9-ba54-547c9de0bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:54.670804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-88e97b8a-a079-47d9-ba54-547c9de0bc79 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:54.672461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-88e97b8a-a079-47d9-ba54-547c9de0bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:54.776914 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 691/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:55.129196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65f93a-a294-49ab-a1db-bb4ac2dc7796 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:55.130724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8a65f93a-a294-49ab-a1db-bb4ac2dc7796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:55.235280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:55.281299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f61d6c-c419-4ce1-af05-253df2af7d07 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:55.281591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f61d6c-c419-4ce1-af05-253df2af7d07 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:55.281893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f61d6c-c419-4ce1-af05-253df2af7d07 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:55.283409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 692/1391] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:38:55.285497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65f93a-a294-49ab-a1db-bb4ac2dc7796 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:55.288501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043ed69-333f-4efe-b8e0-c80c7e9b453f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:55.288975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043ed69-333f-4efe-b8e0-c80c7e9b453f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:55.289386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043ed69-333f-4efe-b8e0-c80c7e9b453f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:55.368482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a043ed69-333f-4efe-b8e0-c80c7e9b453f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:55.369710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a043ed69-333f-4efe-b8e0-c80c7e9b453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:55.391104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:55.408455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f92a79-b156-4fc5-a3ff-443837947087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:55.409058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92a79-b156-4fc5-a3ff-443837947087 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:55.409333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92a79-b156-4fc5-a3ff-443837947087 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:55.409601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92a79-b156-4fc5-a3ff-443837947087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:55.411205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b9f92a79-b156-4fc5-a3ff-443837947087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:55.473766 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 693/1392] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:38:55.483343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:55.506057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f92a79-b156-4fc5-a3ff-443837947087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:55.508143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f92a79-b156-4fc5-a3ff-443837947087 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:55.526551 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 700/1393] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:55.850118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c9647-e4c1-433e-b146-970a65d468dd None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:55.851160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c9647-e4c1-433e-b146-970a65d468dd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:55.851334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c9647-e4c1-433e-b146-970a65d468dd None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:55.852637 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 694/1394] 10.222.0.33 () {54 vars in 966 bytes} [Sat Sep 10 20:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:55.858047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c9d06-7fa7-4b16-be46-2cddf6c94557 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:55.858491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c9d06-7fa7-4b16-be46-2cddf6c94557 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:55.858605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c9d06-7fa7-4b16-be46-2cddf6c94557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:55.940837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6f2c9d06-7fa7-4b16-be46-2cddf6c94557 None None] MFA Rules not processed for user `5cdff5f5500040439f4acafa6c9ad948`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:55.942090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6f2c9d06-7fa7-4b16-be46-2cddf6c94557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:56.023617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 701/1395] 10.222.0.33 () {58 vars in 1063 bytes} [Sat Sep 10 20:38:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:38:56.031506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c9647-e4c1-433e-b146-970a65d468dd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:56.033035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc0c9647-e4c1-433e-b146-970a65d468dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:56.141945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c9647-e4c1-433e-b146-970a65d468dd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:56.191838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:56.217132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fefe63-18be-4519-9144-cd416e1dec14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:56.217743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fefe63-18be-4519-9144-cd416e1dec14 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:56.218061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fefe63-18be-4519-9144-cd416e1dec14 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:56.218328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fefe63-18be-4519-9144-cd416e1dec14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:56.219913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90fefe63-18be-4519-9144-cd416e1dec14 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:56.329728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fefe63-18be-4519-9144-cd416e1dec14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:56.331206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fefe63-18be-4519-9144-cd416e1dec14 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:56.350687 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 695/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:38:56.937453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c8fc-bb19-429c-9433-af17a96ab598 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:56.938064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c8fc-bb19-429c-9433-af17a96ab598 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:56.941357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c8fc-bb19-429c-9433-af17a96ab598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:57.016369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-50f1c8fc-bb19-429c-9433-af17a96ab598 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:57.017575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-50f1c8fc-bb19-429c-9433-af17a96ab598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:57.024677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:57.107448 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 702/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:38:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:57.157172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07020cc0-2eb2-4b81-984c-9f138f46a240 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:57.157664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07020cc0-2eb2-4b81-984c-9f138f46a240 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:57.157802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07020cc0-2eb2-4b81-984c-9f138f46a240 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:57.159070 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 696/1398] 10.222.0.33 () {54 vars in 943 bytes} [Sat Sep 10 20:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:57.165708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b1393-6939-4eaa-9499-7c3ef1426e79 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:57.166178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b1393-6939-4eaa-9499-7c3ef1426e79 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:57.166592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b1393-6939-4eaa-9499-7c3ef1426e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:57.243455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-997b1393-6939-4eaa-9499-7c3ef1426e79 None None] MFA Rules not processed for user `32f1fd57a99e4c0b9e50e997c5ee74f1`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:57.244647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-997b1393-6939-4eaa-9499-7c3ef1426e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:57.327799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:57.357425 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 703/1399] 10.222.0.33 () {58 vars in 1040 bytes} [Sat Sep 10 20:38:57 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:57.366757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07020cc0-2eb2-4b81-984c-9f138f46a240 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:57.368408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-07020cc0-2eb2-4b81-984c-9f138f46a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:57.451232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:57.510155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07020cc0-2eb2-4b81-984c-9f138f46a240 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:57.622723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94ec3e-4334-4aa4-882d-9be69cccaad3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:57.623242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94ec3e-4334-4aa4-882d-9be69cccaad3 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:57.623523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94ec3e-4334-4aa4-882d-9be69cccaad3 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:57.623795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94ec3e-4334-4aa4-882d-9be69cccaad3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:57.625432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7c94ec3e-4334-4aa4-882d-9be69cccaad3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:57.692657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:57.713655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94ec3e-4334-4aa4-882d-9be69cccaad3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:57.715787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94ec3e-4334-4aa4-882d-9be69cccaad3 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:57.724177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 697/1400] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 20:38:58.040048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936dca3-3c07-4f5a-aaca-4d84d42e2bd0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:58.040627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936dca3-3c07-4f5a-aaca-4d84d42e2bd0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:58.040627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936dca3-3c07-4f5a-aaca-4d84d42e2bd0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:58.041936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 704/1401] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:38:58.047274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930628c-c35e-4901-986d-eb88b8021b98 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:58.047631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930628c-c35e-4901-986d-eb88b8021b98 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:58.047971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930628c-c35e-4901-986d-eb88b8021b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:58.124407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6930628c-c35e-4901-986d-eb88b8021b98 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:38:58.125715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6930628c-c35e-4901-986d-eb88b8021b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:58.241318 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 698/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:38:58.592632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936dca3-3c07-4f5a-aaca-4d84d42e2bd0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:38:58.594238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0936dca3-3c07-4f5a-aaca-4d84d42e2bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:58.768895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936dca3-3c07-4f5a-aaca-4d84d42e2bd0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:38:58.839299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:38:58.902772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c77c3-9619-486f-b93c-4fca4c58f0f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:38:58.903732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c77c3-9619-486f-b93c-4fca4c58f0f7 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:38:58.904239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c77c3-9619-486f-b93c-4fca4c58f0f7 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:38:58.904512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c77c3-9619-486f-b93c-4fca4c58f0f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:38:58.906168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c26c77c3-9619-486f-b93c-4fca4c58f0f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:38:59.030064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c77c3-9619-486f-b93c-4fca4c58f0f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:38:59.031690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c77c3-9619-486f-b93c-4fca4c58f0f7 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:38:59.052478 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 705/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:01.604135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab39bae-dba5-4dd9-8be2-d571bc60cf2b None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:01.604779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab39bae-dba5-4dd9-8be2-d571bc60cf2b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:01.604779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab39bae-dba5-4dd9-8be2-d571bc60cf2b None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:01.606060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 699/1404] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:01.611323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfdfa5-5575-4c0e-aa58-19d221f4b6d0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:01.611768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfdfa5-5575-4c0e-aa58-19d221f4b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:01.611871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfdfa5-5575-4c0e-aa58-19d221f4b6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:01.683824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-36dfdfa5-5575-4c0e-aa58-19d221f4b6d0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:01.685032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36dfdfa5-5575-4c0e-aa58-19d221f4b6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:01.702877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:01.793877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 706/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:02.147400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab39bae-dba5-4dd9-8be2-d571bc60cf2b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:02.149174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9ab39bae-dba5-4dd9-8be2-d571bc60cf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:02.220697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:02.317378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab39bae-dba5-4dd9-8be2-d571bc60cf2b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:02.452592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7965c80-6d88-427f-9763-02775a8c4f23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:02.453192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965c80-6d88-427f-9763-02775a8c4f23 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:02.453495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965c80-6d88-427f-9763-02775a8c4f23 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:02.453763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965c80-6d88-427f-9763-02775a8c4f23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:02.456958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f7965c80-6d88-427f-9763-02775a8c4f23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:02.478981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:02.556830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7965c80-6d88-427f-9763-02775a8c4f23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:02.558486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7965c80-6d88-427f-9763-02775a8c4f23 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:02.577691 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 700/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:05.105071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c49-2767-4ac2-aee3-d76b9284040b None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:05.105723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c49-2767-4ac2-aee3-d76b9284040b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:05.105723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c49-2767-4ac2-aee3-d76b9284040b None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:05.107057 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 707/1407] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:05.112595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffd017-6f14-464b-99c0-c8ca1808a2cd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:05.113054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffd017-6f14-464b-99c0-c8ca1808a2cd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:05.113146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffd017-6f14-464b-99c0-c8ca1808a2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:05.184584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9fffd017-6f14-464b-99c0-c8ca1808a2cd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:05.185807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9fffd017-6f14-464b-99c0-c8ca1808a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:05.285235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:05.292805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 701/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:05.665659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4c49-2767-4ac2-aee3-d76b9284040b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:05.668075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a52d4c49-2767-4ac2-aee3-d76b9284040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:05.839703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:05.843168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4c49-2767-4ac2-aee3-d76b9284040b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:05.999903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3f566-6cf1-4651-a5d2-a2e241ea39ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:06.000510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3f566-6cf1-4651-a5d2-a2e241ea39ea service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:06.000788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3f566-6cf1-4651-a5d2-a2e241ea39ea service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:06.001059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3f566-6cf1-4651-a5d2-a2e241ea39ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:06.002828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b7e3f566-6cf1-4651-a5d2-a2e241ea39ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:06.109137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3f566-6cf1-4651-a5d2-a2e241ea39ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:06.110945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3f566-6cf1-4651-a5d2-a2e241ea39ea service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:06.132609 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 708/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:15.293714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:16.127593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:21.702618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2d717-7929-4d2f-8ccc-b3e810f1f045 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:21.703206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2d717-7929-4d2f-8ccc-b3e810f1f045 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:21.703206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2d717-7929-4d2f-8ccc-b3e810f1f045 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:21.704524 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 702/1410] 10.222.0.33 () {54 vars in 918 bytes} [Sat Sep 10 20:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:21.709869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0dde-8d9a-4eec-9f77-303c6558f19a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:21.710347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0dde-8d9a-4eec-9f77-303c6558f19a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:21.711072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0dde-8d9a-4eec-9f77-303c6558f19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:21.788279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fa2a0dde-8d9a-4eec-9f77-303c6558f19a None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:21.789526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fa2a0dde-8d9a-4eec-9f77-303c6558f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:21.895332 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 709/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Sep 10 20:39:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:21.910065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63232c30-4d44-4fed-81a4-c15a195a0f0e None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:21.910351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63232c30-4d44-4fed-81a4-c15a195a0f0e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:21.910628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63232c30-4d44-4fed-81a4-c15a195a0f0e None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:21.912478 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 703/1412] 10.222.0.33 () {54 vars in 918 bytes} [Sat Sep 10 20:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:21.917345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683e7ef-4d90-4db3-bcb0-2b4557d0e6a8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:21.917635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683e7ef-4d90-4db3-bcb0-2b4557d0e6a8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:21.917940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683e7ef-4d90-4db3-bcb0-2b4557d0e6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:21.991441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:21.994629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2683e7ef-4d90-4db3-bcb0-2b4557d0e6a8 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:21.996415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2683e7ef-4d90-4db3-bcb0-2b4557d0e6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:22.113260 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 710/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Sep 10 20:39:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:24.951674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9cd7-a047-43de-959d-0aeaf1c44426 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:24.952248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9cd7-a047-43de-959d-0aeaf1c44426 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:24.952248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9cd7-a047-43de-959d-0aeaf1c44426 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:24.953513 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:24.959324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288066-a6b6-4373-a947-88d7b12270c5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:24.959775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288066-a6b6-4373-a947-88d7b12270c5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:24.959914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288066-a6b6-4373-a947-88d7b12270c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:25.026754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-22288066-a6b6-4373-a947-88d7b12270c5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:25.027953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-22288066-a6b6-4373-a947-88d7b12270c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:25.083547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:25.125921 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:25.469584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9cd7-a047-43de-959d-0aeaf1c44426 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:25.471311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c7b9cd7-a047-43de-959d-0aeaf1c44426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:25.618471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9cd7-a047-43de-959d-0aeaf1c44426 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:25.690673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:25.731560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388ae7f-e000-43ea-b482-83acd90f572f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:25.732085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388ae7f-e000-43ea-b482-83acd90f572f service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:25.732363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388ae7f-e000-43ea-b482-83acd90f572f service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:25.732636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388ae7f-e000-43ea-b482-83acd90f572f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:25.734275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4388ae7f-e000-43ea-b482-83acd90f572f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:25.824934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388ae7f-e000-43ea-b482-83acd90f572f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:25.826573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388ae7f-e000-43ea-b482-83acd90f572f service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:25.844522 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 705/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:27.582087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee9923-48da-4504-a145-675798df6f4f None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:27.582661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee9923-48da-4504-a145-675798df6f4f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:27.582661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee9923-48da-4504-a145-675798df6f4f None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:27.584509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:27.589388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523534-e4ae-48a7-a37c-7d54afe68b07 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:27.589809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523534-e4ae-48a7-a37c-7d54afe68b07 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:27.589974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44523534-e4ae-48a7-a37c-7d54afe68b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:27.669131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-44523534-e4ae-48a7-a37c-7d54afe68b07 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:27.671300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-44523534-e4ae-48a7-a37c-7d54afe68b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:27.703285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:27.773234 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:27.976951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee9923-48da-4504-a145-675798df6f4f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:27.978024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-adee9923-48da-4504-a145-675798df6f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:28.126738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee9923-48da-4504-a145-675798df6f4f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:28.167245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:28.239837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2837a-4d9b-4f20-9091-4176693f933f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:28.240438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2837a-4d9b-4f20-9091-4176693f933f service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:28.240721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2837a-4d9b-4f20-9091-4176693f933f service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:28.241001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2837a-4d9b-4f20-9091-4176693f933f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:28.242646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7cb2837a-4d9b-4f20-9091-4176693f933f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:28.331087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2837a-4d9b-4f20-9091-4176693f933f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:28.332713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2837a-4d9b-4f20-9091-4176693f933f service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:28.352414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 713/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:30.076485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8d87-8d3a-46a3-beff-d33a98726102 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:30.077043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8d87-8d3a-46a3-beff-d33a98726102 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:30.077043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8d87-8d3a-46a3-beff-d33a98726102 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:30.078288 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:30.083252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62f927-d366-40d0-be8d-eea8ce3bc90a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:30.084908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62f927-d366-40d0-be8d-eea8ce3bc90a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:30.084908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62f927-d366-40d0-be8d-eea8ce3bc90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:30.138554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:30.160984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8f62f927-d366-40d0-be8d-eea8ce3bc90a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:30.162437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8f62f927-d366-40d0-be8d-eea8ce3bc90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:30.259268 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:30.421312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8d87-8d3a-46a3-beff-d33a98726102 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:30.422817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-88fe8d87-8d3a-46a3-beff-d33a98726102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:30.565553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8d87-8d3a-46a3-beff-d33a98726102 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:30.577923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:30.679287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee408f2-e570-4e54-81a4-5c2606c4aece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:30.679806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee408f2-e570-4e54-81a4-5c2606c4aece service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:30.680083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee408f2-e570-4e54-81a4-5c2606c4aece service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:30.680352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee408f2-e570-4e54-81a4-5c2606c4aece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:30.681998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1ee408f2-e570-4e54-81a4-5c2606c4aece service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:30.773078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee408f2-e570-4e54-81a4-5c2606c4aece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:30.775321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee408f2-e570-4e54-81a4-5c2606c4aece service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:30.793724 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 708/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:32.472763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b74a-881a-4ff0-8704-050a347890e3 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:32.473319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b74a-881a-4ff0-8704-050a347890e3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:32.473319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b74a-881a-4ff0-8704-050a347890e3 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:32.474899 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:32.479725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b6620-00a0-4366-8186-f91d59f237d0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:32.480147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b6620-00a0-4366-8186-f91d59f237d0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:32.480280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b6620-00a0-4366-8186-f91d59f237d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:32.500019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:32.555744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3d8b6620-00a0-4366-8186-f91d59f237d0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:32.556949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3d8b6620-00a0-4366-8186-f91d59f237d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:32.659186 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:32.850597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930b74a-881a-4ff0-8704-050a347890e3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:32.852164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4930b74a-881a-4ff0-8704-050a347890e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:32.963749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:32.999869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930b74a-881a-4ff0-8704-050a347890e3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:33.114050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa906c-6376-4863-9f8a-20b1c7f42d37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:33.115478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa906c-6376-4863-9f8a-20b1c7f42d37 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:33.115751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa906c-6376-4863-9f8a-20b1c7f42d37 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:33.116019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa906c-6376-4863-9f8a-20b1c7f42d37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:33.117603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a5fa906c-6376-4863-9f8a-20b1c7f42d37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:33.195859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:33.210148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa906c-6376-4863-9f8a-20b1c7f42d37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:33.211759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa906c-6376-4863-9f8a-20b1c7f42d37 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:33.230549 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 716/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:34.912272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2af83-3897-41aa-8340-05050bfdb044 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:34.912842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2af83-3897-41aa-8340-05050bfdb044 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:34.912842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2af83-3897-41aa-8340-05050bfdb044 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:34.914169 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:39:34.919002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c69f8-4496-4e5c-98c7-cc60a7c2dcf3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:34.919460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c69f8-4496-4e5c-98c7-cc60a7c2dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:34.919560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c69f8-4496-4e5c-98c7-cc60a7c2dcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:34.995657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7d7c69f8-4496-4e5c-98c7-cc60a7c2dcf3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:34.997130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7d7c69f8-4496-4e5c-98c7-cc60a7c2dcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:35.094124 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:35.255489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2af83-3897-41aa-8340-05050bfdb044 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:35.257236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-61f2af83-3897-41aa-8340-05050bfdb044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:35.341599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:35.402850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2af83-3897-41aa-8340-05050bfdb044 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:35.515569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc2041-88eb-44f0-8e49-def9c99e6fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:35.516092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc2041-88eb-44f0-8e49-def9c99e6fad service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:35.516365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc2041-88eb-44f0-8e49-def9c99e6fad service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:35.516632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc2041-88eb-44f0-8e49-def9c99e6fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:35.518241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c9cc2041-88eb-44f0-8e49-def9c99e6fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:35.577207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:35.619383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc2041-88eb-44f0-8e49-def9c99e6fad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:35.620978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc2041-88eb-44f0-8e49-def9c99e6fad service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:35.639721 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 711/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:37.264603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332a98-40c6-4b19-b353-95eb0804e7a7 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:37.265140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332a98-40c6-4b19-b353-95eb0804e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:37.265140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332a98-40c6-4b19-b353-95eb0804e7a7 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:37.266711 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:37.271876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda62bf4-38a6-47c7-a6f3-f3e9f493414d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:37.272324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda62bf4-38a6-47c7-a6f3-f3e9f493414d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:37.272435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda62bf4-38a6-47c7-a6f3-f3e9f493414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:37.354980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fda62bf4-38a6-47c7-a6f3-f3e9f493414d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:37.356184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fda62bf4-38a6-47c7-a6f3-f3e9f493414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:37.454846 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:39:37.619780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1332a98-40c6-4b19-b353-95eb0804e7a7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:37.621367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d1332a98-40c6-4b19-b353-95eb0804e7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:37.671890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:37.768813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1332a98-40c6-4b19-b353-95eb0804e7a7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:37.878471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bb502-e258-4421-bea7-fb18ce01e675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:37.878997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bb502-e258-4421-bea7-fb18ce01e675 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:37.879273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bb502-e258-4421-bea7-fb18ce01e675 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:37.879543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bb502-e258-4421-bea7-fb18ce01e675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:37.881862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-893bb502-e258-4421-bea7-fb18ce01e675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:37.888556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:37.972522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bb502-e258-4421-bea7-fb18ce01e675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:37.975126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bb502-e258-4421-bea7-fb18ce01e675 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:37.993374 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 719/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:39.610780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771d713-752e-40b6-849a-cc4a00bf2772 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:39.611328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771d713-752e-40b6-849a-cc4a00bf2772 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:39.611328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771d713-752e-40b6-849a-cc4a00bf2772 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:39.612640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:39.617946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094e52-45ef-4ab6-ad9a-6714cd865af6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:39.618354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094e52-45ef-4ab6-ad9a-6714cd865af6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:39.618498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094e52-45ef-4ab6-ad9a-6714cd865af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:39.693656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-af094e52-45ef-4ab6-ad9a-6714cd865af6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:39.694916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af094e52-45ef-4ab6-ad9a-6714cd865af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:39.763470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:39.793348 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:39.955168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771d713-752e-40b6-849a-cc4a00bf2772 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:39.956742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0771d713-752e-40b6-849a-cc4a00bf2772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:39.975456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:40.102493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771d713-752e-40b6-849a-cc4a00bf2772 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:40.190884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:40.217137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bba0b-7aac-4088-98ef-31417ec6036a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:40.217764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bba0b-7aac-4088-98ef-31417ec6036a service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:40.218070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bba0b-7aac-4088-98ef-31417ec6036a service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:40.218337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bba0b-7aac-4088-98ef-31417ec6036a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:40.219913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e68bba0b-7aac-4088-98ef-31417ec6036a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:40.307747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bba0b-7aac-4088-98ef-31417ec6036a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:40.309314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bba0b-7aac-4088-98ef-31417ec6036a service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:40.326729 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 714/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:41.945021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690e5c3-032e-4b72-bac9-2ee0e3b1285e None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:41.945521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690e5c3-032e-4b72-bac9-2ee0e3b1285e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:41.945628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690e5c3-032e-4b72-bac9-2ee0e3b1285e None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:41.946992 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:41.951782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60627d9-272a-4c16-b464-e460e228fcd8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:41.952170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60627d9-272a-4c16-b464-e460e228fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:41.952334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60627d9-272a-4c16-b464-e460e228fcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:42.027806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c60627d9-272a-4c16-b464-e460e228fcd8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:42.029010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c60627d9-272a-4c16-b464-e460e228fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:42.070608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:42.127266 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:42.289210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690e5c3-032e-4b72-bac9-2ee0e3b1285e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:42.290693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a690e5c3-032e-4b72-bac9-2ee0e3b1285e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:42.435224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690e5c3-032e-4b72-bac9-2ee0e3b1285e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:42.489417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:42.547310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7b6c2-95fb-48ae-9268-875118309546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:42.547838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7b6c2-95fb-48ae-9268-875118309546 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:42.548116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7b6c2-95fb-48ae-9268-875118309546 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:42.548382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7b6c2-95fb-48ae-9268-875118309546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:42.550026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bcd7b6c2-95fb-48ae-9268-875118309546 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:42.639639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7b6c2-95fb-48ae-9268-875118309546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:42.641239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7b6c2-95fb-48ae-9268-875118309546 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:42.660017 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 722/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:44.381415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb874-d1a1-4a47-802f-24458b8babec None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:44.381994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb874-d1a1-4a47-802f-24458b8babec None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:44.381994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cb874-d1a1-4a47-802f-24458b8babec None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:44.383378 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:39:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:44.388776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a28dc-d39e-41ca-a877-d848b97dcfa2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:44.389237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a28dc-d39e-41ca-a877-d848b97dcfa2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:44.389342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a28dc-d39e-41ca-a877-d848b97dcfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:44.480852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b66a28dc-d39e-41ca-a877-d848b97dcfa2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:44.482104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b66a28dc-d39e-41ca-a877-d848b97dcfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:44.484178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:44.594084 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:44.757009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cb874-d1a1-4a47-802f-24458b8babec None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:44.758508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-220cb874-d1a1-4a47-802f-24458b8babec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:44.906550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cb874-d1a1-4a47-802f-24458b8babec None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:44.934099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:45.020902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3607ec-ec37-4319-9eca-93a5d55fafc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:45.021433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3607ec-ec37-4319-9eca-93a5d55fafc7 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:45.021712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3607ec-ec37-4319-9eca-93a5d55fafc7 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:45.022019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3607ec-ec37-4319-9eca-93a5d55fafc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:45.023628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0e3607ec-ec37-4319-9eca-93a5d55fafc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:45.114009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3607ec-ec37-4319-9eca-93a5d55fafc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:45.116360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3607ec-ec37-4319-9eca-93a5d55fafc7 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:45.134319 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 717/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:47.468284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab93aa6-8f4c-4643-95f9-b2e5ce0d571d None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:47.468882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab93aa6-8f4c-4643-95f9-b2e5ce0d571d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:47.468882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab93aa6-8f4c-4643-95f9-b2e5ce0d571d None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:47.470281 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:47.475566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cc23-94aa-47f9-9f9c-6b9929428838 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:47.476013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cc23-94aa-47f9-9f9c-6b9929428838 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:47.476146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cc23-94aa-47f9-9f9c-6b9929428838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:47.516516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:47.543205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7981cc23-94aa-47f9-9f9c-6b9929428838 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:47.544409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7981cc23-94aa-47f9-9f9c-6b9929428838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:47.641457 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:47.992186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab93aa6-8f4c-4643-95f9-b2e5ce0d571d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:47.993726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eab93aa6-8f4c-4643-95f9-b2e5ce0d571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:48.103288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab93aa6-8f4c-4643-95f9-b2e5ce0d571d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:48.136039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:48.180243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08534363-d6e4-4f2c-b3b0-370640b844e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:48.180767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534363-d6e4-4f2c-b3b0-370640b844e0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:48.181048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534363-d6e4-4f2c-b3b0-370640b844e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:48.181322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534363-d6e4-4f2c-b3b0-370640b844e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:48.182956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08534363-d6e4-4f2c-b3b0-370640b844e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:48.291371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08534363-d6e4-4f2c-b3b0-370640b844e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:48.292795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08534363-d6e4-4f2c-b3b0-370640b844e0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:48.310847 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 725/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:50.891079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bb97f-f7d2-4032-9e3f-dc200a13e6c1 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:50.891618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bb97f-f7d2-4032-9e3f-dc200a13e6c1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:50.891618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bb97f-f7d2-4032-9e3f-dc200a13e6c1 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:50.892957 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:50.899180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490bed9-286e-429a-93d5-607c8ba28cad None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:50.899626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490bed9-286e-429a-93d5-607c8ba28cad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:50.899724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490bed9-286e-429a-93d5-607c8ba28cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:50.965786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c490bed9-286e-429a-93d5-607c8ba28cad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:50.967021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c490bed9-286e-429a-93d5-607c8ba28cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:50.980858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:51.067774 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:51.408752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bb97f-f7d2-4032-9e3f-dc200a13e6c1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:51.410335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-094bb97f-f7d2-4032-9e3f-dc200a13e6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:51.513662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bb97f-f7d2-4032-9e3f-dc200a13e6c1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:51.528477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:51.589542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29be38-2797-4385-9c9a-98ec7b76fd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:51.590412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29be38-2797-4385-9c9a-98ec7b76fd16 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:51.590703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29be38-2797-4385-9c9a-98ec7b76fd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:51.590976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29be38-2797-4385-9c9a-98ec7b76fd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:51.592576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fe29be38-2797-4385-9c9a-98ec7b76fd16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:51.704307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29be38-2797-4385-9c9a-98ec7b76fd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:51.705693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29be38-2797-4385-9c9a-98ec7b76fd16 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:51.725676 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 720/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:54.469576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba631f7-8cad-431e-a2d3-ec9bf2a63ebc None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:54.470165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba631f7-8cad-431e-a2d3-ec9bf2a63ebc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:54.470165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba631f7-8cad-431e-a2d3-ec9bf2a63ebc None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:54.471541 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:54.477931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49085-c5da-4b1d-95b8-254493fdd0b1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:54.478385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49085-c5da-4b1d-95b8-254493fdd0b1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:54.478486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49085-c5da-4b1d-95b8-254493fdd0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:54.526437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:54.545202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d2f49085-c5da-4b1d-95b8-254493fdd0b1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:54.546429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2f49085-c5da-4b1d-95b8-254493fdd0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:54.642995 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:54.992733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba631f7-8cad-431e-a2d3-ec9bf2a63ebc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:54.994291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3ba631f7-8cad-431e-a2d3-ec9bf2a63ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:55.097012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba631f7-8cad-431e-a2d3-ec9bf2a63ebc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:55.136321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:55.169996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7d4b6-3ad0-436f-bfee-c877f9446723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:55.170597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d4b6-3ad0-436f-bfee-c877f9446723 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:55.170873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d4b6-3ad0-436f-bfee-c877f9446723 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:55.171141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d4b6-3ad0-436f-bfee-c877f9446723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:55.172725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f4e7d4b6-3ad0-436f-bfee-c877f9446723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:55.282575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7d4b6-3ad0-436f-bfee-c877f9446723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:55.283927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7d4b6-3ad0-436f-bfee-c877f9446723 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:55.302592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 728/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:39:57.839199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ee1b7-ef72-4d87-a6a7-95ec4d04c583 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:57.839809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ee1b7-ef72-4d87-a6a7-95ec4d04c583 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:57.839809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ee1b7-ef72-4d87-a6a7-95ec4d04c583 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:57.841332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:39:57.846597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc47c-3dc2-41b7-9666-8dd9467f1975 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:57.847055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc47c-3dc2-41b7-9666-8dd9467f1975 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:57.847167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc47c-3dc2-41b7-9666-8dd9467f1975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:57.913095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-322fc47c-3dc2-41b7-9666-8dd9467f1975 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:39:57.914750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-322fc47c-3dc2-41b7-9666-8dd9467f1975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:57.937356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:58.015161 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:39:58.363432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ee1b7-ef72-4d87-a6a7-95ec4d04c583 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:39:58.365679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-298ee1b7-ef72-4d87-a6a7-95ec4d04c583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:58.467393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ee1b7-ef72-4d87-a6a7-95ec4d04c583 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:39:58.490277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:39:58.543977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1a6ec-ded7-47cf-869c-7e1d0a1781bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:39:58.544574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a6ec-ded7-47cf-869c-7e1d0a1781bd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:39:58.544855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a6ec-ded7-47cf-869c-7e1d0a1781bd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:39:58.545127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a6ec-ded7-47cf-869c-7e1d0a1781bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:39:58.546759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fed1a6ec-ded7-47cf-869c-7e1d0a1781bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:39:58.653406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1a6ec-ded7-47cf-869c-7e1d0a1781bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:39:58.654780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1a6ec-ded7-47cf-869c-7e1d0a1781bd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:39:58.675008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 723/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:01.141357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcf505-8b65-484a-b988-f3f4951cc93f None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:01.141958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcf505-8b65-484a-b988-f3f4951cc93f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:01.141958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcf505-8b65-484a-b988-f3f4951cc93f None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:01.143264 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:01.149064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9aeb-6f25-4a94-88f2-0ca9aa9d4d38 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:01.149489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9aeb-6f25-4a94-88f2-0ca9aa9d4d38 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:01.149597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9aeb-6f25-4a94-88f2-0ca9aa9d4d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:01.213582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:01.216977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-92dc9aeb-6f25-4a94-88f2-0ca9aa9d4d38 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:01.218826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-92dc9aeb-6f25-4a94-88f2-0ca9aa9d4d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:01.315077 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:01.663205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcf505-8b65-484a-b988-f3f4951cc93f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:01.664785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-91dcf505-8b65-484a-b988-f3f4951cc93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:01.779409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcf505-8b65-484a-b988-f3f4951cc93f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:01.957120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:01.983998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23576-45b4-452a-94da-11c1ba95e8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:01.984603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23576-45b4-452a-94da-11c1ba95e8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:01.984878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23576-45b4-452a-94da-11c1ba95e8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:01.985144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23576-45b4-452a-94da-11c1ba95e8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:01.986799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-15f23576-45b4-452a-94da-11c1ba95e8a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:02.095232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f23576-45b4-452a-94da-11c1ba95e8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:02.096710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f23576-45b4-452a-94da-11c1ba95e8a7 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:02.115214 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 731/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:04.598755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a6505-f419-476b-8910-44ef4f70048c None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:04.599325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a6505-f419-476b-8910-44ef4f70048c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:04.599325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a6505-f419-476b-8910-44ef4f70048c None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:04.600660 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:04.605909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3366a-5f94-4c1c-af10-b26d11ce777a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:04.606358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3366a-5f94-4c1c-af10-b26d11ce777a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:04.606461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3366a-5f94-4c1c-af10-b26d11ce777a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:04.673703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-efc3366a-5f94-4c1c-af10-b26d11ce777a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:04.674968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-efc3366a-5f94-4c1c-af10-b26d11ce777a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:04.704456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:04.772390 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:05.120012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a6505-f419-476b-8910-44ef4f70048c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:05.121574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a92a6505-f419-476b-8910-44ef4f70048c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:05.223907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a6505-f419-476b-8910-44ef4f70048c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:05.251741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:05.298299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cf870-9145-4ce2-8c96-86c78fae6195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:05.298898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf870-9145-4ce2-8c96-86c78fae6195 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:05.299175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf870-9145-4ce2-8c96-86c78fae6195 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:05.299445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf870-9145-4ce2-8c96-86c78fae6195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:05.301062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-df6cf870-9145-4ce2-8c96-86c78fae6195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:05.407985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cf870-9145-4ce2-8c96-86c78fae6195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:05.409415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cf870-9145-4ce2-8c96-86c78fae6195 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:05.429490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 726/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:07.950195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f18a8-c84d-4913-a816-6701827e45da None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:07.950647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f18a8-c84d-4913-a816-6701827e45da None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:07.950756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f18a8-c84d-4913-a816-6701827e45da None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:07.952067 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:07.958125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befa40a-c120-42ea-a9a9-98d299e3e40e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:07.958569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befa40a-c120-42ea-a9a9-98d299e3e40e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:07.958689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befa40a-c120-42ea-a9a9-98d299e3e40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:08.025446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8befa40a-c120-42ea-a9a9-98d299e3e40e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:08.026707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8befa40a-c120-42ea-a9a9-98d299e3e40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:08.034887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:08.123695 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:08.468594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f18a8-c84d-4913-a816-6701827e45da None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:08.470144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7b7f18a8-c84d-4913-a816-6701827e45da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:08.572941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f18a8-c84d-4913-a816-6701827e45da None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:08.628301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:08.646719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921728d6-e765-4ffe-b145-7322853ddcc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:08.647316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921728d6-e765-4ffe-b145-7322853ddcc7 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:08.647622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921728d6-e765-4ffe-b145-7322853ddcc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:08.647915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921728d6-e765-4ffe-b145-7322853ddcc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:08.649493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-921728d6-e765-4ffe-b145-7322853ddcc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:08.759307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921728d6-e765-4ffe-b145-7322853ddcc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:08.760660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921728d6-e765-4ffe-b145-7322853ddcc7 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:08.778873 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 734/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:11.335486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9895ce0-da5d-4fc3-8a00-702082b57c4f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:11.336041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9895ce0-da5d-4fc3-8a00-702082b57c4f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:11.336041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9895ce0-da5d-4fc3-8a00-702082b57c4f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:11.337347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:40:11.342864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374074-da2e-4fc7-be32-a96f46c765c6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:11.343265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374074-da2e-4fc7-be32-a96f46c765c6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:11.343417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374074-da2e-4fc7-be32-a96f46c765c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:11.408870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ce374074-da2e-4fc7-be32-a96f46c765c6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:11.410638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ce374074-da2e-4fc7-be32-a96f46c765c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:11.448079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:11.507051 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:11.848520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9895ce0-da5d-4fc3-8a00-702082b57c4f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:11.850401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e9895ce0-da5d-4fc3-8a00-702082b57c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:11.953902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9895ce0-da5d-4fc3-8a00-702082b57c4f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:11.995273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:12.031444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820f16b-078f-45af-9675-2f286332c159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:12.032053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820f16b-078f-45af-9675-2f286332c159 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:12.032325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820f16b-078f-45af-9675-2f286332c159 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:12.032596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820f16b-078f-45af-9675-2f286332c159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:12.034232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a820f16b-078f-45af-9675-2f286332c159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:12.144434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820f16b-078f-45af-9675-2f286332c159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:12.145893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820f16b-078f-45af-9675-2f286332c159 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:12.164341 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 729/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:14.587105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d3ba6-77f1-45a7-8c39-5d3129fecaf9 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:14.587656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d3ba6-77f1-45a7-8c39-5d3129fecaf9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:14.587656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d3ba6-77f1-45a7-8c39-5d3129fecaf9 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:14.589093 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:14.594783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58175bcc-7a07-472c-886f-1152da7649c2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:14.595202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58175bcc-7a07-472c-886f-1152da7649c2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:14.595355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58175bcc-7a07-472c-886f-1152da7649c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:14.662419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-58175bcc-7a07-472c-886f-1152da7649c2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:14.663629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-58175bcc-7a07-472c-886f-1152da7649c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:14.681500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:14.760920 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:15.099598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d3ba6-77f1-45a7-8c39-5d3129fecaf9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:15.101136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6a0d3ba6-77f1-45a7-8c39-5d3129fecaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:15.202579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d3ba6-77f1-45a7-8c39-5d3129fecaf9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:15.270325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:15.278140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b50d9d-63aa-4c85-9df7-5a76e1149c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:15.278737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50d9d-63aa-4c85-9df7-5a76e1149c72 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:15.279011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50d9d-63aa-4c85-9df7-5a76e1149c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:15.279280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50d9d-63aa-4c85-9df7-5a76e1149c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:15.280952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c0b50d9d-63aa-4c85-9df7-5a76e1149c72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:15.387759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50d9d-63aa-4c85-9df7-5a76e1149c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:15.389184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50d9d-63aa-4c85-9df7-5a76e1149c72 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:15.407227 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 737/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:17.871936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f292982-b104-4d46-8c44-2003b1d64eac None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:17.872511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f292982-b104-4d46-8c44-2003b1d64eac None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:17.872511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f292982-b104-4d46-8c44-2003b1d64eac None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:17.873788 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:17.879071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da1c29-5c0f-427d-8187-4dd64d81f485 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:17.879496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da1c29-5c0f-427d-8187-4dd64d81f485 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:17.879612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da1c29-5c0f-427d-8187-4dd64d81f485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:17.946469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-52da1c29-5c0f-427d-8187-4dd64d81f485 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:17.947704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-52da1c29-5c0f-427d-8187-4dd64d81f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:17.996827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:18.045832 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:18.387092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f292982-b104-4d46-8c44-2003b1d64eac None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:18.388652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7f292982-b104-4d46-8c44-2003b1d64eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:18.492740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f292982-b104-4d46-8c44-2003b1d64eac None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:18.541066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:18.567158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd326656-25f0-484f-9ec2-f0167d8bc3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:18.568415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd326656-25f0-484f-9ec2-f0167d8bc3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:18.568700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd326656-25f0-484f-9ec2-f0167d8bc3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:18.568968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd326656-25f0-484f-9ec2-f0167d8bc3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:18.570662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fd326656-25f0-484f-9ec2-f0167d8bc3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:18.680425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd326656-25f0-484f-9ec2-f0167d8bc3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:18.681886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd326656-25f0-484f-9ec2-f0167d8bc3a1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:18.700349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 732/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:28.044350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=22 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:28.698038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=66 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:43.790358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9d79-90bb-4102-aa94-9df2a552d2f1 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:43.790965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9d79-90bb-4102-aa94-9df2a552d2f1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:43.790965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9d79-90bb-4102-aa94-9df2a552d2f1 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:43.792230 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 739/1471] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:43.797343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8853-7fa5-44f4-b8ae-e689fd7804d0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:43.797813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8853-7fa5-44f4-b8ae-e689fd7804d0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:43.797945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8853-7fa5-44f4-b8ae-e689fd7804d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:43.873521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c05f8853-7fa5-44f4-b8ae-e689fd7804d0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:43.874789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c05f8853-7fa5-44f4-b8ae-e689fd7804d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:43.997670 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 733/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:44.402514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591789-7e12-43b0-9d5d-616fab095621 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:44.402791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591789-7e12-43b0-9d5d-616fab095621 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:44.403062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591789-7e12-43b0-9d5d-616fab095621 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:44.404420 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 740/1473] 10.222.0.33 () {54 vars in 966 bytes} [Sat Sep 10 20:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:44.410283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a445b28-dcce-4a7e-9c92-9ecc2315ec88 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:44.412024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a445b28-dcce-4a7e-9c92-9ecc2315ec88 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:44.412363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a445b28-dcce-4a7e-9c92-9ecc2315ec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:44.505922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0a445b28-dcce-4a7e-9c92-9ecc2315ec88 None None] MFA Rules not processed for user `5cdff5f5500040439f4acafa6c9ad948`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:44.507488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a445b28-dcce-4a7e-9c92-9ecc2315ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:44.531203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:44.600799 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 734/1474] 10.222.0.33 () {58 vars in 1063 bytes} [Sat Sep 10 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:44.608810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2591789-7e12-43b0-9d5d-616fab095621 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:44.610384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d2591789-7e12-43b0-9d5d-616fab095621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:44.713449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2591789-7e12-43b0-9d5d-616fab095621 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:44.788154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742c377-5999-4522-a9b2-bf1a77987371 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:44.788754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742c377-5999-4522-a9b2-bf1a77987371 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:44.789039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742c377-5999-4522-a9b2-bf1a77987371 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:44.789312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742c377-5999-4522-a9b2-bf1a77987371 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:44.790971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d742c377-5999-4522-a9b2-bf1a77987371 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:44.845306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:44.899020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742c377-5999-4522-a9b2-bf1a77987371 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:44.900452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742c377-5999-4522-a9b2-bf1a77987371 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:44.919609 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 741/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:47.430129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5512704-70ca-4c2c-8f3a-f74dd906ccc7 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:47.430713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5512704-70ca-4c2c-8f3a-f74dd906ccc7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:47.430713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5512704-70ca-4c2c-8f3a-f74dd906ccc7 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:47.432022 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 735/1476] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:47.437237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e829e0-d122-4bdf-8c43-5b8591f1e7c6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:47.437667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e829e0-d122-4bdf-8c43-5b8591f1e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:47.437764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e829e0-d122-4bdf-8c43-5b8591f1e7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:47.504283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a7e829e0-d122-4bdf-8c43-5b8591f1e7c6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:47.505516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7e829e0-d122-4bdf-8c43-5b8591f1e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:47.599651 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 742/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:47.929735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5512704-70ca-4c2c-8f3a-f74dd906ccc7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:47.931338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b5512704-70ca-4c2c-8f3a-f74dd906ccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:48.063904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5512704-70ca-4c2c-8f3a-f74dd906ccc7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:48.116602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:48.164512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a4e6b-cf30-43ac-b1f9-39c0a7e77e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:48.165126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a4e6b-cf30-43ac-b1f9-39c0a7e77e9b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:48.165401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a4e6b-cf30-43ac-b1f9-39c0a7e77e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:48.165671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a4e6b-cf30-43ac-b1f9-39c0a7e77e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:48.166330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a4e6b-cf30-43ac-b1f9-39c0a7e77e9b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:48.168063 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-282a4e6b-cf30-43ac-b1f9-39c0a7e77e9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:40:48.168063 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:40:48.168063 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:40:48.169204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a4e6b-cf30-43ac-b1f9-39c0a7e77e9b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:48.188871 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 736/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Sat Sep 10 20:40:47 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:40:50.624216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ecb1-f966-4087-a960-6a6c21fa24d2 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:50.624746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ecb1-f966-4087-a960-6a6c21fa24d2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:50.624746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581ecb1-f966-4087-a960-6a6c21fa24d2 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:50.626119 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 743/1479] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:50.632137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a83928-c6cd-441a-9546-b2dcef528ca0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:50.632543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a83928-c6cd-441a-9546-b2dcef528ca0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:50.632685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a83928-c6cd-441a-9546-b2dcef528ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:50.699133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-31a83928-c6cd-441a-9546-b2dcef528ca0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:50.700349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-31a83928-c6cd-441a-9546-b2dcef528ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:50.796513 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 737/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:51.156061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581ecb1-f966-4087-a960-6a6c21fa24d2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:51.157721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c581ecb1-f966-4087-a960-6a6c21fa24d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:51.167448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:51.303111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581ecb1-f966-4087-a960-6a6c21fa24d2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:51.380818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:51.414296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b936989-9dff-4bbe-b3ba-31aa281ee1e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:51.414890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b936989-9dff-4bbe-b3ba-31aa281ee1e7 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:51.415163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b936989-9dff-4bbe-b3ba-31aa281ee1e7 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:51.415429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b936989-9dff-4bbe-b3ba-31aa281ee1e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:51.416991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7b936989-9dff-4bbe-b3ba-31aa281ee1e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:51.508864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b936989-9dff-4bbe-b3ba-31aa281ee1e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:51.510501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b936989-9dff-4bbe-b3ba-31aa281ee1e7 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:51.528254 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 744/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:54.102821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a29b7-a9c8-4abb-89f8-d0c9a283a954 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:54.103403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a29b7-a9c8-4abb-89f8-d0c9a283a954 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:54.103403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a29b7-a9c8-4abb-89f8-d0c9a283a954 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:54.104668 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 738/1482] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:54.109814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf393e0-ce3c-49f2-8f24-efdfa421dc78 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:54.110262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf393e0-ce3c-49f2-8f24-efdfa421dc78 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:54.110389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf393e0-ce3c-49f2-8f24-efdfa421dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:54.176381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8cf393e0-ce3c-49f2-8f24-efdfa421dc78 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:54.177631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8cf393e0-ce3c-49f2-8f24-efdfa421dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:54.222762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:54.274407 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 745/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:54.624018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a29b7-a9c8-4abb-89f8-d0c9a283a954 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:54.625543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-025a29b7-a9c8-4abb-89f8-d0c9a283a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:54.650131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:54.730776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a29b7-a9c8-4abb-89f8-d0c9a283a954 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:54.805415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a02fe-4b68-4109-b795-d0a62a350ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:54.806036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a02fe-4b68-4109-b795-d0a62a350ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:54.806318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a02fe-4b68-4109-b795-d0a62a350ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:54.806586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a02fe-4b68-4109-b795-d0a62a350ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:54.808139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-911a02fe-4b68-4109-b795-d0a62a350ca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:54.884302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:54.918622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a02fe-4b68-4109-b795-d0a62a350ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:54.920064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a02fe-4b68-4109-b795-d0a62a350ca3 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:54.938178 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 739/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:40:57.424369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f90c8-d905-4885-b927-ddefddacd6f0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:57.424933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f90c8-d905-4885-b927-ddefddacd6f0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:57.424933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f90c8-d905-4885-b927-ddefddacd6f0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:57.426884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 746/1485] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:40:57.432007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151f92b-cda6-4e2c-a16d-4926ea9c5227 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:57.432414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151f92b-cda6-4e2c-a16d-4926ea9c5227 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:57.432558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151f92b-cda6-4e2c-a16d-4926ea9c5227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:57.499082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3151f92b-cda6-4e2c-a16d-4926ea9c5227 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:40:57.500296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3151f92b-cda6-4e2c-a16d-4926ea9c5227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:57.595688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 740/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:40:57.943727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f90c8-d905-4885-b927-ddefddacd6f0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:40:57.945267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-917f90c8-d905-4885-b927-ddefddacd6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:58.051656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f90c8-d905-4885-b927-ddefddacd6f0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:40:58.124308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:40:58.125925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc95f6a-1474-4cc8-b1f6-a485d129f208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:40:58.126504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc95f6a-1474-4cc8-b1f6-a485d129f208 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:40:58.127008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc95f6a-1474-4cc8-b1f6-a485d129f208 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:40:58.127460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc95f6a-1474-4cc8-b1f6-a485d129f208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:40:58.130168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7cc95f6a-1474-4cc8-b1f6-a485d129f208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:40:58.238401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc95f6a-1474-4cc8-b1f6-a485d129f208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:40:58.240009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc95f6a-1474-4cc8-b1f6-a485d129f208 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:40:58.258506 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 747/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:41:00.741705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f441d8-e0ff-4873-bca3-b77138f9c919 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:41:00.742407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f441d8-e0ff-4873-bca3-b77138f9c919 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:41:00.742407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f441d8-e0ff-4873-bca3-b77138f9c919 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:41:00.743588 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 741/1488] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:41:00.748820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058471-2d8b-4562-a8b2-821201cd3c49 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:41:00.749248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058471-2d8b-4562-a8b2-821201cd3c49 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:41:00.749459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058471-2d8b-4562-a8b2-821201cd3c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:41:00.815036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b4058471-2d8b-4562-a8b2-821201cd3c49 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:41:00.816416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b4058471-2d8b-4562-a8b2-821201cd3c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:41:00.867743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:41:00.911273 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 748/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:41:01.258598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f441d8-e0ff-4873-bca3-b77138f9c919 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:41:01.260180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-33f441d8-e0ff-4873-bca3-b77138f9c919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:41:01.296061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:41:01.365242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f441d8-e0ff-4873-bca3-b77138f9c919 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:41:01.438914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e4966-36d9-4b44-8725-e8129f1d98a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:41:01.439521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e4966-36d9-4b44-8725-e8129f1d98a0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:41:01.439801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e4966-36d9-4b44-8725-e8129f1d98a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:41:01.440077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e4966-36d9-4b44-8725-e8129f1d98a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:41:01.441671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-607e4966-36d9-4b44-8725-e8129f1d98a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:41:01.521318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:41:01.551575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e4966-36d9-4b44-8725-e8129f1d98a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:41:01.553087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e4966-36d9-4b44-8725-e8129f1d98a0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:41:01.571348 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 742/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:41:04.187392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34e0ba-a7bf-4e0d-b402-807e98515cf0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:41:04.187987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34e0ba-a7bf-4e0d-b402-807e98515cf0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:41:04.187987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34e0ba-a7bf-4e0d-b402-807e98515cf0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:41:04.189307 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 749/1491] 10.222.0.33 () {54 vars in 927 bytes} [Sat Sep 10 20:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:41:04.194764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76402ff3-8936-47bd-8cbb-04d27cb0629f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:41:04.195204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76402ff3-8936-47bd-8cbb-04d27cb0629f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:41:04.195314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76402ff3-8936-47bd-8cbb-04d27cb0629f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:41:04.262234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-76402ff3-8936-47bd-8cbb-04d27cb0629f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:41:04.263365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76402ff3-8936-47bd-8cbb-04d27cb0629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:41:04.362619 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 743/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Sat Sep 10 20:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:41:04.708443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34e0ba-a7bf-4e0d-b402-807e98515cf0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:41:04.710016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be34e0ba-a7bf-4e0d-b402-807e98515cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:41:04.812236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34e0ba-a7bf-4e0d-b402-807e98515cf0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:41:04.886140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56646050-9501-47c6-b50b-d05784af4de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:41:04.886741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646050-9501-47c6-b50b-d05784af4de6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:41:04.887021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646050-9501-47c6-b50b-d05784af4de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:41:04.887296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56646050-9501-47c6-b50b-d05784af4de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:41:04.888886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-56646050-9501-47c6-b50b-d05784af4de6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:41:04.897820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:41:04.997231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56646050-9501-47c6-b50b-d05784af4de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:41:04.998714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56646050-9501-47c6-b50b-d05784af4de6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:41:05.017248 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 750/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:41:14.361009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:41:15.015032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=50 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:42:19.737409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:19.738006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:19.738006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:19.813722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:42:19.815569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:19.914474 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 744/1494] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:42:19.922696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef9b2a-8080-403e-ac7b-c9c7f402d433 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:19.923205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef9b2a-8080-403e-ac7b-c9c7f402d433 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:19.923316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef9b2a-8080-403e-ac7b-c9c7f402d433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.000349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-dfef9b2a-8080-403e-ac7b-c9c7f402d433 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:42:20.002071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dfef9b2a-8080-403e-ac7b-c9c7f402d433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:20.103028 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 751/1495] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:42:20.107792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:20.108994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:20.121233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc895a-26b2-4fee-938e-45c5afb5ca7b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:20.186126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:42:20.224285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:20.224886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:20.225167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:20.225443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.226108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:42:20.226415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:20.227413 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:20.227413 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:42:20.227413 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:42:20.228305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:20.240722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 745/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:42:20.253215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef9b2a-8080-403e-ac7b-c9c7f402d433 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:20.260313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef9b2a-8080-403e-ac7b-c9c7f402d433 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:20.326933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:42:20.360377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:20.361539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:20.361932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:20.362210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.362847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:20.364578 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:20.364578 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 20:42:20.364578 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 20:42:20.365763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:20.420402 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 752/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:42:20 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:42:20.425413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:20.427967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:20.463614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04d0b3-f899-426d-91c0-bf1218a31276 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:20.468504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:20.469101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:20.469382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:20.469656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.470912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:20.471916 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:20.471916 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:42:20.471916 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:42:20.473134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:20.494665 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 746/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:42:20 2022] POST /identity/v3/users => generated 290 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:42:20.498975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:20.500557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:20.563586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e59249-91f4-413d-87cb-e318fa3adb18 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:20.646116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:42:20.658370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:20.659183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:20.659463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:20.659730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.660363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:20.661389 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:20.661389 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 20:42:20.661389 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 20:42:20.662301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:20.675387 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 753/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:42:20.679620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:20.689450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a8cfc-37b1-4f9b-8600-9a717742d89c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:20.694030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37f3c8-b0e7-4a5a-847f-c525742bf272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:20.694622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37f3c8-b0e7-4a5a-847f-c525742bf272 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:20.694902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37f3c8-b0e7-4a5a-847f-c525742bf272 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:20.695176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37f3c8-b0e7-4a5a-847f-c525742bf272 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.695806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37f3c8-b0e7-4a5a-847f-c525742bf272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:20.696837 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-fb37f3c8-b0e7-4a5a-847f-c525742bf272 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:20.696837 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:42:20.696837 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:42:20.697729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37f3c8-b0e7-4a5a-847f-c525742bf272 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:20.710551 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 747/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:42:20.714592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:20.716377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:20.753079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37552750-bd6d-498e-a76a-e0bad881f3ce admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:20.849602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763f5d0-842a-4a91-908c-36d86cac8622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:20.850218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763f5d0-842a-4a91-908c-36d86cac8622 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:20.850541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763f5d0-842a-4a91-908c-36d86cac8622 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:20.850814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763f5d0-842a-4a91-908c-36d86cac8622 admin admin] PATH_INFO: `/v3/projects/f696306fc39c461a9ae7894fc6adc282/users/732e01a57149468382ed631a7d2a71fa/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.873790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763f5d0-842a-4a91-908c-36d86cac8622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f696306fc39c461a9ae7894fc6adc282, user_id=732e01a57149468382ed631a7d2a71fa, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:20.874845 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-4763f5d0-842a-4a91-908c-36d86cac8622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:20.874845 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 20:42:20.874845 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 20:42:20.876817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763f5d0-842a-4a91-908c-36d86cac8622 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:20.901781 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 754/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:42:20 2022] PUT /identity/v3/projects/f696306fc39c461a9ae7894fc6adc282/users/732e01a57149468382ed631a7d2a71fa/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:42:20.914518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36cabe-9e4a-49aa-a795-1a896561613d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:20.915021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36cabe-9e4a-49aa-a795-1a896561613d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:20.915129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36cabe-9e4a-49aa-a795-1a896561613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:20.963210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5d36cabe-9e4a-49aa-a795-1a896561613d None None] MFA Rules not processed for user `732e01a57149468382ed631a7d2a71fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:42:20.964405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5d36cabe-9e4a-49aa-a795-1a896561613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:21.031091 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 748/1502] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:42:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:42:21.045043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b957d9-073c-4c79-b915-b5a296afe2d7 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:21.045462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b957d9-073c-4c79-b915-b5a296afe2d7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:21.045595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b957d9-073c-4c79-b915-b5a296afe2d7 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:21.047149 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 755/1503] 10.222.0.33 () {54 vars in 964 bytes} [Sat Sep 10 20:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:42:21.052199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38516-fa77-4b93-8908-d7f8333097e5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:21.052519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38516-fa77-4b93-8908-d7f8333097e5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:21.052796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38516-fa77-4b93-8908-d7f8333097e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:21.108346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:42:21.122565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a4d38516-fa77-4b93-8908-d7f8333097e5 None None] MFA Rules not processed for user `5fe5d3512475454080c8528db2208241`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:42:21.123783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a4d38516-fa77-4b93-8908-d7f8333097e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:21.198748 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 749/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Sat Sep 10 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:42:21.207393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b957d9-073c-4c79-b915-b5a296afe2d7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:21.218462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b957d9-073c-4c79-b915-b5a296afe2d7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:21.222819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:21.223405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:21.223680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:21.223948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:21.231911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:21.233328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:21.242301 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 756/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:42:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:42:21.288375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3474874-4340-4c78-91b6-fb022b393519 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:21.288667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3474874-4340-4c78-91b6-fb022b393519 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:21.288942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3474874-4340-4c78-91b6-fb022b393519 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:21.290320 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 750/1506] 10.222.0.33 () {52 vars in 830 bytes} [Sat Sep 10 20:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:42:21.294050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:21.327653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:42:21.356642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03446-0c6e-4162-b970-5c61832a608c service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:21.361884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:21.362474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:21.362754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:21.363025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] PATH_INFO: `/v3/users/732e01a57149468382ed631a7d2a71fa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:21.364142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=732e01a57149468382ed631a7d2a71fa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:21.365167 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:21.365167 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 20:42:21.365167 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 20:42:21.366409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:21.433352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732e01a57149468382ed631a7d2a71fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:21.447689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732e01a57149468382ed631a7d2a71fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:21.448535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732e01a57149468382ed631a7d2a71fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:21.458126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732e01a57149468382ed631a7d2a71fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:21.458852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732e01a57149468382ed631a7d2a71fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:21.490443 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 757/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:42:21 2022] DELETE /identity/v3/users/732e01a57149468382ed631a7d2a71fa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:42:21.494844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:21.495181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:21.495429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:21.522836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:42:21.523971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:21.582508 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 751/1508] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:42:21.592263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:21.593704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:21.671456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8c852-09d5-4766-9e44-51f4418aded1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:21.675145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ce5bc-d284-481e-ab55-f6d4d79cbdee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:21.675751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ce5bc-d284-481e-ab55-f6d4d79cbdee service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:21.676034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ce5bc-d284-481e-ab55-f6d4d79cbdee service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:21.676307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ce5bc-d284-481e-ab55-f6d4d79cbdee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:21.685203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ce5bc-d284-481e-ab55-f6d4d79cbdee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:21.686662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ce5bc-d284-481e-ab55-f6d4d79cbdee service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:21.696001 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 758/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:42:22.058066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:42:22.059461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:42:22.071709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bf415-50c3-4a86-a3f0-71f01236af59 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:42:22.076893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:42:22.077513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:42:22.077818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:42:22.078012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] PATH_INFO: `/v3/projects/f696306fc39c461a9ae7894fc6adc282` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:42:22.079137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f696306fc39c461a9ae7894fc6adc282)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:42:22.080939 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:42:22.080939 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:42:22.080939 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:42:22.081960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:42:22.146570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f696306fc39c461a9ae7894fc6adc282', 'request_id': 'req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:22.152530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f696306fc39c461a9ae7894fc6adc282', 'request_id': 'req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:22.165704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f696306fc39c461a9ae7894fc6adc282', 'request_id': 'req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:22.180818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] The token cache is being invalidate because project f696306fc39c461a9ae7894fc6adc282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:42:22.181109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:22.181843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a45d66f-a65d-40af-a7bd-2a3c5fc4738b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:42:22.183378 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 752/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:42:22 2022] DELETE /identity/v3/projects/f696306fc39c461a9ae7894fc6adc282 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:42:31.691890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=73,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:42:32.177590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:42.746178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292afad4-a34e-4fe3-a068-b6d83144a19a None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.746737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292afad4-a34e-4fe3-a068-b6d83144a19a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.746737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292afad4-a34e-4fe3-a068-b6d83144a19a None None] PATH_INFO: `/v3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.747915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 759/1511] 10.222.0.17 () {54 vars in 925 bytes} [Sat Sep 10 20:45:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:42.754804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc5b7-cb3c-4920-bfda-64cf124544fd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:42.755222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc5b7-cb3c-4920-bfda-64cf124544fd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:42.755351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc5b7-cb3c-4920-bfda-64cf124544fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:42.804609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-aa8cc5b7-cb3c-4920-bfda-64cf124544fd None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:42.805813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aa8cc5b7-cb3c-4920-bfda-64cf124544fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:42.877089 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 753/1512] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Sep 10 20:45:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:45:46.828916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3d70b-cd44-47ae-9eb4-a0c0abf01c55 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:46.829369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3d70b-cd44-47ae-9eb4-a0c0abf01c55 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:46.829477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3d70b-cd44-47ae-9eb4-a0c0abf01c55 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:46.830850 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 760/1513] 10.222.0.33 () {54 vars in 961 bytes} [Sat Sep 10 20:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:45:46.836257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20047f9e-3cce-4f24-b5ac-4ee2b6134c9c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:46.836689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20047f9e-3cce-4f24-b5ac-4ee2b6134c9c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:46.836817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20047f9e-3cce-4f24-b5ac-4ee2b6134c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:46.856792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-20047f9e-3cce-4f24-b5ac-4ee2b6134c9c None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:46.858020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20047f9e-3cce-4f24-b5ac-4ee2b6134c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:46.867072 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 754/1514] 10.222.0.33 () {58 vars in 1058 bytes} [Sat Sep 10 20:45:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:45:46.875337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3d70b-cd44-47ae-9eb4-a0c0abf01c55 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:45:46.888692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3d70b-cd44-47ae-9eb4-a0c0abf01c55 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:45:46.893723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec82ea-d8d0-474a-a896-d9558db828eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:45:46.894358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec82ea-d8d0-474a-a896-d9558db828eb service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:46.894635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec82ea-d8d0-474a-a896-d9558db828eb service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:46.894910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec82ea-d8d0-474a-a896-d9558db828eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:46.903883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec82ea-d8d0-474a-a896-d9558db828eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:45:46.905455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec82ea-d8d0-474a-a896-d9558db828eb service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:45:46.915537 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 761/1515] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:45:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:45:56.851985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=47 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:56.910980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:45:58.608687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ec734-2562-4a33-9b57-87bfc472e579 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:58.609277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ec734-2562-4a33-9b57-87bfc472e579 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:58.609277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ec734-2562-4a33-9b57-87bfc472e579 None None] PATH_INFO: `/v3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:58.610525 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 755/1516] 10.222.0.26 () {54 vars in 925 bytes} [Sat Sep 10 20:45:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:45:58.616672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e9833-e8aa-45a5-8ea6-ff462b3ef41d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:45:58.617124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e9833-e8aa-45a5-8ea6-ff462b3ef41d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:45:58.617231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e9833-e8aa-45a5-8ea6-ff462b3ef41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:45:58.637270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5a0e9833-e8aa-45a5-8ea6-ff462b3ef41d None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:45:58.638555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a0e9833-e8aa-45a5-8ea6-ff462b3ef41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:45:58.647074 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 762/1517] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 10 20:45:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:46:07.564382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb0f8b-1fa7-4a59-94b6-3914a9af1c1d None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.564797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb0f8b-1fa7-4a59-94b6-3914a9af1c1d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.564955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb0f8b-1fa7-4a59-94b6-3914a9af1c1d None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.566346 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 756/1518] 10.222.0.33 () {54 vars in 961 bytes} [Sat Sep 10 20:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:07.571976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9149-0b0a-4168-aee4-2d10a1e6747e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.572440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9149-0b0a-4168-aee4-2d10a1e6747e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.572557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9149-0b0a-4168-aee4-2d10a1e6747e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.591551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6e8b9149-0b0a-4168-aee4-2d10a1e6747e None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:07.592752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6e8b9149-0b0a-4168-aee4-2d10a1e6747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:07.602790 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 763/1519] 10.222.0.33 () {58 vars in 1058 bytes} [Sat Sep 10 20:46:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:46:07.609987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb0f8b-1fa7-4a59-94b6-3914a9af1c1d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:07.624334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb0f8b-1fa7-4a59-94b6-3914a9af1c1d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:07.629221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b90d7-938d-4a9e-bf19-7decae509221 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:07.629803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b90d7-938d-4a9e-bf19-7decae509221 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:07.630115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b90d7-938d-4a9e-bf19-7decae509221 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:07.630386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b90d7-938d-4a9e-bf19-7decae509221 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:07.638673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b90d7-938d-4a9e-bf19-7decae509221 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:07.640238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b90d7-938d-4a9e-bf19-7decae509221 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:07.650013 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 757/1520] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:46:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:46:17.586236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:17.645945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:25.314691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093fe35-d592-4c04-a965-35049ab4a3c5 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:25.315298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093fe35-d592-4c04-a965-35049ab4a3c5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:25.315298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093fe35-d592-4c04-a965-35049ab4a3c5 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:25.316736 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 764/1521] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:25.321487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ca3a3-92cc-47c7-b6e8-90ebbb674649 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:25.321915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ca3a3-92cc-47c7-b6e8-90ebbb674649 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:25.322067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ca3a3-92cc-47c7-b6e8-90ebbb674649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:25.360540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-846ca3a3-92cc-47c7-b6e8-90ebbb674649 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:25.362623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-846ca3a3-92cc-47c7-b6e8-90ebbb674649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:25.420106 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 758/1522] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:46:25.584353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093fe35-d592-4c04-a965-35049ab4a3c5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:25.586175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c093fe35-d592-4c04-a965-35049ab4a3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:25.690841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093fe35-d592-4c04-a965-35049ab4a3c5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:25.696472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cba948-fe17-489f-a24f-b6926c450abc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:25.697076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cba948-fe17-489f-a24f-b6926c450abc service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:25.697353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cba948-fe17-489f-a24f-b6926c450abc service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:25.697622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cba948-fe17-489f-a24f-b6926c450abc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:25.706572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cba948-fe17-489f-a24f-b6926c450abc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:25.708146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cba948-fe17-489f-a24f-b6926c450abc service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:25.717352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 765/1523] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:46:27.367348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13829cc3-0737-454e-90d5-5c61889245cc None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.367922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13829cc3-0737-454e-90d5-5c61889245cc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.368126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13829cc3-0737-454e-90d5-5c61889245cc None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.369495 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 759/1524] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 20:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:27.374419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9198f-0549-433a-8c40-44a04fba1a01 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.374820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9198f-0549-433a-8c40-44a04fba1a01 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.374978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9198f-0549-433a-8c40-44a04fba1a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.394671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d5e9198f-0549-433a-8c40-44a04fba1a01 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:27.395859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d5e9198f-0549-433a-8c40-44a04fba1a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:27.403634 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 766/1525] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 20:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:46:27.565221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13829cc3-0737-454e-90d5-5c61889245cc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:27.583286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13829cc3-0737-454e-90d5-5c61889245cc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:27.588733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279b8aa-cdcf-4fa8-8e05-40d670aa6478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:27.589327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279b8aa-cdcf-4fa8-8e05-40d670aa6478 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:27.589607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279b8aa-cdcf-4fa8-8e05-40d670aa6478 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:27.589927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279b8aa-cdcf-4fa8-8e05-40d670aa6478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:27.599081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279b8aa-cdcf-4fa8-8e05-40d670aa6478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:27.600660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279b8aa-cdcf-4fa8-8e05-40d670aa6478 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:27.609924 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 760/1526] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:46:31.877248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb91629-e24b-4514-b04c-342d4365f1f6 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.878000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb91629-e24b-4514-b04c-342d4365f1f6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.878000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb91629-e24b-4514-b04c-342d4365f1f6 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.879333 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 767/1527] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:31.884793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645fd2b-e92a-4470-9cc1-86c4a676de2e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:31.885195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645fd2b-e92a-4470-9cc1-86c4a676de2e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:31.885351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645fd2b-e92a-4470-9cc1-86c4a676de2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:31.906366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2645fd2b-e92a-4470-9cc1-86c4a676de2e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:31.907569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2645fd2b-e92a-4470-9cc1-86c4a676de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:31.915434 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 761/1528] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:46:32.267225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb91629-e24b-4514-b04c-342d4365f1f6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:32.269064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2cb91629-e24b-4514-b04c-342d4365f1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:32.326867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb91629-e24b-4514-b04c-342d4365f1f6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:32.330673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20bbbf-17a0-4375-a811-397519df0cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:32.331276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20bbbf-17a0-4375-a811-397519df0cdd service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:32.331556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20bbbf-17a0-4375-a811-397519df0cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:32.331834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20bbbf-17a0-4375-a811-397519df0cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:32.340748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20bbbf-17a0-4375-a811-397519df0cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:32.342221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20bbbf-17a0-4375-a811-397519df0cdd service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:32.352326 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 768/1529] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:46:34.756649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962db55b-d167-453e-96fd-a25f25d0e901 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.757173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962db55b-d167-453e-96fd-a25f25d0e901 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.757173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962db55b-d167-453e-96fd-a25f25d0e901 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.758458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 762/1530] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 20:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:46:34.763609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12cd6-7065-4a80-90c1-c59bbfe9249c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:34.764053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12cd6-7065-4a80-90c1-c59bbfe9249c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:34.764152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e12cd6-7065-4a80-90c1-c59bbfe9249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:34.784044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-74e12cd6-7065-4a80-90c1-c59bbfe9249c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:46:34.785251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74e12cd6-7065-4a80-90c1-c59bbfe9249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:46:34.793145 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 769/1531] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 20:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:46:35.139954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962db55b-d167-453e-96fd-a25f25d0e901 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:46:35.150847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962db55b-d167-453e-96fd-a25f25d0e901 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:46:35.154353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872bd5f-260c-4e39-a26f-260b87b3f748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:46:35.154945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872bd5f-260c-4e39-a26f-260b87b3f748 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:46:35.155222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872bd5f-260c-4e39-a26f-260b87b3f748 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:46:35.155493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872bd5f-260c-4e39-a26f-260b87b3f748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:46:35.164794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872bd5f-260c-4e39-a26f-260b87b3f748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:46:35.166268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872bd5f-260c-4e39-a26f-260b87b3f748 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:46:35.176340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 763/1532] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:46:44.778822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=78 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:46:45.171363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=49 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:47:26.982577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e39a9-700d-4c52-a1e9-91ce3dba64d8 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.983089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e39a9-700d-4c52-a1e9-91ce3dba64d8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.983307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e39a9-700d-4c52-a1e9-91ce3dba64d8 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:26.984734 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 770/1533] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:47:26.990324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0332028-2425-4cd0-9547-4a722dad71b0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:47:26.990728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0332028-2425-4cd0-9547-4a722dad71b0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:47:26.990875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0332028-2425-4cd0-9547-4a722dad71b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:47:27.010894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d0332028-2425-4cd0-9547-4a722dad71b0 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:47:27.012517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d0332028-2425-4cd0-9547-4a722dad71b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:47:27.021147 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 764/1534] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:47:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:47:37.005668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:00.987924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c9fd-49ca-4e10-921c-71a56df32180 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:00.988478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c9fd-49ca-4e10-921c-71a56df32180 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:00.988478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5c9fd-49ca-4e10-921c-71a56df32180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.031652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f8a5c9fd-49ca-4e10-921c-71a56df32180 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.033282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f8a5c9fd-49ca-4e10-921c-71a56df32180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.060903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 771/1535] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.072005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec222d0-785d-48dd-b842-f08b2d43553e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.073240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec222d0-785d-48dd-b842-f08b2d43553e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.073780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec222d0-785d-48dd-b842-f08b2d43553e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.109090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-eec222d0-785d-48dd-b842-f08b2d43553e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.110339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eec222d0-785d-48dd-b842-f08b2d43553e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.125302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 765/1536] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.130165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c9fd-49ca-4e10-921c-71a56df32180 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.149112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5c9fd-49ca-4e10-921c-71a56df32180 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.161410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.162479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.163058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.163664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.165580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:01.166248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.167533 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:48:01.167533 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 20:48:01.167533 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 20:48:01.168788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.191329 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 772/1537] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:01.199789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec222d0-785d-48dd-b842-f08b2d43553e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.214800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec222d0-785d-48dd-b842-f08b2d43553e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.221935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.222551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.222828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.223099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.223742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.224902 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:48:01.224902 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:48:01.224902 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:48:01.230329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.293798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 766/1538] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:01.303039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.320360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d747a9-95b7-455a-8f7a-5ed550cd7908 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.408935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.409965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.410575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.411060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.411888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.413094 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:48:01.413094 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 20:48:01.413094 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 20:48:01.414662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.449404 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 773/1539] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:01.453774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.472224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df511da8-e629-4a23-b668-e35d87086084 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.484261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7215a4-9a86-47bc-bca7-0a02984fbb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.484890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7215a4-9a86-47bc-bca7-0a02984fbb82 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.485171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7215a4-9a86-47bc-bca7-0a02984fbb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.485443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7215a4-9a86-47bc-bca7-0a02984fbb82 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.486326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7215a4-9a86-47bc-bca7-0a02984fbb82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.487727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7215a4-9a86-47bc-bca7-0a02984fbb82 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.511721 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 767/1540] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:01.515463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.532212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ea1fe-df1a-4679-b755-59a068154682 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.532902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcf5ce-424b-4eff-8dc5-a1020c651ef7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.533189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcf5ce-424b-4eff-8dc5-a1020c651ef7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.533456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcf5ce-424b-4eff-8dc5-a1020c651ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.546443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c540c-5715-461c-9bba-7a940da7fe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.547396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c540c-5715-461c-9bba-7a940da7fe55 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.547878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c540c-5715-461c-9bba-7a940da7fe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.548429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c540c-5715-461c-9bba-7a940da7fe55 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.549290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c540c-5715-461c-9bba-7a940da7fe55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.551147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c540c-5715-461c-9bba-7a940da7fe55 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.564567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c0bcf5ce-424b-4eff-8dc5-a1020c651ef7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.565776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c0bcf5ce-424b-4eff-8dc5-a1020c651ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.569352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 774/1541] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:01.570880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e03184-5496-4384-aff0-ad346ab2c2a0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.572066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e03184-5496-4384-aff0-ad346ab2c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.572549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e03184-5496-4384-aff0-ad346ab2c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.585467 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 768/1542] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:48:01.586322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcf5ce-424b-4eff-8dc5-a1020c651ef7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.599929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcf5ce-424b-4eff-8dc5-a1020c651ef7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.606889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba04b4d-aa1b-41e9-8161-f7bc2465ee15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.609860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba04b4d-aa1b-41e9-8161-f7bc2465ee15 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.609860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba04b4d-aa1b-41e9-8161-f7bc2465ee15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.609860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba04b4d-aa1b-41e9-8161-f7bc2465ee15 admin admin] PATH_INFO: `/v3/projects/5f6c275f77df48e196553dbc20db8ed9/users/736fcd7811e341468d7080da18b011ea/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.611429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-41e03184-5496-4384-aff0-ad346ab2c2a0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.613102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-41e03184-5496-4384-aff0-ad346ab2c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.629883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba04b4d-aa1b-41e9-8161-f7bc2465ee15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6c275f77df48e196553dbc20db8ed9, user_id=736fcd7811e341468d7080da18b011ea, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.631008 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-0ba04b4d-aa1b-41e9-8161-f7bc2465ee15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:48:01.631008 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:48:01.631008 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:48:01.632506 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 775/1543] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.634865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e7f9-aed1-4b0e-a59b-1200eb402773 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.635392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e7f9-aed1-4b0e-a59b-1200eb402773 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.635864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e7f9-aed1-4b0e-a59b-1200eb402773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.638347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba04b4d-aa1b-41e9-8161-f7bc2465ee15 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.661305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 769/1544] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:01 2022] PUT /identity/v3/projects/5f6c275f77df48e196553dbc20db8ed9/users/736fcd7811e341468d7080da18b011ea/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:01.661305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562aa49-23ea-4410-94ed-6cbbd5ac412c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.661305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562aa49-23ea-4410-94ed-6cbbd5ac412c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.661305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562aa49-23ea-4410-94ed-6cbbd5ac412c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.670915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6c49e7f9-aed1-4b0e-a59b-1200eb402773 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.672626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6c49e7f9-aed1-4b0e-a59b-1200eb402773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.682553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d562aa49-23ea-4410-94ed-6cbbd5ac412c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.683787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d562aa49-23ea-4410-94ed-6cbbd5ac412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.769854 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 770/1545] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.770610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b49978-a1e9-4983-96c9-b4f333c711b3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.770893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b49978-a1e9-4983-96c9-b4f333c711b3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.771165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b49978-a1e9-4983-96c9-b4f333c711b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.772508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 776/1546] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.774234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c441fe3-64b2-4f9e-90d2-c2aca38ed2f9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.774814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c441fe3-64b2-4f9e-90d2-c2aca38ed2f9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.775304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c441fe3-64b2-4f9e-90d2-c2aca38ed2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.802893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-32b49978-a1e9-4983-96c9-b4f333c711b3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.804264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-32b49978-a1e9-4983-96c9-b4f333c711b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.806843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4c441fe3-64b2-4f9e-90d2-c2aca38ed2f9 None None] MFA Rules not processed for user `736fcd7811e341468d7080da18b011ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.808304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4c441fe3-64b2-4f9e-90d2-c2aca38ed2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.813754 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 771/1547] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.815511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447ccc7-ebb9-4d93-bbc9-5b5351e63cf5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.816006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447ccc7-ebb9-4d93-bbc9-5b5351e63cf5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.816487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447ccc7-ebb9-4d93-bbc9-5b5351e63cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.838423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1447ccc7-ebb9-4d93-bbc9-5b5351e63cf5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.839790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1447ccc7-ebb9-4d93-bbc9-5b5351e63cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.853631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 772/1548] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.862338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27448ce-fc99-4444-b399-3df41dde04f2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.862640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27448ce-fc99-4444-b399-3df41dde04f2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.863735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27448ce-fc99-4444-b399-3df41dde04f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.872746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:01.886892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f27448ce-fc99-4444-b399-3df41dde04f2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.888103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f27448ce-fc99-4444-b399-3df41dde04f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.902475 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 773/1549] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.902475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe0c60-2b34-4823-b2cf-a6cbbee1fae6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.902475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe0c60-2b34-4823-b2cf-a6cbbee1fae6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.902475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe0c60-2b34-4823-b2cf-a6cbbee1fae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.909855 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 777/1550] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.912085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ce6f-9004-4415-8a2d-30400bcd0fcb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.912379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ce6f-9004-4415-8a2d-30400bcd0fcb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.912659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ce6f-9004-4415-8a2d-30400bcd0fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.922757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-74fe0c60-2b34-4823-b2cf-a6cbbee1fae6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.924090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-74fe0c60-2b34-4823-b2cf-a6cbbee1fae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.932892 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 774/1551] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.933399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2068ce6f-9004-4415-8a2d-30400bcd0fcb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.933855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe0c60-2b34-4823-b2cf-a6cbbee1fae6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.934690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2068ce6f-9004-4415-8a2d-30400bcd0fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.942891 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 778/1552] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.944030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704122d6-fa5e-4fe1-80d4-947ac35221c6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.944314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704122d6-fa5e-4fe1-80d4-947ac35221c6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.944586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704122d6-fa5e-4fe1-80d4-947ac35221c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.950880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe0c60-2b34-4823-b2cf-a6cbbee1fae6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.955942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.956541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.956829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.957125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.957774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:01.958112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.959722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:01.964350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-704122d6-fa5e-4fe1-80d4-947ac35221c6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.965548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-704122d6-fa5e-4fe1-80d4-947ac35221c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.980695 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 775/1553] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:01.980695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:01.981536 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 779/1554] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:01.981536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f37447-f544-49f7-aacf-3daff00fa2bf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.981536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f37447-f544-49f7-aacf-3daff00fa2bf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.981536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f37447-f544-49f7-aacf-3daff00fa2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.988897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12da66-27a1-471d-bcc4-eb271f0e0efb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:01.994751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:01.995362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:01.995648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:01.995924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:01.996566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:01.996844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a6f37447-f544-49f7-aacf-3daff00fa2bf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:01.997285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:01.998063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a6f37447-f544-49f7-aacf-3daff00fa2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:01.998508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.005917 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 780/1555] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:02.007101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdd1dd-cb86-4ed0-884a-13f789d97609 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.007384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdd1dd-cb86-4ed0-884a-13f789d97609 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.007659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdd1dd-cb86-4ed0-884a-13f789d97609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.012428 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 776/1556] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:02.013471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.023441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b02087-a1ea-4483-81f0-b1aa4de14a50 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.028139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24efc10-da7b-4c27-98ef-e2aafc586e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.028726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc10-da7b-4c27-98ef-e2aafc586e27 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.029004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc10-da7b-4c27-98ef-e2aafc586e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.029275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc10-da7b-4c27-98ef-e2aafc586e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.029926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24efc10-da7b-4c27-98ef-e2aafc586e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.030707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-36bdd1dd-cb86-4ed0-884a-13f789d97609 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:02.031849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24efc10-da7b-4c27-98ef-e2aafc586e27 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.031946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36bdd1dd-cb86-4ed0-884a-13f789d97609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:02.040398 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 781/1557] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:02.041475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396d7b8-e9d4-40f6-85c3-708ba85c8bd4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.041759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396d7b8-e9d4-40f6-85c3-708ba85c8bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.042066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396d7b8-e9d4-40f6-85c3-708ba85c8bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.062136 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 777/1558] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.063524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461ce37-b928-48b9-9c4b-ce21bbf5da66 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.063883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461ce37-b928-48b9-9c4b-ce21bbf5da66 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.064228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461ce37-b928-48b9-9c4b-ce21bbf5da66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.066082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d396d7b8-e9d4-40f6-85c3-708ba85c8bd4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:02.067330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d396d7b8-e9d4-40f6-85c3-708ba85c8bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:02.084404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:02.088380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a461ce37-b928-48b9-9c4b-ce21bbf5da66 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:02.090829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a461ce37-b928-48b9-9c4b-ce21bbf5da66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:02.132407 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 782/1559] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:02.133199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396d7b8-e9d4-40f6-85c3-708ba85c8bd4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.142865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396d7b8-e9d4-40f6-85c3-708ba85c8bd4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.147492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.148081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.148369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.148646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.149815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:02.150152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.151751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.154140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 778/1560] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:02.154723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461ce37-b928-48b9-9c4b-ce21bbf5da66 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.163867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461ce37-b928-48b9-9c4b-ce21bbf5da66 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.168155 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 783/1561] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:02.169042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.175132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.175889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.176238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.176662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.177250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.179191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.184608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fed91-5465-4401-8d2c-73f04ff76be3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.189315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.189945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.190237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.190515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.191153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:02.191456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.193047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.206729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 784/1562] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:02.207529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.209260 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 779/1563] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.210301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.218028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a76a3d-6989-4bc5-9cf9-4dddf9f436aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.275778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.276310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.276590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.276865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.277141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226b4f9-5555-412e-8dca-1cc135b9541c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.277672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.279626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.282065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.282671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.282954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.283231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.283881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:02.284184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.285762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.298699 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 780/1564] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:02.299518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.307509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 785/1565] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.308400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.310240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72915c24-4d42-49e6-b244-a0dd6fbaca7a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.327966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:02.368239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.368954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.369236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.369511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.370170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.372078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.374689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d60cd-2baa-4562-9df9-986faec09503 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.379266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.379876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.380183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.380467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.381111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:02.381529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.383311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.393065 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 781/1566] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.396127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.396223 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 786/1567] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:02.396735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.405672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49ff6-99ca-4bd8-9ecd-43daea554a89 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.409353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbc02c-e8b9-4721-a4f0-8dcb203b66e8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.410634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.411229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.411512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.411791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.412431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:02.412740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.414094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.414373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.414705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.415581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.416026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.416878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.418005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:02.418859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.428236 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 782/1568] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:02.431395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.442319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc95ffd-fdc6-4e99-b40f-5a608e4b51ea admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.447079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.447677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.447967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.448245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.451651 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 787/1569] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.451790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.451885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.451972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.472561 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 783/1570] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 300 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.473355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.484882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6210a86-ee03-41a2-b452-7b6068e408fc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.524245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90eff79-b92e-4e27-8f70-161835084d0a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.529627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.530220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.530504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.530779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.531407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.533446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.544790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.545389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.545665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.546005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.546631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.548508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.560835 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 788/1571] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.561636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.585600 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 784/1572] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.586631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.631866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68734a-374d-47ba-b870-6f72915aaf22 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.649934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:02.657646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4f7e5-3cd4-4511-ad1d-e6af833eb175 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.663351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.664027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.664304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.664574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.666040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.667865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.678248 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 785/1573] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:02.682957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.692190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.692694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20fd679-b326-42d6-b9e2-793d6f31f37a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.692823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.693069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.693442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.694296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.696556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.698309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.698898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.699178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.699449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.700062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.701957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.719436 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 789/1574] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.720247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.731143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8321b0e-7ec5-43d8-8b0d-d6dd10fb0bb6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.736821 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 786/1575] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.736821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.739218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.739820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.740108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.740382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.741008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.743013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.775777 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 790/1576] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.776833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.781244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:02.803802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e08f3-9aca-4912-96b0-f6c902e62dca admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.844952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa2fc8-5903-4ee0-b7a0-b8f67b1365fb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.849734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.850377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.850664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.850946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.851593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.853237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.862365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.862966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.863151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 791/1577] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:02.863592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.863928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.864028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.864668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.866646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.874262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236a10d-157a-4453-8963-4977836ffa22 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.878905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.879489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.879775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.880129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.880939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.883143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.889850 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 787/1578] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.890744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.902380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a33fe-73c5-4005-a735-e22495e34a86 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.906694 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 792/1579] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 300 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.907493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.907634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.908233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.908509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.908776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.909391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.911466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.917642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f7347-30b9-4913-b9dd-7ded17f39461 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.922481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.923128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.923463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.923688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] PATH_INFO: `/v3/projects/c8dfd36f32b9446d9fcb8551c253e3c9/users/1e75f8b01c4a4939b7f8c2086e3974e5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.933920 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 788/1580] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 285 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:02.934768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.940225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dfd36f32b9446d9fcb8551c253e3c9, user_id=1e75f8b01c4a4939b7f8c2086e3974e5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.943006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.944313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b975-53ec-4532-bc25-24d095b3a5af admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:02.949654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:02.950247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:02.950528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:02.950800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:02.951446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:02.953045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:02.956716 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 793/1581] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:02 2022] PUT /identity/v3/projects/c8dfd36f32b9446d9fcb8551c253e3c9/users/1e75f8b01c4a4939b7f8c2086e3974e5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:02.957530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.964927 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 789/1582] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:02.965676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:02.975833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf1bbc-d57d-4c18-89f3-aa83a08cd252 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.009471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.028069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25a2e9-607c-459a-986c-896eb91dcdd2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.032869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.033420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.033703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.034018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.034645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.036582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.036968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.037589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.037927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.038202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.038872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.041547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.063063 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 794/1583] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:03.063063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.067798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 790/1584] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:02 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:03.067798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.072495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0902768-5500-4d7f-bcf6-3a25d3b640f8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.076120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0d303-6c71-468c-a48f-18ce4691ef00 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.077204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.077799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.078141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.078422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.079062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.080693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.080805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.081415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.081656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.081973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.082615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.084187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.093355 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 795/1585] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.094968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 791/1586] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.095797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.095987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.105950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa2821-5101-4bf6-83ae-1fff6c0f8db2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.108271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e484685-5c4b-4945-9eda-a1cfb83256b6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.111133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.111725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.112002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.112271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.112922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.113026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.113536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.113851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.114219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.114774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.114872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.116396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.126790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 792/1587] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.130416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 796/1588] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.130552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.130640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.139819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e6bda-8590-4fc9-b4fe-3cda0ec15370 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.140565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ea7ac-4565-429f-85fd-508cdf055338 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.144482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.145053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.145336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.145614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.145755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.146283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.146441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.146727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.146997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.147624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.147881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.149272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.159253 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 793/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.163236 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 797/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.163369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.163474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.167095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.171051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89a346-d746-4d5e-9c08-77b25931eadf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.172390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb208b-5838-479e-9227-66427e65726b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.176370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.176965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.177251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.177526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.177819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.178198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.178440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.178723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.178999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.179630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.179781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.182253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.191361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 798/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.192308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.193322 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 794/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.194313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.206311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7182e90-71e0-4af4-8005-80efe684f620 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.207163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab7f47-7673-4da8-a205-b82c0b264239 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.211403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.212061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.212249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.212356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.212783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.212923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.213208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.213430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.213525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] PATH_INFO: `/v3/projects/64caaddd046d4282a6c5dccf553eb5ea/users/ea55e45cd1b94a41baed5bb35ce324f8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.215181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.227057 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 799/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.228180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.229208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64caaddd046d4282a6c5dccf553eb5ea, user_id=ea55e45cd1b94a41baed5bb35ce324f8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.232271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.238124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1be89-af01-443e-9c66-6466646b7dff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.243709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.244507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.244799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.245075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.245708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.247313 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 795/1594] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/64caaddd046d4282a6c5dccf553eb5ea/users/ea55e45cd1b94a41baed5bb35ce324f8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.247419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.248122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.267444 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 800/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.268012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.277878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04cd35-ee75-4645-85a7-9ac0fd7112ea admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.313255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2fd50-d6dc-4619-806c-5729fe6c1316 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.318476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.319082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.319362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.319637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.320289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.321928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.332007 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 796/1596] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.333128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.336237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.337061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.337383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.337802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] PATH_INFO: `/v3/projects/c8dfd36f32b9446d9fcb8551c253e3c9/users/1e75f8b01c4a4939b7f8c2086e3974e5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.339958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dfd36f32b9446d9fcb8551c253e3c9, user_id=1e75f8b01c4a4939b7f8c2086e3974e5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.342944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.343462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1135f8-8fd8-4d7c-8225-5db46464684e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.348372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.349005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.349283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.349555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] PATH_INFO: `/v3/projects/ccebd57cffee4ae7971faf7ca24b94d1/users/3aff800a17b14cb7afcad8c532511246/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.354954 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 801/1597] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/c8dfd36f32b9446d9fcb8551c253e3c9/users/1e75f8b01c4a4939b7f8c2086e3974e5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.355770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.372557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccebd57cffee4ae7971faf7ca24b94d1, user_id=3aff800a17b14cb7afcad8c532511246, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.375253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.388158 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 797/1598] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/ccebd57cffee4ae7971faf7ca24b94d1/users/3aff800a17b14cb7afcad8c532511246/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.389172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.437078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0b355-b793-478e-8d2d-3681cb6b4e1a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.440837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.451960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85651b-1e86-4ccf-b9f3-7ad67ffa110d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.456861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.457471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.457752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.458057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.458698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.460283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.470725 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 798/1599] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.471319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.481136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e2695-5e9a-4fa8-90d4-7291ef7edfca admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.485729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.486452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.486733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.487010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] PATH_INFO: `/v3/projects/b887509639cb4e708a501572e5c6aee8/users/ae0c0bfa95c94ed7a8b31e8a9715675d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.497473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83ffc7-d4d3-42d1-ac61-fe63b0b4876a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.498136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83ffc7-d4d3-42d1-ac61-fe63b0b4876a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.498419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83ffc7-d4d3-42d1-ac61-fe63b0b4876a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.498695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83ffc7-d4d3-42d1-ac61-fe63b0b4876a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.499328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83ffc7-d4d3-42d1-ac61-fe63b0b4876a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.500935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83ffc7-d4d3-42d1-ac61-fe63b0b4876a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.502449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b887509639cb4e708a501572e5c6aee8, user_id=ae0c0bfa95c94ed7a8b31e8a9715675d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.505152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.511976 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 802/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:03.513699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108127b-8c33-466c-b9f5-b79ef917c7ad None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.514009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108127b-8c33-466c-b9f5-b79ef917c7ad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.514286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108127b-8c33-466c-b9f5-b79ef917c7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.518722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 799/1601] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/b887509639cb4e708a501572e5c6aee8/users/ae0c0bfa95c94ed7a8b31e8a9715675d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.519659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.534258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.550518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9108127b-8c33-466c-b9f5-b79ef917c7ad None None] MFA Rules not processed for user `ea55e45cd1b94a41baed5bb35ce324f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:03.551751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9108127b-8c33-466c-b9f5-b79ef917c7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:03.588541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe8ad2-eb0e-40d0-9c26-54b0fc225d04 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.593495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90bb0-e786-466f-88eb-2795d693f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.594460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90bb0-e786-466f-88eb-2795d693f11b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.594745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90bb0-e786-466f-88eb-2795d693f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.595022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90bb0-e786-466f-88eb-2795d693f11b admin admin] PATH_INFO: `/v3/projects/2bc1fda0e9d945c1b8e9e20c8ab20879/users/d00f0f3d675246158f02f2dcf4271d2b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.612223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90bb0-e786-466f-88eb-2795d693f11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc1fda0e9d945c1b8e9e20c8ab20879, user_id=d00f0f3d675246158f02f2dcf4271d2b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.615344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90bb0-e786-466f-88eb-2795d693f11b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.620398 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 803/1602] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:03.621309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108127b-8c33-466c-b9f5-b79ef917c7ad None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.631223 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 800/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/2bc1fda0e9d945c1b8e9e20c8ab20879/users/d00f0f3d675246158f02f2dcf4271d2b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.632205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108127b-8c33-466c-b9f5-b79ef917c7ad None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.632864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c296b3f-a0a6-414e-9e5e-1e68283c3a53 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.633311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c296b3f-a0a6-414e-9e5e-1e68283c3a53 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.633662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c296b3f-a0a6-414e-9e5e-1e68283c3a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.672735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6c296b3f-a0a6-414e-9e5e-1e68283c3a53 None None] MFA Rules not processed for user `1e75f8b01c4a4939b7f8c2086e3974e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:03.674019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6c296b3f-a0a6-414e-9e5e-1e68283c3a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:03.695595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb935a9-fb48-465e-86b1-693cf6e274da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.696541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb935a9-fb48-465e-86b1-693cf6e274da admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.696991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb935a9-fb48-465e-86b1-693cf6e274da admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.697279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb935a9-fb48-465e-86b1-693cf6e274da admin admin] PATH_INFO: `/v3/projects/d44361bde7c94fb6b01d992a8df306bb/users/a020e015186b4c32950c2d4edd26374e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.711423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb935a9-fb48-465e-86b1-693cf6e274da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44361bde7c94fb6b01d992a8df306bb, user_id=a020e015186b4c32950c2d4edd26374e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.714887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb935a9-fb48-465e-86b1-693cf6e274da admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.729793 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 804/1604] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/d44361bde7c94fb6b01d992a8df306bb/users/a020e015186b4c32950c2d4edd26374e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.731079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd93089-4833-4964-bf0d-529416613ab0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.731382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd93089-4833-4964-bf0d-529416613ab0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.731639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd93089-4833-4964-bf0d-529416613ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.744932 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 801/1605] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:03.745586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c296b3f-a0a6-414e-9e5e-1e68283c3a53 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.755530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c296b3f-a0a6-414e-9e5e-1e68283c3a53 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.764939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0fd93089-4833-4964-bf0d-529416613ab0 None None] MFA Rules not processed for user `3aff800a17b14cb7afcad8c532511246`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:03.766513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0fd93089-4833-4964-bf0d-529416613ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:03.771566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.817020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8be308-c3c9-45f9-8f4f-5fc6c54063eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.817751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8be308-c3c9-45f9-8f4f-5fc6c54063eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.818149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8be308-c3c9-45f9-8f4f-5fc6c54063eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.818435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8be308-c3c9-45f9-8f4f-5fc6c54063eb admin admin] PATH_INFO: `/v3/projects/01ab39c389f643d5b49f37fba29d20c1/users/7aa64fd0fcb34afd89097d1932f6eca6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.832786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8be308-c3c9-45f9-8f4f-5fc6c54063eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ab39c389f643d5b49f37fba29d20c1, user_id=7aa64fd0fcb34afd89097d1932f6eca6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.835518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8be308-c3c9-45f9-8f4f-5fc6c54063eb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.838481 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 805/1606] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:03.839438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd93089-4833-4964-bf0d-529416613ab0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.850674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd93089-4833-4964-bf0d-529416613ab0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:03.854151 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 802/1607] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/01ab39c389f643d5b49f37fba29d20c1/users/7aa64fd0fcb34afd89097d1932f6eca6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.859159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d23554-2168-494e-8a78-3ea8b501aada None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.859448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d23554-2168-494e-8a78-3ea8b501aada None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.859846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d23554-2168-494e-8a78-3ea8b501aada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.867071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:03.888934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a2d23554-2168-494e-8a78-3ea8b501aada None None] MFA Rules not processed for user `ae0c0bfa95c94ed7a8b31e8a9715675d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:03.890184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a2d23554-2168-494e-8a78-3ea8b501aada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:03.910665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:03.911400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.911676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.911965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] PATH_INFO: `/v3/projects/e4fab9176a7f4a40bde3bdf0e69bc335/users/2b6af05818814f3db521e4ebbf9d0e56/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.927545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fab9176a7f4a40bde3bdf0e69bc335, user_id=2b6af05818814f3db521e4ebbf9d0e56, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:03.930221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:03.944264 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 806/1608] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:03 2022] PUT /identity/v3/projects/e4fab9176a7f4a40bde3bdf0e69bc335/users/2b6af05818814f3db521e4ebbf9d0e56/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:03.950684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:03.960318 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 803/1609] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:03.960318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba715d2b-9c74-49a1-a78c-43aeb5982db1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:03.960318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba715d2b-9c74-49a1-a78c-43aeb5982db1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:03.960318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba715d2b-9c74-49a1-a78c-43aeb5982db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:03.990174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ba715d2b-9c74-49a1-a78c-43aeb5982db1 None None] MFA Rules not processed for user `d00f0f3d675246158f02f2dcf4271d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:03.991424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ba715d2b-9c74-49a1-a78c-43aeb5982db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:04.014159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99c74-5e6b-4757-a3cd-eb390bbaeb60 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.018998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013deaf-6bc3-47ca-a848-ad899b930530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.019601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013deaf-6bc3-47ca-a848-ad899b930530 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.019877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013deaf-6bc3-47ca-a848-ad899b930530 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.020145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013deaf-6bc3-47ca-a848-ad899b930530 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.020766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013deaf-6bc3-47ca-a848-ad899b930530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.022697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013deaf-6bc3-47ca-a848-ad899b930530 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.056759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 807/1610] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/projects => generated 397 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.058217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303bac-f5b6-4062-b236-8d8459009fb9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.058503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303bac-f5b6-4062-b236-8d8459009fb9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.058803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71303bac-f5b6-4062-b236-8d8459009fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.061572 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 804/1611] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:04.061572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a78a-200e-4f3f-87f9-71c102327fae None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.061572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a78a-200e-4f3f-87f9-71c102327fae None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.061572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a78a-200e-4f3f-87f9-71c102327fae None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.061572 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 805/1612] 10.222.0.33 () {54 vars in 962 bytes} [Sat Sep 10 20:48:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:04.066519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a78a-200e-4f3f-87f9-71c102327fae None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.078376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-71303bac-f5b6-4062-b236-8d8459009fb9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:04.079566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-71303bac-f5b6-4062-b236-8d8459009fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:04.089115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.134560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a78a-200e-4f3f-87f9-71c102327fae None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.139488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715ed85-3bc3-4a41-bf06-a21c627bdcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.140417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ed85-3bc3-4a41-bf06-a21c627bdcaa admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.140735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ed85-3bc3-4a41-bf06-a21c627bdcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.141017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ed85-3bc3-4a41-bf06-a21c627bdcaa admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.141677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715ed85-3bc3-4a41-bf06-a21c627bdcaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.143654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715ed85-3bc3-4a41-bf06-a21c627bdcaa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.144842 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 808/1613] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:04.146197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e18c2b-c50d-4b55-a053-fea8005a7f8d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.146554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e18c2b-c50d-4b55-a053-fea8005a7f8d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.146835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e18c2b-c50d-4b55-a053-fea8005a7f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.174331 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 806/1614] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.175568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a440f5-ff74-4130-af2a-44813c7a816d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.176047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a440f5-ff74-4130-af2a-44813c7a816d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.176565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a440f5-ff74-4130-af2a-44813c7a816d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.178586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c8e18c2b-c50d-4b55-a053-fea8005a7f8d None None] MFA Rules not processed for user `7aa64fd0fcb34afd89097d1932f6eca6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:04.179834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8e18c2b-c50d-4b55-a053-fea8005a7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:04.189804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.205103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-88a440f5-ff74-4130-af2a-44813c7a816d None None] MFA Rules not processed for user `2b6af05818814f3db521e4ebbf9d0e56`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:04.206329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-88a440f5-ff74-4130-af2a-44813c7a816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:04.247387 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 809/1615] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:04.248023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e18c2b-c50d-4b55-a053-fea8005a7f8d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.275870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 807/1616] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:04.275870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a440f5-ff74-4130-af2a-44813c7a816d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.312913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e18c2b-c50d-4b55-a053-fea8005a7f8d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.317773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.318422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.318709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.319061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.319698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.321664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.336180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a440f5-ff74-4130-af2a-44813c7a816d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.342653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf9b3-0545-41b4-9522-730d440726aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.343278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf9b3-0545-41b4-9522-730d440726aa admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.343565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf9b3-0545-41b4-9522-730d440726aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.343850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf9b3-0545-41b4-9522-730d440726aa admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.344731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf9b3-0545-41b4-9522-730d440726aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.346689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf9b3-0545-41b4-9522-730d440726aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.354483 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 810/1617] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.354483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.368648 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 808/1618] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/users => generated 305 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.370246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fac1b-c9f2-4543-bb2c-4a2160a58e7c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.370605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fac1b-c9f2-4543-bb2c-4a2160a58e7c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.370953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fac1b-c9f2-4543-bb2c-4a2160a58e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.386349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.411140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4a8fac1b-c9f2-4543-bb2c-4a2160a58e7c None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:04.412464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4a8fac1b-c9f2-4543-bb2c-4a2160a58e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:04.419845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea10c2-5000-4d18-b04b-5e2871160948 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.424523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.425127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.425407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.425678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.426342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.428242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.457363 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 811/1619] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.458215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.484641 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 809/1620] 10.222.0.33 () {58 vars in 1059 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:04.485536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fac1b-c9f2-4543-bb2c-4a2160a58e7c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.493008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.497653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fac1b-c9f2-4543-bb2c-4a2160a58e7c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.516453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3425d-b0a1-446a-8e6f-b4ffccaeed6e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.520271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.520873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.521154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.521430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.558387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.559029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.559313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.559718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.560387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.562847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.583946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.584678 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 810/1621] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.585520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.585606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.596541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c56da-6e57-4bdd-9412-463291692695 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.597429 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 812/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:04.598238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.601434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.602059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.602404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.603395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.604090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.606018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.608140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e580d-b10e-49b2-877d-0959ed330c3a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.612941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.613623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.614007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.614357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.615053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.617024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.636654 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 811/1623] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.637667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.648894 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 813/1624] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.648894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.699101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.710941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb858baa-1bfd-49d9-b04f-9d6c1b115ac3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.725790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84daaf-f38c-47fc-90db-2c217a0bc8f0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.731285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.732030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.732315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.732593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.733247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.734895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.746740 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 814/1625] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:04.747948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.759360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee8874-5110-4003-bf8e-dd4fadc1e5da admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.765026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.765819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.766340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.766718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.767542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.769819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.783732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.784420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.784717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.784994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.785899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.788196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.798990 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 815/1626] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.798990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.814190 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 812/1627] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.818417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.835097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6afa32-ff86-4173-8642-d937406ee348 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.842829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.844333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.844728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.845108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.845879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.847681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.850431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:04.862231 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 813/1628] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:04.863069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.873846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeae62d-9521-4597-90ac-3f92774eb8ce admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.879627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.880561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.880850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.881128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.881768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.884339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.898809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736777a-9e2c-4a24-97bb-065c5f8f7d5a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.906415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.907206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.907604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.908013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.912764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 814/1629] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.914443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.927248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e0516-c7b4-4074-9068-34bed5175802 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.933261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.933600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.933923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.934205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.935285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.938167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:04.964554 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 815/1630] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:04 2022] POST /identity/v3/users => generated 297 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:04.965295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:04.977071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2cc25-ee24-44ab-9c6c-fd86b8550e09 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:04.982417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:04.983165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:04.983519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:04.983892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] PATH_INFO: `/v3/projects/3a825f49753c4baaa6661e82f2c30df7/users/f794e66e441b4e489ea613b164ed6b29/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:04.988794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:04.990595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.001609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a825f49753c4baaa6661e82f2c30df7, user_id=f794e66e441b4e489ea613b164ed6b29, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.002695 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 816/1631] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:05.003429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.004739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.015998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc544-db9b-4123-8f36-32465eb95d67 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.022128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 816/1632] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:04 2022] PUT /identity/v3/projects/3a825f49753c4baaa6661e82f2c30df7/users/f794e66e441b4e489ea613b164ed6b29/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.023246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.084627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.101066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.101707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.102023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.102320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.103075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.109965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.115865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e083-70df-405d-abd6-0158abfef3d0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.124914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.125739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.126547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.126818 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 817/1633] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.127001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.127658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.127817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.129965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.140866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebbb70-4993-4d74-9dd0-f1c4fd81927d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.144442 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 817/1634] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.145369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.150412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.151235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.151533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.151948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] PATH_INFO: `/v3/projects/b237bd9a5ed447919289225cff98741a/users/e3793b9a5b7f4c55bd98a47495e4ad8b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.159672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411b79-823b-46c9-ae66-91dbdce84f3f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.171805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b237bd9a5ed447919289225cff98741a, user_id=e3793b9a5b7f4c55bd98a47495e4ad8b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.174739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.177854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.179740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.180091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.180938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.182701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.186720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.194810 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 818/1635] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:05 2022] PUT /identity/v3/projects/b237bd9a5ed447919289225cff98741a/users/e3793b9a5b7f4c55bd98a47495e4ad8b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.196756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.209177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 818/1636] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.210702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.224587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e1588-8188-48e5-a938-f5812688db5b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.284242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99428d1-0e22-4213-8e26-7fd4512c5afd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.291593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.292531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.292918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.293711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.295136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.299257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.303455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.304291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.304828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.305271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.306604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.308639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.315222 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 819/1637] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.316066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.329237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f00e0-cdeb-4224-985d-c6f62a944457 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.329951 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 819/1638] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.331171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.339004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.339825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.340203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.340579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] PATH_INFO: `/v3/projects/7a05f73817a247858ea70e46d0dd7a33/users/99e139d10b82472e9b46d49ea14aa35b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.342549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883adac0-3e8f-4791-bd6b-7eab8220191a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.353805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.354612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.354989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.355403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] PATH_INFO: `/v3/projects/05112a99b5054250bb739e5632cc77ff/users/89ed19cfebd84df6b1935e8fcf7dfa90/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.370360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a05f73817a247858ea70e46d0dd7a33, user_id=99e139d10b82472e9b46d49ea14aa35b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.374046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.378541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05112a99b5054250bb739e5632cc77ff, user_id=89ed19cfebd84df6b1935e8fcf7dfa90, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.381571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.385594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.395235 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 820/1639] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:05 2022] PUT /identity/v3/projects/7a05f73817a247858ea70e46d0dd7a33/users/99e139d10b82472e9b46d49ea14aa35b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.396330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.397328 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 820/1640] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:05 2022] PUT /identity/v3/projects/05112a99b5054250bb739e5632cc77ff/users/89ed19cfebd84df6b1935e8fcf7dfa90/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.401407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.472923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e800f-3429-4e8a-bae6-401b52d560c9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.473086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082d07f-9afb-475c-88b5-ef0f191ed71a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.481203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.481422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.482141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.482983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.483537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.484235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.484428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.485111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.486038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] PATH_INFO: `/v3/projects/8c184615b31044a88b36e83e5982beec/users/efb6ac4e1dfe40ac884705af3189d734/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.486851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.499315 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 821/1641] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.500282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.505932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c184615b31044a88b36e83e5982beec, user_id=efb6ac4e1dfe40ac884705af3189d734, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.509980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.515319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fd83c-3488-4db2-9606-56f6b2a50275 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.520643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.521670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.522505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.522792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] PATH_INFO: `/v3/projects/8b32eb78eef140bab4197fe048c5d68c/users/29b26ef474d041c6a4fcf55e0d17d223/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.527105 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 821/1642] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:05 2022] PUT /identity/v3/projects/8c184615b31044a88b36e83e5982beec/users/efb6ac4e1dfe40ac884705af3189d734/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.530065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.545399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b32eb78eef140bab4197fe048c5d68c, user_id=29b26ef474d041c6a4fcf55e0d17d223, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.548194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.587320 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 822/1643] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:05 2022] PUT /identity/v3/projects/8b32eb78eef140bab4197fe048c5d68c/users/29b26ef474d041c6a4fcf55e0d17d223/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.588582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.612780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.625138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94f42-5ccc-4ab2-a828-90c13dc62171 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.708901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.709695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.710266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.710697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] PATH_INFO: `/v3/projects/3a825f49753c4baaa6661e82f2c30df7/users/f794e66e441b4e489ea613b164ed6b29/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.715533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a825f49753c4baaa6661e82f2c30df7, user_id=f794e66e441b4e489ea613b164ed6b29, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.719564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.729266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea1353-3453-44a5-b667-f0d68de9861d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.744959 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 822/1644] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:05 2022] PUT /identity/v3/projects/3a825f49753c4baaa6661e82f2c30df7/users/f794e66e441b4e489ea613b164ed6b29/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.744959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.763958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:05.855863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.856637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.856929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.858190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.861007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.862657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ee5bb-f59e-4d00-b47e-e92ac4806e8c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.867249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.873474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.874211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.874560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.874909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.875959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.877312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.882804 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 823/1645] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.883861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.900669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 823/1646] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.901848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.920231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb40a9-1a0d-428e-b33a-6c8013ecb53a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.922620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd8b90-a38a-4020-a340-5b33857d6b5f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.925723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.926396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.926681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.926961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.927876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.929698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.936947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cc198-1161-45f8-a39e-d3a7d2d531b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.937635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cc198-1161-45f8-a39e-d3a7d2d531b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.937947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cc198-1161-45f8-a39e-d3a7d2d531b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.938226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cc198-1161-45f8-a39e-d3a7d2d531b9 admin admin] PATH_INFO: `/v3/projects/b237bd9a5ed447919289225cff98741a/users/e3793b9a5b7f4c55bd98a47495e4ad8b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.941570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cc198-1161-45f8-a39e-d3a7d2d531b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b237bd9a5ed447919289225cff98741a, user_id=e3793b9a5b7f4c55bd98a47495e4ad8b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.942479 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 824/1647] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:05.943405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:05.947405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cc198-1161-45f8-a39e-d3a7d2d531b9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:05.956688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001eb3c-5fd5-4db7-9643-4886290ac5c8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:05.979610 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 824/1648] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:05 2022] PUT /identity/v3/projects/b237bd9a5ed447919289225cff98741a/users/e3793b9a5b7f4c55bd98a47495e4ad8b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:05.982594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8d08-0024-48bd-9cb9-001bef462e73 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.983001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8d08-0024-48bd-9cb9-001bef462e73 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.983527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8d08-0024-48bd-9cb9-001bef462e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.989763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:05.990759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:05.991108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:05.991425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:05.992247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:05.994397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.010732 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 825/1649] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:06.011412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:06.023866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce9f18-9ab3-4a5a-86f1-10dce824e9ba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:06.055641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fddd8d08-0024-48bd-9cb9-001bef462e73 None None] MFA Rules not processed for user `f794e66e441b4e489ea613b164ed6b29`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.056912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fddd8d08-0024-48bd-9cb9-001bef462e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.112152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:06.113035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.113326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.113605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] PATH_INFO: `/v3/projects/7a05f73817a247858ea70e46d0dd7a33/users/99e139d10b82472e9b46d49ea14aa35b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.119057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a05f73817a247858ea70e46d0dd7a33, user_id=99e139d10b82472e9b46d49ea14aa35b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.121886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.143145 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 826/1650] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:06 2022] PUT /identity/v3/projects/7a05f73817a247858ea70e46d0dd7a33/users/99e139d10b82472e9b46d49ea14aa35b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:06.143145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:06.176276 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 825/1651] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:05 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:06.180259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd8d08-0024-48bd-9cb9-001bef462e73 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:06.190277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd8d08-0024-48bd-9cb9-001bef462e73 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:06.217902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:06.234370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b0ef7-ca0a-46dc-81a1-d7656f210ba4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:06.239966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa0d96-4db3-4bf1-bc9f-63a3a37da434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:06.240643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa0d96-4db3-4bf1-bc9f-63a3a37da434 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.240950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa0d96-4db3-4bf1-bc9f-63a3a37da434 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.241201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa0d96-4db3-4bf1-bc9f-63a3a37da434 admin admin] PATH_INFO: `/v3/projects/05112a99b5054250bb739e5632cc77ff/users/89ed19cfebd84df6b1935e8fcf7dfa90/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.243790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa0d96-4db3-4bf1-bc9f-63a3a37da434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05112a99b5054250bb739e5632cc77ff, user_id=89ed19cfebd84df6b1935e8fcf7dfa90, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.247388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa0d96-4db3-4bf1-bc9f-63a3a37da434 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.266045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 827/1652] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:06 2022] PUT /identity/v3/projects/05112a99b5054250bb739e5632cc77ff/users/89ed19cfebd84df6b1935e8fcf7dfa90/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:06.272501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:06.274194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.274979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.275484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] PATH_INFO: `/v3/projects/8c184615b31044a88b36e83e5982beec/users/efb6ac4e1dfe40ac884705af3189d734/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.277627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496ecd7-abf7-49d8-b734-39c24fba3636 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.277740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c184615b31044a88b36e83e5982beec, user_id=efb6ac4e1dfe40ac884705af3189d734, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.277892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496ecd7-abf7-49d8-b734-39c24fba3636 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.278107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496ecd7-abf7-49d8-b734-39c24fba3636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.280577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.294883 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 826/1653] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:06 2022] PUT /identity/v3/projects/8c184615b31044a88b36e83e5982beec/users/efb6ac4e1dfe40ac884705af3189d734/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:06.295990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:06.322722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5496ecd7-abf7-49d8-b734-39c24fba3636 None None] MFA Rules not processed for user `e3793b9a5b7f4c55bd98a47495e4ad8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.324301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5496ecd7-abf7-49d8-b734-39c24fba3636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.335461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:06.380520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9a7f-9b92-4321-b401-9f1c5cd33e89 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:06.389492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be95ad-f24f-4dfc-b397-ea2b37319d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:06.390509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be95ad-f24f-4dfc-b397-ea2b37319d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.390874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be95ad-f24f-4dfc-b397-ea2b37319d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.391495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be95ad-f24f-4dfc-b397-ea2b37319d36 admin admin] PATH_INFO: `/v3/projects/8b32eb78eef140bab4197fe048c5d68c/users/29b26ef474d041c6a4fcf55e0d17d223/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.394137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be95ad-f24f-4dfc-b397-ea2b37319d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b32eb78eef140bab4197fe048c5d68c, user_id=29b26ef474d041c6a4fcf55e0d17d223, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.397856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be95ad-f24f-4dfc-b397-ea2b37319d36 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.411677 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 827/1654] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:06 2022] PUT /identity/v3/projects/8b32eb78eef140bab4197fe048c5d68c/users/29b26ef474d041c6a4fcf55e0d17d223/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:06.414335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a9b9c-0b22-43a7-9b98-16244298d36b None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.414691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a9b9c-0b22-43a7-9b98-16244298d36b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.415069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a9b9c-0b22-43a7-9b98-16244298d36b None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.416851 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 828/1655] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:06.417957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a338a-04a9-4671-bc13-c4cce4d86e79 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.418309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a338a-04a9-4671-bc13-c4cce4d86e79 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.418657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a338a-04a9-4671-bc13-c4cce4d86e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.442298 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 828/1656] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:06.443721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496ecd7-abf7-49d8-b734-39c24fba3636 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:06.453924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1a5a338a-04a9-4671-bc13-c4cce4d86e79 None None] MFA Rules not processed for user `99e139d10b82472e9b46d49ea14aa35b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.455532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a5a338a-04a9-4671-bc13-c4cce4d86e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.555560 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 829/1657] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:06.556637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877a04-f21f-4aca-99a0-81651861ed5f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.556927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877a04-f21f-4aca-99a0-81651861ed5f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.557205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877a04-f21f-4aca-99a0-81651861ed5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.565381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496ecd7-abf7-49d8-b734-39c24fba3636 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:06.592284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-06877a04-f21f-4aca-99a0-81651861ed5f None None] MFA Rules not processed for user `89ed19cfebd84df6b1935e8fcf7dfa90`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.593605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-06877a04-f21f-4aca-99a0-81651861ed5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.645375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:06.665574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f3b14-45c2-4051-ae6b-63039c06e02a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:06.666479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3b14-45c2-4051-ae6b-63039c06e02a service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.666772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3b14-45c2-4051-ae6b-63039c06e02a service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.667052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3b14-45c2-4051-ae6b-63039c06e02a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.733442 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 830/1658] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:06.734931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0397018-83a1-4b09-8ce2-321ae91ef2bc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.735223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0397018-83a1-4b09-8ce2-321ae91ef2bc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.735501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0397018-83a1-4b09-8ce2-321ae91ef2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.740281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:06.768408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a0397018-83a1-4b09-8ce2-321ae91ef2bc None None] MFA Rules not processed for user `efb6ac4e1dfe40ac884705af3189d734`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.769563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f3b14-45c2-4051-ae6b-63039c06e02a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:06.769732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a0397018-83a1-4b09-8ce2-321ae91ef2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.771714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f3b14-45c2-4051-ae6b-63039c06e02a service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:06.787730 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 829/1659] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:06 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:06.789187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3578fd-3247-465e-a0d4-086b735eec94 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.789575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3578fd-3247-465e-a0d4-086b735eec94 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.790028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3578fd-3247-465e-a0d4-086b735eec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.830804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5b3578fd-3247-465e-a0d4-086b735eec94 None None] MFA Rules not processed for user `29b26ef474d041c6a4fcf55e0d17d223`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.832042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5b3578fd-3247-465e-a0d4-086b735eec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.877566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 831/1660] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:06.877566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7a606-d745-475c-80de-428975956216 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:06.877566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7a606-d745-475c-80de-428975956216 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:06.877566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7a606-d745-475c-80de-428975956216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:06.909263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ded7a606-d745-475c-80de-428975956216 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:06.910519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ded7a606-d745-475c-80de-428975956216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:06.970337 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 830/1661] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:06.975187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3578fd-3247-465e-a0d4-086b735eec94 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:06.989064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3578fd-3247-465e-a0d4-086b735eec94 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.000641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.001278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.001564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.001868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.038190 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 832/1662] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:07.064736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7a606-d745-475c-80de-428975956216 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.077982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7a606-d745-475c-80de-428975956216 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.083668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10d4e3-8b15-4388-aaa2-44113ab662ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.084293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d4e3-8b15-4388-aaa2-44113ab662ac service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.084579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d4e3-8b15-4388-aaa2-44113ab662ac service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.084852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d4e3-8b15-4388-aaa2-44113ab662ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.088527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.091696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.100901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10d4e3-8b15-4388-aaa2-44113ab662ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.103036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10d4e3-8b15-4388-aaa2-44113ab662ac service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.108183 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 831/1663] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:06 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:07.114989 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 833/1664] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:07.240159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.255290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765854d-7261-4d48-934f-7b4ede65d837 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.255658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765854d-7261-4d48-934f-7b4ede65d837 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.256387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765854d-7261-4d48-934f-7b4ede65d837 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.257644 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 834/1665] 10.222.0.33 () {54 vars in 962 bytes} [Sat Sep 10 20:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:07.263798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec56e1-ce16-40dd-9993-aa9ead6f1297 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.264042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d4969-c55e-4463-8fbb-d9571ef13e57 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.264397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d4969-c55e-4463-8fbb-d9571ef13e57 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.264685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d4969-c55e-4463-8fbb-d9571ef13e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.273386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0847f-5a0f-47a0-a17d-9c3682722f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.274289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0847f-5a0f-47a0-a17d-9c3682722f03 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.275129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0847f-5a0f-47a0-a17d-9c3682722f03 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.275414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0847f-5a0f-47a0-a17d-9c3682722f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.284689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0847f-5a0f-47a0-a17d-9c3682722f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.286314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0847f-5a0f-47a0-a17d-9c3682722f03 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.287331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b82d4969-c55e-4463-8fbb-d9571ef13e57 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:07.288921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b82d4969-c55e-4463-8fbb-d9571ef13e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:07.299508 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 832/1666] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:07.300344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924fc31-9193-4f7b-a120-f6a8f5a1461f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.300638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924fc31-9193-4f7b-a120-f6a8f5a1461f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.301045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924fc31-9193-4f7b-a120-f6a8f5a1461f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.318370 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 835/1667] 10.222.0.33 () {58 vars in 1059 bytes} [Sat Sep 10 20:48:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:48:07.321670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2924fc31-9193-4f7b-a120-f6a8f5a1461f None None] MFA Rules not processed for user `736fcd7811e341468d7080da18b011ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:07.322940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2924fc31-9193-4f7b-a120-f6a8f5a1461f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:07.327952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d4969-c55e-4463-8fbb-d9571ef13e57 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.342690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d4969-c55e-4463-8fbb-d9571ef13e57 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.351403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.352025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.353226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.353508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.374240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.375379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:07.377467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.389368 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 836/1668] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:07.390243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.393616 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 833/1669] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:07.402563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9eb6d-5606-4f31-8852-81e447fc3f22 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.408831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.409427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.409704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.410022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.419583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.421343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.427445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:07.435107 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 837/1670] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:07.453365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924fc31-9193-4f7b-a120-f6a8f5a1461f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.468553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924fc31-9193-4f7b-a120-f6a8f5a1461f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.474672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.475357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.475581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.475861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.485207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.486971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.506031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 834/1671] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:07.613918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.623102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.638193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c3877-bbf9-4656-b020-21de39079511 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.665624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.666275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.666816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.666816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.684117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.685743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.703275 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 835/1672] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:07.708597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.726045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b139f1-571f-4c5e-a751-4f14723e9260 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.741433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef50430-fd81-42d3-9515-8405604aea64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.742083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef50430-fd81-42d3-9515-8405604aea64 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.742364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef50430-fd81-42d3-9515-8405604aea64 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.742637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef50430-fd81-42d3-9515-8405604aea64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.755029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e647-3d1f-48eb-9a83-169f755d8271 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:07.774874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:07.775591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.775877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.776149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] PATH_INFO: `/v3/users/d00f0f3d675246158f02f2dcf4271d2b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.779135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d00f0f3d675246158f02f2dcf4271d2b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.781105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.866797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef50430-fd81-42d3-9515-8405604aea64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:07.868495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef50430-fd81-42d3-9515-8405604aea64 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:07.869661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd00f0f3d675246158f02f2dcf4271d2b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:07.878859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd00f0f3d675246158f02f2dcf4271d2b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:07.879432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd00f0f3d675246158f02f2dcf4271d2b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:07.880677 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 836/1673] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:07.884763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf339f6-654a-46a0-b551-d103f8e4802f None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.885876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf339f6-654a-46a0-b551-d103f8e4802f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.885876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf339f6-654a-46a0-b551-d103f8e4802f None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.888092 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 837/1674] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:07.894330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.894790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.895314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.909498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd00f0f3d675246158f02f2dcf4271d2b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:07.910386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4704c52f-8223-43a9-b01a-094fb87d1de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd00f0f3d675246158f02f2dcf4271d2b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:07.922980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:07.924731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:07.939090 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 838/1675] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:48:07.955448 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 838/1676] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:07 2022] DELETE /identity/v3/users/d00f0f3d675246158f02f2dcf4271d2b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:07.958849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:07.959932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:07.960230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:07.960503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:07.961232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:07.999474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:08.001342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.135016 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 839/1677] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:48:08.136000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.137545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.162109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c6e11-d420-4250-bd12-cbe4562dcbae None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.170904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d884fc-2030-47b6-9379-1f011e1623c4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.172598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.173270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.173539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.173813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] PATH_INFO: `/v3/users/2b6af05818814f3db521e4ebbf9d0e56` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.176853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6af05818814f3db521e4ebbf9d0e56)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:08.179032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:08.205133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.292381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6af05818814f3db521e4ebbf9d0e56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.299304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.300067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.300474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.300921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.305160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6af05818814f3db521e4ebbf9d0e56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.308391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.309911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6af05818814f3db521e4ebbf9d0e56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.324817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6af05818814f3db521e4ebbf9d0e56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.325725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6af05818814f3db521e4ebbf9d0e56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.361853 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 840/1678] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:08 2022] DELETE /identity/v3/users/2b6af05818814f3db521e4ebbf9d0e56 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:08.362869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.368606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.471659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:08.476983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:08.477361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291e4b4-c7f5-462b-be44-58a2e01a41d8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.484503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.485281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.485639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.486418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] PATH_INFO: `/v3/users/3aff800a17b14cb7afcad8c532511246` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.488561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aff800a17b14cb7afcad8c532511246)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:08.490957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:08.513403 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 839/1679] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:08.517741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.520627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.583771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aff800a17b14cb7afcad8c532511246'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.611962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aff800a17b14cb7afcad8c532511246'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.613038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aff800a17b14cb7afcad8c532511246'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.635097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aff800a17b14cb7afcad8c532511246'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.635907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91cec42c-3f54-406d-8877-f4156c5c6448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aff800a17b14cb7afcad8c532511246'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:08.675566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 841/1680] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:08 2022] DELETE /identity/v3/users/3aff800a17b14cb7afcad8c532511246 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:08.676849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1244c-fa71-437a-83a6-1c980d8b5e5a service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.679794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ddd2d-0d02-4d5f-9057-bb1636afe3dd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.680015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ddd2d-0d02-4d5f-9057-bb1636afe3dd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.681273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ddd2d-0d02-4d5f-9057-bb1636afe3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.737417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-096ddd2d-0d02-4d5f-9057-bb1636afe3dd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:08.738934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-096ddd2d-0d02-4d5f-9057-bb1636afe3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.753345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.760750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.761382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.761767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.762091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.767800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.871394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:08.873727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:08.878875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:08.888634 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 840/1681] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:08.889686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.890149 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 842/1682] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:08.891093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.891415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.891701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.891974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.906182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e519c80-6386-47f4-aef9-1da66dfb8e00 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:08.916254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:08.916960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:08.917416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:08.917897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] PATH_INFO: `/v3/users/7aa64fd0fcb34afd89097d1932f6eca6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:08.920391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa64fd0fcb34afd89097d1932f6eca6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:08.921761 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:48:08.921761 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 20:48:08.921761 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 20:48:08.924586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:08.937055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:08.938340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:08.974892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 843/1683] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:08.975834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:08.984333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:09.008318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b16d77-738c-4118-8f85-b7c7a9b637e6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.018364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.019000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.019282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.019558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.036176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa64fd0fcb34afd89097d1932f6eca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.045178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.047507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.072015 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 844/1684] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:09.073022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.081260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa64fd0fcb34afd89097d1932f6eca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.090970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4506007-f7ac-49e6-bb60-72d1903a1748 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.099685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa64fd0fcb34afd89097d1932f6eca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.105512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.106161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.106445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.106739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.115304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa64fd0fcb34afd89097d1932f6eca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.116586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4548f616-031a-49d4-9507-7f039dd07411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa64fd0fcb34afd89097d1932f6eca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.130314 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 841/1685] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:08 2022] DELETE /identity/v3/users/7aa64fd0fcb34afd89097d1932f6eca6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:09.130878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.132362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.132942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.134128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.134564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.153787 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 845/1686] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:09.174140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:09.175984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:09.262301 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 842/1687] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:09.283268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.284900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:09.302586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5552ad-301a-441a-8115-29dfbd4b1570 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.381491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.386847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.387215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.387472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.387923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.389547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:09.400386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.402119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.409516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660638a-08f4-4fd5-997f-3117e60b84d8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.422207 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 846/1688] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:09.428947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.429614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.429939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.430227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] PATH_INFO: `/v3/projects/2bc1fda0e9d945c1b8e9e20c8ab20879` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.435164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc1fda0e9d945c1b8e9e20c8ab20879)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.437123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.551418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc1fda0e9d945c1b8e9e20c8ab20879', 'request_id': 'req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.560120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc1fda0e9d945c1b8e9e20c8ab20879', 'request_id': 'req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.570001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc1fda0e9d945c1b8e9e20c8ab20879', 'request_id': 'req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.583166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.616504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] The token cache is being invalidate because project 2bc1fda0e9d945c1b8e9e20c8ab20879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:09.617360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.619393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.622381 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 843/1689] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:09 2022] DELETE /identity/v3/projects/2bc1fda0e9d945c1b8e9e20c8ab20879 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:09.626798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.641581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135029-f881-4feb-8444-4d14a1422f03 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.763761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:09.764349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.765121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.765244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] PATH_INFO: `/v3/users/99e139d10b82472e9b46d49ea14aa35b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.766553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e139d10b82472e9b46d49ea14aa35b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:09.768567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:09.800467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:09.834661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e139d10b82472e9b46d49ea14aa35b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.844933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e139d10b82472e9b46d49ea14aa35b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.846175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e139d10b82472e9b46d49ea14aa35b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.856602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e139d10b82472e9b46d49ea14aa35b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.866695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e81d06cf-ca65-4f49-8b64-4a621bdf891b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e139d10b82472e9b46d49ea14aa35b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:09.907564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:09.909264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:09.980189 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 847/1690] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:09 2022] DELETE /identity/v3/users/99e139d10b82472e9b46d49ea14aa35b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:09.982005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1b30a-f2e3-446e-ba37-8b426ace954f None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:09.984628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1b30a-f2e3-446e-ba37-8b426ace954f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:09.987478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1b30a-f2e3-446e-ba37-8b426ace954f None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:09.992320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c2139-5b18-4515-9ea7-1cf8a78c80bd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:09.998569 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 848/1691] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:09 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:10.005019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.005738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.007323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.094674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:10.113262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.191113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.191971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.193050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.194147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] PATH_INFO: `/v3/projects/e4fab9176a7f4a40bde3bdf0e69bc335` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.201354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fab9176a7f4a40bde3bdf0e69bc335)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.204432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.308394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 849/1692] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:10.316049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.317668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fab9176a7f4a40bde3bdf0e69bc335', 'request_id': 'req-b37de277-a4f4-46ff-9481-40a0aba8f1c0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.323448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.342813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fab9176a7f4a40bde3bdf0e69bc335', 'request_id': 'req-b37de277-a4f4-46ff-9481-40a0aba8f1c0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.357483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fab9176a7f4a40bde3bdf0e69bc335', 'request_id': 'req-b37de277-a4f4-46ff-9481-40a0aba8f1c0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.360839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29540c-0a43-4a9b-9a2b-90c37ee9d567 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.408998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] The token cache is being invalidate because project e4fab9176a7f4a40bde3bdf0e69bc335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:10.409820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.410887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.413172 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 844/1693] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:09 2022] DELETE /identity/v3/projects/e4fab9176a7f4a40bde3bdf0e69bc335 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:10.414182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.417953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.451717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.452477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.452821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.453160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] PATH_INFO: `/v3/projects/01ab39c389f643d5b49f37fba29d20c1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.454615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ab39c389f643d5b49f37fba29d20c1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.456205 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 20:48:10.456205 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 20:48:10.456205 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 20:48:10.457750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.512286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.544168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ab39c389f643d5b49f37fba29d20c1', 'request_id': 'req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.551181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de277-a4f4-46ff-9481-40a0aba8f1c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.567198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ab39c389f643d5b49f37fba29d20c1', 'request_id': 'req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.583750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ab39c389f643d5b49f37fba29d20c1', 'request_id': 'req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.595832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:10.610177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] The token cache is being invalidate because project 01ab39c389f643d5b49f37fba29d20c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:10.610914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.612291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:10.616219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 850/1694] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:10 2022] DELETE /identity/v3/projects/01ab39c389f643d5b49f37fba29d20c1 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:10.618654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.620304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.728700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8273fd-f1cb-4fa8-ac6f-29f8484fe9b6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.736734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.737925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.737925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.738333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.740360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.755477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.756745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.757291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.757718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.768987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.821876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.825287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.845083 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 845/1695] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:10.847249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.851256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.908208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:10.913144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:10.960054 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 851/1696] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:10.965083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:10.965245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:10.989220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bada0-0529-4de9-9d40-a024d9cdbed4 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:10.996065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:10.997188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:10.997573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:10.997997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] PATH_INFO: `/v3/projects/ccebd57cffee4ae7971faf7ca24b94d1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:10.999806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccebd57cffee4ae7971faf7ca24b94d1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:11.002031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:11.010685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61eefe-b74b-4411-b393-244afa021959 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:11.044695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:11.045949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.046272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.046457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] PATH_INFO: `/v3/users/29b26ef474d041c6a4fcf55e0d17d223` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.047817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b26ef474d041c6a4fcf55e0d17d223)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:11.057290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:11.113268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccebd57cffee4ae7971faf7ca24b94d1', 'request_id': 'req-811ac877-a185-4618-b888-6f64712cfc20'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.129278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccebd57cffee4ae7971faf7ca24b94d1', 'request_id': 'req-811ac877-a185-4618-b888-6f64712cfc20'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.145727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccebd57cffee4ae7971faf7ca24b94d1', 'request_id': 'req-811ac877-a185-4618-b888-6f64712cfc20'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.187435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] The token cache is being invalidate because project ccebd57cffee4ae7971faf7ca24b94d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:11.187755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.195415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.199965 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 846/1697] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:10 2022] DELETE /identity/v3/projects/ccebd57cffee4ae7971faf7ca24b94d1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:11.199965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:11.199965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.216288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b26ef474d041c6a4fcf55e0d17d223'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.238439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b26ef474d041c6a4fcf55e0d17d223'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.240397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b26ef474d041c6a4fcf55e0d17d223'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.243081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:11.258566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b26ef474d041c6a4fcf55e0d17d223'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.259719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b26ef474d041c6a4fcf55e0d17d223'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.299223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:11.327237 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 852/1698] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:10 2022] DELETE /identity/v3/users/29b26ef474d041c6a4fcf55e0d17d223 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:11.328201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:11.330103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.342701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ac877-a185-4618-b888-6f64712cfc20 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:11.455518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:11.456585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.457058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.457510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] PATH_INFO: `/v3/users/efb6ac4e1dfe40ac884705af3189d734` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.458531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35895ba9-383b-4b1b-9617-fd7ba8941cc5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:11.460767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb6ac4e1dfe40ac884705af3189d734)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:11.465326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:11.472813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:11.474321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.475062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.475165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] PATH_INFO: `/v3/projects/7a05f73817a247858ea70e46d0dd7a33` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.478573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a05f73817a247858ea70e46d0dd7a33)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:11.483023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:11.563879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb6ac4e1dfe40ac884705af3189d734'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.581044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb6ac4e1dfe40ac884705af3189d734'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.589084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a05f73817a247858ea70e46d0dd7a33', 'request_id': 'req-5d210641-418b-4a72-ab18-7d3b4f373f42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.589639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb6ac4e1dfe40ac884705af3189d734'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.600336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a05f73817a247858ea70e46d0dd7a33', 'request_id': 'req-5d210641-418b-4a72-ab18-7d3b4f373f42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.600539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb6ac4e1dfe40ac884705af3189d734'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.601700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb6ac4e1dfe40ac884705af3189d734'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.612639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a05f73817a247858ea70e46d0dd7a33', 'request_id': 'req-5d210641-418b-4a72-ab18-7d3b4f373f42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.614585 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 847/1699] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:11 2022] DELETE /identity/v3/users/efb6ac4e1dfe40ac884705af3189d734 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:11.617290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:11.634660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] The token cache is being invalidate because project 7a05f73817a247858ea70e46d0dd7a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:11.635915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.636195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.638518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d210641-418b-4a72-ab18-7d3b4f373f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.643117 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 853/1700] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:11 2022] DELETE /identity/v3/projects/7a05f73817a247858ea70e46d0dd7a33 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:11.646554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae28bb-dade-4ad0-ab80-12e3f7efd2cc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.647792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae28bb-dade-4ad0-ab80-12e3f7efd2cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.651291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae28bb-dade-4ad0-ab80-12e3f7efd2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.699229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-90ae28bb-dade-4ad0-ab80-12e3f7efd2cc None None] MFA Rules not processed for user `a020e015186b4c32950c2d4edd26374e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:11.700653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-90ae28bb-dade-4ad0-ab80-12e3f7efd2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.780390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:11.793217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 854/1701] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:11.794680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.795679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.796035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.805389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043449ad-4e73-4d8b-b3e3-ef1bc2a607c1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:11.818639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:11.819326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.819642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.819921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] PATH_INFO: `/v3/users/e3793b9a5b7f4c55bd98a47495e4ad8b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.822640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3793b9a5b7f4c55bd98a47495e4ad8b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:11.824854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:11.827006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:11.828825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.856492 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 855/1702] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:11.858234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:11.859918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.864065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:11.885020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba036a3b-f902-4016-b4db-6198a5138ebe None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:11.891001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:11.891912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.892375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.892511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:11.893100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:11.895219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:11.916737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3793b9a5b7f4c55bd98a47495e4ad8b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.937556 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 856/1703] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:11 2022] POST /identity/v3/projects => generated 403 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:11.937556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:11.939956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:11.942163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3793b9a5b7f4c55bd98a47495e4ad8b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.950648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3793b9a5b7f4c55bd98a47495e4ad8b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.966765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3793b9a5b7f4c55bd98a47495e4ad8b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.967997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a14782fe-e382-4183-8021-ccb46c2ba852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3793b9a5b7f4c55bd98a47495e4ad8b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:11.982932 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 848/1704] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:11 2022] DELETE /identity/v3/users/e3793b9a5b7f4c55bd98a47495e4ad8b => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:11.984759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:11.985201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:11.985643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.009103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:12.010492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.042898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff360-05d2-403f-bc5d-805d68f16ca5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.117129 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 849/1705] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:12.118970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.121271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.128572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.130982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.131611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.131959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] PATH_INFO: `/v3/projects/8b32eb78eef140bab4197fe048c5d68c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.133316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b32eb78eef140bab4197fe048c5d68c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.137104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.141441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad575-62fd-4201-86bf-225a2476e339 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.146662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.147284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.147688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.147967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.148742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.151821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.214236 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 850/1706] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:12 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:12.224085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.229044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b32eb78eef140bab4197fe048c5d68c', 'request_id': 'req-a1ae1a87-2171-40df-a384-9f4e113d7b2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.238967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797af93-4322-42f8-8f9f-8bdc384b8702 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.243073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b32eb78eef140bab4197fe048c5d68c', 'request_id': 'req-a1ae1a87-2171-40df-a384-9f4e113d7b2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.263174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b32eb78eef140bab4197fe048c5d68c', 'request_id': 'req-a1ae1a87-2171-40df-a384-9f4e113d7b2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.314135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] The token cache is being invalidate because project 8b32eb78eef140bab4197fe048c5d68c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:12.314438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.316614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.325145 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 857/1707] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:11 2022] DELETE /identity/v3/projects/8b32eb78eef140bab4197fe048c5d68c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:12.327742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.331425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.349817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:12.359188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.359749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.361974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.361974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.361974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:12.361974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.364834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.390085 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 851/1708] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:12.391046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.395071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.412813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55cb2e-2d11-4aaa-8f7a-d87a3e72378b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.420021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.421780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.422345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.422770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] PATH_INFO: `/v3/projects/8c184615b31044a88b36e83e5982beec` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.425339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c184615b31044a88b36e83e5982beec)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.427921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.432352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:12.471917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae1a87-2171-40df-a384-9f4e113d7b2f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.478856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dd461-bab9-4f86-b0c5-98c5c86e20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.488550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dd461-bab9-4f86-b0c5-98c5c86e20ba admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.489110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dd461-bab9-4f86-b0c5-98c5c86e20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.489499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dd461-bab9-4f86-b0c5-98c5c86e20ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.490947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dd461-bab9-4f86-b0c5-98c5c86e20ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.495884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dd461-bab9-4f86-b0c5-98c5c86e20ba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.527014 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 858/1709] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:12.532577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.532863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.533130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.540327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c184615b31044a88b36e83e5982beec', 'request_id': 'req-abadbc8e-b970-4001-8909-7eadc9bc15af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.548636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c184615b31044a88b36e83e5982beec', 'request_id': 'req-abadbc8e-b970-4001-8909-7eadc9bc15af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.564729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c184615b31044a88b36e83e5982beec', 'request_id': 'req-abadbc8e-b970-4001-8909-7eadc9bc15af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.565130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:12.566946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.586186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] The token cache is being invalidate because project 8c184615b31044a88b36e83e5982beec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:12.586518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.589422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.596712 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 852/1710] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:12 2022] DELETE /identity/v3/projects/8c184615b31044a88b36e83e5982beec => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:12.597664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.599272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.716993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadbc8e-b970-4001-8909-7eadc9bc15af admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.728999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.729673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.730191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.730617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.731891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.738009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.738498 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 859/1711] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:12.739493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.744797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.779838 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 853/1712] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:12 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:12.781777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:12.785054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.787769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3faf4-9262-4e2a-8ff5-5f96cbb76eee None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.865092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218012b0-178c-4674-b00d-d083ecafaa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.865656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218012b0-178c-4674-b00d-d083ecafaa69 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.865994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218012b0-178c-4674-b00d-d083ecafaa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.866286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218012b0-178c-4674-b00d-d083ecafaa69 admin admin] PATH_INFO: `/v3/projects/ca6b53504e15462ea8552410a6481fca/users/7ded5b10c6294acba430ea04b44fb72f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.875484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58022078-6cb9-4f5b-a802-0cbe74c9383c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:12.882623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:12.884064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.884299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.884648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] PATH_INFO: `/v3/projects/b237bd9a5ed447919289225cff98741a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.886802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b237bd9a5ed447919289225cff98741a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.890214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.894306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218012b0-178c-4674-b00d-d083ecafaa69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6b53504e15462ea8552410a6481fca, user_id=7ded5b10c6294acba430ea04b44fb72f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:12.896614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218012b0-178c-4674-b00d-d083ecafaa69 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:12.918422 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 860/1713] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:12 2022] PUT /identity/v3/projects/ca6b53504e15462ea8552410a6481fca/users/7ded5b10c6294acba430ea04b44fb72f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:12.918422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:12.918422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:12.918422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:12.921646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:12.942835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:12.944400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:12.955266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b237bd9a5ed447919289225cff98741a', 'request_id': 'req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.962474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b237bd9a5ed447919289225cff98741a', 'request_id': 'req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.967392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:12.978059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b237bd9a5ed447919289225cff98741a', 'request_id': 'req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.995459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] The token cache is being invalidate because project b237bd9a5ed447919289225cff98741a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:12.996313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.997039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4673a6c-4373-499a-b1b2-258cbc9ac0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:12.999011 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 854/1714] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:12 2022] DELETE /identity/v3/projects/b237bd9a5ed447919289225cff98741a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:13.003574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.003946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.004300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.016163 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 861/1715] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:13.022863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.024788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:13.028502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:13.029784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:13.037726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec54a70-9dfa-4932-97a5-fc44dfbb8af7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.038998 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 855/1716] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:13.040561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.041993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:13.046218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306d3fd-af1b-4e07-bb7e-985f97235671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.046852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306d3fd-af1b-4e07-bb7e-985f97235671 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.047181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306d3fd-af1b-4e07-bb7e-985f97235671 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.047458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306d3fd-af1b-4e07-bb7e-985f97235671 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.048224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306d3fd-af1b-4e07-bb7e-985f97235671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.051240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306d3fd-af1b-4e07-bb7e-985f97235671 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.054633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3bd17-304b-4bd5-97ab-da468f735a55 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.060564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bafe0c-583d-4e93-9fef-27616d65e5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.061335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bafe0c-583d-4e93-9fef-27616d65e5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.061718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bafe0c-583d-4e93-9fef-27616d65e5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.062126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bafe0c-583d-4e93-9fef-27616d65e5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.062869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bafe0c-583d-4e93-9fef-27616d65e5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.065164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bafe0c-583d-4e93-9fef-27616d65e5c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.076365 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 856/1717] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:13.077105 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 862/1718] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/users => generated 292 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:13.078639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a4b4-fb9a-4472-a29e-7b0ae5c95caa None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.079076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a4b4-fb9a-4472-a29e-7b0ae5c95caa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.079737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a4b4-fb9a-4472-a29e-7b0ae5c95caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.080615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a959e7-adf4-413d-a7f5-8bf824c9f27c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.080909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a959e7-adf4-413d-a7f5-8bf824c9f27c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.081188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a959e7-adf4-413d-a7f5-8bf824c9f27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.100748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6ad4a4b4-fb9a-4472-a29e-7b0ae5c95caa None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:13.102018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6ad4a4b4-fb9a-4472-a29e-7b0ae5c95caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:13.103515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-87a959e7-adf4-413d-a7f5-8bf824c9f27c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:13.104735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87a959e7-adf4-413d-a7f5-8bf824c9f27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:13.113687 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 863/1719] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:13.113687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4a4b4-fb9a-4472-a29e-7b0ae5c95caa None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.115621 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 857/1720] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:13.119660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a959e7-adf4-413d-a7f5-8bf824c9f27c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.125377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4a4b4-fb9a-4472-a29e-7b0ae5c95caa None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.130464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.131220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.131638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.131979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.132705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:13.133096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.135225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a959e7-adf4-413d-a7f5-8bf824c9f27c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.135355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.140739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.141469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.141832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.142193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] PATH_INFO: `/v3/projects/ca6b53504e15462ea8552410a6481fca/users/7ded5b10c6294acba430ea04b44fb72f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.144326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6b53504e15462ea8552410a6481fca, user_id=7ded5b10c6294acba430ea04b44fb72f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.147148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.148859 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 864/1721] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:13.149878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.160289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc560e-5056-4788-96a2-c32aa71567b4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.165320 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 858/1722] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:13 2022] PUT /identity/v3/projects/ca6b53504e15462ea8552410a6481fca/users/7ded5b10c6294acba430ea04b44fb72f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:13.165320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.249873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ae758-5d81-4dd2-83da-da5cbd6ea051 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.253468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974d96e-e95c-457c-be51-44830ce3c0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.254349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d96e-e95c-457c-be51-44830ce3c0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.254735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d96e-e95c-457c-be51-44830ce3c0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.255109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d96e-e95c-457c-be51-44830ce3c0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.255850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.255986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974d96e-e95c-457c-be51-44830ce3c0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.257005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.257750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.257889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974d96e-e95c-457c-be51-44830ce3c0d9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.258516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.259189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:13.259627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.262252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.270323 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 865/1723] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:13.271861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf2e5-69eb-4e7d-9100-005deeb8e020 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.272247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf2e5-69eb-4e7d-9100-005deeb8e020 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.272621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf2e5-69eb-4e7d-9100-005deeb8e020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.280277 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 859/1724] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:13.281894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.298990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680fb65-1eb0-45db-8e6b-9194d5a60911 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.304054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-989cf2e5-69eb-4e7d-9100-005deeb8e020 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:13.306239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-989cf2e5-69eb-4e7d-9100-005deeb8e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:13.306872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.308162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.308452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.308935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.310542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.313065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.316362 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 866/1725] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:13.317479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cf2e5-69eb-4e7d-9100-005deeb8e020 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.330599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cf2e5-69eb-4e7d-9100-005deeb8e020 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.337479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.338482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.339018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.340008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.340943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.343099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.362870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 867/1726] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:13.364523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.389051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5e2d0-00be-4b7f-b7d7-d15acb5c10af admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.396718 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 860/1727] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/projects => generated 399 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:13.397595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.403481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.404363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.404890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.405311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.406109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.408543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.452367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 868/1728] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:13.454805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.490501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.520317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd689e-5541-4cf6-baa2-f38a07736bad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.525796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:13.580110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4c61c-2fc9-4bed-a9d3-e0ed88967e8b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.600164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.601205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.601694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.602794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] PATH_INFO: `/v3/projects/db0a0cc5f17b4fdbab61b12a89ba84a8/users/9c8ace9428404ed4a6ef798e7dc2e628/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.632680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.634002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.634543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.634914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.635886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:13.636199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.640096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.654193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0a0cc5f17b4fdbab61b12a89ba84a8, user_id=9c8ace9428404ed4a6ef798e7dc2e628, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.659505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.672386 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 861/1729] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:13.673452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.682874 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 869/1730] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:13 2022] PUT /identity/v3/projects/db0a0cc5f17b4fdbab61b12a89ba84a8/users/9c8ace9428404ed4a6ef798e7dc2e628/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:13.684316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.687658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f59a5-f72a-4f18-9ad8-4155c776d101 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.796440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.797600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.798263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.798631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.800662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.803305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.833700 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 862/1731] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:13.834576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.847292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae267c-881f-466d-9881-c9a10a80fb06 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.847615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69382006-5c23-4965-b835-fe46a65cb912 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:13.854540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c323c0-a4c2-433c-8552-025240e229be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.855162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c323c0-a4c2-433c-8552-025240e229be admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.855437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c323c0-a4c2-433c-8552-025240e229be admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.855712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c323c0-a4c2-433c-8552-025240e229be admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.856346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c323c0-a4c2-433c-8552-025240e229be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.858545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c323c0-a4c2-433c-8552-025240e229be admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.885670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:13.886657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.887105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.887522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.888314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:13.890488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:13.903568 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 863/1732] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:13.904891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46ec9c-ac5c-469e-ba7b-72d5f5fa28d6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:13.905178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46ec9c-ac5c-469e-ba7b-72d5f5fa28d6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:13.905451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46ec9c-ac5c-469e-ba7b-72d5f5fa28d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:13.946921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5f46ec9c-ac5c-469e-ba7b-72d5f5fa28d6 None None] MFA Rules not processed for user `9c8ace9428404ed4a6ef798e7dc2e628`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:13.949488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f46ec9c-ac5c-469e-ba7b-72d5f5fa28d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:13.953859 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 870/1733] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:13.954811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:13.974636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dacde-558e-48e1-bc6f-6bc345e77d6e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.013086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.046650 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 864/1734] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:14.048896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46ec9c-ac5c-469e-ba7b-72d5f5fa28d6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.072689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb41741-f723-4450-819e-9a69a38feffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.073892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb41741-f723-4450-819e-9a69a38feffc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.074371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb41741-f723-4450-819e-9a69a38feffc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.075283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb41741-f723-4450-819e-9a69a38feffc admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.076243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb41741-f723-4450-819e-9a69a38feffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.078414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb41741-f723-4450-819e-9a69a38feffc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.092476 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 871/1735] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:14.094382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20999d-14b9-4f05-8688-6ec5d800baaa None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.095114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20999d-14b9-4f05-8688-6ec5d800baaa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.095652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20999d-14b9-4f05-8688-6ec5d800baaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.169621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-dd20999d-14b9-4f05-8688-6ec5d800baaa None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:14.171077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46ec9c-ac5c-469e-ba7b-72d5f5fa28d6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.171741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dd20999d-14b9-4f05-8688-6ec5d800baaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:14.176422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.177310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.178424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.195492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.196957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.200717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.214497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.239267 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 865/1736] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:14 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:14.239267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.257136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f56bb-0005-4a1d-a79f-d6518e693381 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.264782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.265667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.266637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.267075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.267913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.269848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.286443 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 866/1737] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:14.287460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.291267 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 872/1738] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:14.292150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20999d-14b9-4f05-8688-6ec5d800baaa None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.307181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20999d-14b9-4f05-8688-6ec5d800baaa None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.313005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.313640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.319402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.319402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.319402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.319402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.330928 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 873/1739] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:14.331997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.347201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8164caa-ca8e-4018-b741-bb0b33c47dd8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.347325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9fae5-3f19-4dc2-9519-6d2b293543b3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.354099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.354099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.354351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.354740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.355215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.356882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.361958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.369961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 874/1740] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:14.371087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.371774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.372072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.372595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.373104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.375082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.387885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6dc0f-c83e-46ac-8b36-1c2304587beb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.394180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.395179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.395641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.396517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.397730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.401103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.412877 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 867/1741] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:14 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:14.414132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.417413 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 875/1742] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:14.418657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.432145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f016f-a8fd-4166-ac80-917c7920e490 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.511734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.513002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.513457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.513991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.514875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.517192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.530157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba438072-ac72-4831-a184-4db5595d6419 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.535869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 876/1743] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:14.535869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.539378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.540991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.541357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.541708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] PATH_INFO: `/v3/projects/137b68e3885041a6bac6f4a55d96cf92/users/2cb680a77e4549499bf8f2616686d147/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.548446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f4ead-a1c3-45b5-9548-ab41f842b2f1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.554814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.556311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.556673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.557000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.557762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.560621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.563147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] RBAC: Authorizing `identity:create_grant(project_id=137b68e3885041a6bac6f4a55d96cf92, user_id=2cb680a77e4549499bf8f2616686d147, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.565819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.587297 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 868/1744] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:14 2022] PUT /identity/v3/projects/137b68e3885041a6bac6f4a55d96cf92/users/2cb680a77e4549499bf8f2616686d147/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:14.588597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.591246 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 877/1745] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:14 2022] POST /identity/v3/users => generated 299 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:14.592471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.594679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:14.621227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.691347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7aca0-7172-421a-9eea-d9ea0d1e929e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.702933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.704651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.705052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.705970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] PATH_INFO: `/v3/projects/10ac4880864d4024bbba272c908d1b67/users/b4859b9cd38c4aad92e0d54b9f4cbaaf/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.709763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:14.732158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ac4880864d4024bbba272c908d1b67, user_id=b4859b9cd38c4aad92e0d54b9f4cbaaf, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.735403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.752611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6da30c-5149-4976-b6d8-c11da15b6f43 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.770847 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 869/1746] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:14 2022] PUT /identity/v3/projects/10ac4880864d4024bbba272c908d1b67/users/b4859b9cd38c4aad92e0d54b9f4cbaaf/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:14.770847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:14.843197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.843885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.844817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.845002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.858313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea63d-ba7c-4a11-8768-8be0e50a7b52 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:14.864661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea490eb-ab6a-4270-a059-c3ba14872c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:14.865660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea490eb-ab6a-4270-a059-c3ba14872c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.866037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea490eb-ab6a-4270-a059-c3ba14872c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.866378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea490eb-ab6a-4270-a059-c3ba14872c4c admin admin] PATH_INFO: `/v3/projects/35ef1b5d5dba4e3c93d8bb622ff79c5e/users/5aab292e925940568e9e5db8b6910f45/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.887438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea490eb-ab6a-4270-a059-c3ba14872c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ef1b5d5dba4e3c93d8bb622ff79c5e, user_id=5aab292e925940568e9e5db8b6910f45, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:14.891175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea490eb-ab6a-4270-a059-c3ba14872c4c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:14.909365 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 870/1747] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:14 2022] PUT /identity/v3/projects/35ef1b5d5dba4e3c93d8bb622ff79c5e/users/5aab292e925940568e9e5db8b6910f45/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:14.911050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb378-99a9-4f46-b761-868ffea92be0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:14.911412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb378-99a9-4f46-b761-868ffea92be0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:14.912638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb378-99a9-4f46-b761-868ffea92be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:14.954157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7b9cb378-99a9-4f46-b761-868ffea92be0 None None] MFA Rules not processed for user `2cb680a77e4549499bf8f2616686d147`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:14.956336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b9cb378-99a9-4f46-b761-868ffea92be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:15.036357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:15.041360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:15.067327 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 878/1748] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:48:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:48:15.068661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:15.096016 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 871/1749] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:15.096661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4721731-d563-469f-adac-e28163e425c2 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:15.098597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0b87c-2200-4ef6-b525-c5f39ceb3ae3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.099181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0b87c-2200-4ef6-b525-c5f39ceb3ae3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.108507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0b87c-2200-4ef6-b525-c5f39ceb3ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.205455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dab0b87c-2200-4ef6-b525-c5f39ceb3ae3 None None] MFA Rules not processed for user `b4859b9cd38c4aad92e0d54b9f4cbaaf`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:15.206855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dab0b87c-2200-4ef6-b525-c5f39ceb3ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:15.215741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:15.264642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aae701-bad3-431d-ba3c-94259e6b7d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:15.265379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aae701-bad3-431d-ba3c-94259e6b7d1d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.266513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aae701-bad3-431d-ba3c-94259e6b7d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.266878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aae701-bad3-431d-ba3c-94259e6b7d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.267605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aae701-bad3-431d-ba3c-94259e6b7d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:15.269466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aae701-bad3-431d-ba3c-94259e6b7d1d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:15.295254 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 879/1750] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:15.297523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c83d-f2ca-455d-8aa5-ba1fdc73829b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.298730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c83d-f2ca-455d-8aa5-ba1fdc73829b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.298900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c83d-f2ca-455d-8aa5-ba1fdc73829b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.370781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9000c83d-f2ca-455d-8aa5-ba1fdc73829b None None] MFA Rules not processed for user `5aab292e925940568e9e5db8b6910f45`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:15.378151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9000c83d-f2ca-455d-8aa5-ba1fdc73829b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:15.416694 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 872/1751] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:15.417896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0b87c-2200-4ef6-b525-c5f39ceb3ae3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:15.424137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:15.447564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0b87c-2200-4ef6-b525-c5f39ceb3ae3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:15.461903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:15.462516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.462810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.463083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.463787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:15.465806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:15.557730 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 873/1752] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:15 2022] POST /identity/v3/projects => generated 397 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:15.557730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:15.570077 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 880/1753] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:15.571326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c83d-f2ca-455d-8aa5-ba1fdc73829b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:15.604824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c83d-f2ca-455d-8aa5-ba1fdc73829b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:15.689417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc17cdb-2d71-4d0f-9d10-aa38b698b1ac admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:15.715448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404ca6-901e-459a-9244-85880452928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:15.716212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404ca6-901e-459a-9244-85880452928f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.716625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404ca6-901e-459a-9244-85880452928f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.717576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404ca6-901e-459a-9244-85880452928f admin admin] PATH_INFO: `/v3/projects/4727b4b68ccd44c8b2dd41aa67fd7d68/users/340a27eb1140487b806bbcdc232a204b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.766536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:15.767591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:15.768081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:15.768557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:15.769392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:15.771893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:15.776586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76404ca6-901e-459a-9244-85880452928f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4727b4b68ccd44c8b2dd41aa67fd7d68, user_id=340a27eb1140487b806bbcdc232a204b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:15.780094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76404ca6-901e-459a-9244-85880452928f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:15.817939 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 874/1754] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:15 2022] PUT /identity/v3/projects/4727b4b68ccd44c8b2dd41aa67fd7d68/users/340a27eb1140487b806bbcdc232a204b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:15.820056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404ca6-901e-459a-9244-85880452928f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:15.859693 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 881/1755] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:15 2022] POST /identity/v3/projects => generated 383 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:15.860706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.013227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404ca6-901e-459a-9244-85880452928f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.047276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:16.053455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a6b2c-7a11-4ada-be63-9ea31a82b98c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.083232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.084016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.084317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.084580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.085222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.093874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.166977 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 882/1756] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:15 2022] POST /identity/v3/projects => generated 385 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:16.168060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.198633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.199965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.200740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.201349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.206098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.214757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.280744 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 875/1757] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:15 2022] POST /identity/v3/users => generated 305 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:16.281495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.285763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:16.315476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596940b-119d-4e91-a76a-f44a6f04a09b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.364605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d47fb-e489-439c-8dfe-8b56e4ef5e6a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.374610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0189cc2-a5d1-44b5-9933-1f796b87177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.376124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189cc2-a5d1-44b5-9933-1f796b87177e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.376562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189cc2-a5d1-44b5-9933-1f796b87177e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.376989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189cc2-a5d1-44b5-9933-1f796b87177e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.377791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0189cc2-a5d1-44b5-9933-1f796b87177e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.380244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0189cc2-a5d1-44b5-9933-1f796b87177e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.406696 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 883/1758] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:16.408305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c088d7-a1f5-4853-9c5c-b7d4e9580a78 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.408747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c088d7-a1f5-4853-9c5c-b7d4e9580a78 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.409187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c088d7-a1f5-4853-9c5c-b7d4e9580a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.419270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.420353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.420747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.421347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.422333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.426651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.481856 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 876/1759] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:16 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:16.481856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.507014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-42c088d7-a1f5-4853-9c5c-b7d4e9580a78 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:16.508836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-42c088d7-a1f5-4853-9c5c-b7d4e9580a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:16.522567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e673b-7872-41e6-92b8-2d4b75154d29 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.545778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.546706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.547097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.547570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.548457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.551553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.613615 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 884/1760] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Sep 10 20:48:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:16.614854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c088d7-a1f5-4853-9c5c-b7d4e9580a78 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.636178 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 877/1761] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:16 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:16.636178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.643055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c088d7-a1f5-4853-9c5c-b7d4e9580a78 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.660816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19041ba0-55a8-4ea3-9920-e879cb8e7aee admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.666594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.667377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.667791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.668234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.669263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.674157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.675043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.675946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.676330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.677053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] PATH_INFO: `/v3/projects/4727b4b68ccd44c8b2dd41aa67fd7d68/users/340a27eb1140487b806bbcdc232a204b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.680533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4727b4b68ccd44c8b2dd41aa67fd7d68, user_id=340a27eb1140487b806bbcdc232a204b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.691552 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 885/1762] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:16.692667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.693539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.707711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f958e6-7687-48b1-b997-c916814f3cbe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.724484 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 878/1763] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:16 2022] PUT /identity/v3/projects/4727b4b68ccd44c8b2dd41aa67fd7d68/users/340a27eb1140487b806bbcdc232a204b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:16.726019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.739525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:16.804252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.805423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.807870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.808667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.809865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.813244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.840687 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 886/1764] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:16.843739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.867214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909cdfd-5bb6-4fa0-b887-cbdafe48d066 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.876835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff92891-4624-48d9-b79c-b78ae66fa2a9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:16.881890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.892450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.893447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.894949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.897160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.901803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19dd6e-63ea-480c-bdc7-0a807e179b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:16.902775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19dd6e-63ea-480c-bdc7-0a807e179b9c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.903387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19dd6e-63ea-480c-bdc7-0a807e179b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.903607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.905461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19dd6e-63ea-480c-bdc7-0a807e179b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.937651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19dd6e-63ea-480c-bdc7-0a807e179b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:16.942188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19dd6e-63ea-480c-bdc7-0a807e179b9c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:16.950669 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 887/1765] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:16.952016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:16.979798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 879/1766] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:16.984511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7df0-b77d-4cb1-bc6d-d7424d1e5e07 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:16.985069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7df0-b77d-4cb1-bc6d-d7424d1e5e07 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:16.986386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c7df0-b77d-4cb1-bc6d-d7424d1e5e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:16.996097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6e6bc-a3d4-4995-9005-a8f1bb6659cf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:17.005112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:17.005847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:17.010171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:17.013285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:17.013684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] PATH_INFO: `/v3/projects/6f0a69129cee4059b288fa530eb3c195/users/4826a2055974462f9251416540a7212c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:17.069880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f0a69129cee4059b288fa530eb3c195, user_id=4826a2055974462f9251416540a7212c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:17.074091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:17.092070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a10c7df0-b77d-4cb1-bc6d-d7424d1e5e07 None None] MFA Rules not processed for user `340a27eb1140487b806bbcdc232a204b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:17.096893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a10c7df0-b77d-4cb1-bc6d-d7424d1e5e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:17.118747 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 888/1767] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:16 2022] PUT /identity/v3/projects/6f0a69129cee4059b288fa530eb3c195/users/4826a2055974462f9251416540a7212c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:17.119791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:17.168940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:17.328124 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 880/1768] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:17.329963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7df0-b77d-4cb1-bc6d-d7424d1e5e07 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:17.332866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630f198-5ccc-4ca3-a982-b399db0993b0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:17.347417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:17.348112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:17.348390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:17.348676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] PATH_INFO: `/v3/projects/57d5b4d739ff4018a4842a2c243a897d/users/c3744c54e14342c8b7f1cc9bdb370de7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:17.355406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c7df0-b77d-4cb1-bc6d-d7424d1e5e07 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:17.363669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:17.364590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:17.365038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:17.365460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] PATH_INFO: `/v3/projects/1b12288ef70e4b38b128866c7f2efd47/users/35c7c4e102e94a5c92d52b94099d50b5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:17.395450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b12288ef70e4b38b128866c7f2efd47, user_id=35c7c4e102e94a5c92d52b94099d50b5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:17.400082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:17.408206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d5b4d739ff4018a4842a2c243a897d, user_id=c3744c54e14342c8b7f1cc9bdb370de7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:17.412873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:17.437660 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 881/1769] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:17 2022] PUT /identity/v3/projects/1b12288ef70e4b38b128866c7f2efd47/users/35c7c4e102e94a5c92d52b94099d50b5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:17.438950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:17.447592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 889/1770] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:17 2022] PUT /identity/v3/projects/57d5b4d739ff4018a4842a2c243a897d/users/c3744c54e14342c8b7f1cc9bdb370de7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:17.448617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:17.454717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:17.648135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69264605-507b-4f9c-9a3d-409b762e17cc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:17.688326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09b81b-6f8e-41d0-a77c-4bb38dbf2a0d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:17.704496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:17.712044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:17.712576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:17.712870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:17.831394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:17.832117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:17.832317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:17.832601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:17.833259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:17.834931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:17.860874 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 882/1771] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:17.871698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:17.882124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:17.883997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:17.898290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1e90-f80b-44de-aa61-1e4dffc9c780 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:17.940431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:17.942777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:17.944915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 890/1772] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:17.945403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:17.946177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:17.948694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:17.953756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:17.955468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:17.971180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:18.010554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfde11-f972-4416-839f-8876fc95e2c1 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:18.031880 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 883/1773] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:18.033451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:18.035883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:18.036929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.039286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.040800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.041727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:18.044229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:18.050174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f93825-ede9-4c9a-82b7-922c757c9bf0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:18.061202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:18.062590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.062883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.063160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] PATH_INFO: `/v3/projects/6f0a69129cee4059b288fa530eb3c195/users/4826a2055974462f9251416540a7212c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.065133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f0a69129cee4059b288fa530eb3c195, user_id=4826a2055974462f9251416540a7212c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:18.069048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:18.088966 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 884/1774] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:18 2022] PUT /identity/v3/projects/6f0a69129cee4059b288fa530eb3c195/users/4826a2055974462f9251416540a7212c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:18.088966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:18.089605 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 891/1775] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:18.097867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:18.112852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9bcb-64b4-4556-a2c1-29c27fb20e1e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:18.144593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:18.188395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfc502-d243-4686-b9f1-12b97a8bd0a6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:18.194557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ba950-a0fe-4799-9d08-70372833d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:18.196860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba950-a0fe-4799-9d08-70372833d17b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.197294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba950-a0fe-4799-9d08-70372833d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.197719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba950-a0fe-4799-9d08-70372833d17b admin admin] PATH_INFO: `/v3/projects/1b12288ef70e4b38b128866c7f2efd47/users/35c7c4e102e94a5c92d52b94099d50b5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.200108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ba950-a0fe-4799-9d08-70372833d17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b12288ef70e4b38b128866c7f2efd47, user_id=35c7c4e102e94a5c92d52b94099d50b5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:18.205434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ba950-a0fe-4799-9d08-70372833d17b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:18.234113 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 885/1776] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:18 2022] PUT /identity/v3/projects/1b12288ef70e4b38b128866c7f2efd47/users/35c7c4e102e94a5c92d52b94099d50b5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:18.236555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b11553-cf0b-4a80-b1a5-4304317e1d6e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.237337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b11553-cf0b-4a80-b1a5-4304317e1d6e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.237999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b11553-cf0b-4a80-b1a5-4304317e1d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.293017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-67b11553-cf0b-4a80-b1a5-4304317e1d6e None None] MFA Rules not processed for user `4826a2055974462f9251416540a7212c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:18.294347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-67b11553-cf0b-4a80-b1a5-4304317e1d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:18.300701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb957f-3ad8-479b-9116-43a071f919b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:18.302343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb957f-3ad8-479b-9116-43a071f919b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.307545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb957f-3ad8-479b-9116-43a071f919b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.308968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb957f-3ad8-479b-9116-43a071f919b0 admin admin] PATH_INFO: `/v3/projects/57d5b4d739ff4018a4842a2c243a897d/users/c3744c54e14342c8b7f1cc9bdb370de7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.314692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb957f-3ad8-479b-9116-43a071f919b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d5b4d739ff4018a4842a2c243a897d, user_id=c3744c54e14342c8b7f1cc9bdb370de7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:18.338730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb957f-3ad8-479b-9116-43a071f919b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:18.389357 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 892/1777] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:18 2022] PUT /identity/v3/projects/57d5b4d739ff4018a4842a2c243a897d/users/c3744c54e14342c8b7f1cc9bdb370de7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:18.395296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699bf5c-41d3-4047-8aa7-07eae4859239 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.395697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699bf5c-41d3-4047-8aa7-07eae4859239 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.396610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699bf5c-41d3-4047-8aa7-07eae4859239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.397042 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 886/1778] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:18 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:18.398532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.399921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.400271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.446855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] MFA Rules not processed for user `c3744c54e14342c8b7f1cc9bdb370de7`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:18.448726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:18.510549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f699bf5c-41d3-4047-8aa7-07eae4859239 None None] MFA Rules not processed for user `35c7c4e102e94a5c92d52b94099d50b5`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:18.517000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f699bf5c-41d3-4047-8aa7-07eae4859239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:18.654159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:18.747575 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 887/1779] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:18.750860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:18.759435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:18.845341 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 893/1780] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:18.852914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e14e9-444a-41a6-af3f-f908f99b05ac None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.853213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e14e9-444a-41a6-af3f-f908f99b05ac None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.855180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e14e9-444a-41a6-af3f-f908f99b05ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:18.965818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c128cf-53f4-4dac-849f-76f7b1f91cb7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:18.990313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:18.991203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:18.991633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4a8e14e9-444a-41a6-af3f-f908f99b05ac None None] MFA Rules not processed for user `5aab292e925940568e9e5db8b6910f45`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:18.991764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:18.992104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.000307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4a8e14e9-444a-41a6-af3f-f908f99b05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:19.169122 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 894/1781] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:19.182161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.186318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.236146 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 888/1782] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:19.282544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e14e9-444a-41a6-af3f-f908f99b05ac None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:19.299767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:19.314975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e14e9-444a-41a6-af3f-f908f99b05ac None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:19.318443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba509009-c48d-4d70-bd70-d047c1720642 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:19.333229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed04c3-3072-46ce-b3a6-ddb73b7be4a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:19.334322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed04c3-3072-46ce-b3a6-ddb73b7be4a0 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.334612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed04c3-3072-46ce-b3a6-ddb73b7be4a0 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.334893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed04c3-3072-46ce-b3a6-ddb73b7be4a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.345935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:19.346685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.346977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.347270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.365125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed04c3-3072-46ce-b3a6-ddb73b7be4a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.369381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed04c3-3072-46ce-b3a6-ddb73b7be4a0 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.370722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.372659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.396795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 895/1783] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:19.424514 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 889/1784] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:19.581472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:19.632171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:19.674730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be2800-7f8b-4f84-bfb1-3382b79c00c7 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:19.679622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:19.680572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.680917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.682291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] PATH_INFO: `/v3/users/9c8ace9428404ed4a6ef798e7dc2e628` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.683579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8ace9428404ed4a6ef798e7dc2e628)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:19.685690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:19.749967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675905-6da2-4fe2-b1f1-ee403cd92033 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.750436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675905-6da2-4fe2-b1f1-ee403cd92033 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.750803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675905-6da2-4fe2-b1f1-ee403cd92033 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.753207 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 890/1785] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:48:19.764560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8ace9428404ed4a6ef798e7dc2e628'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:19.782556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8ace9428404ed4a6ef798e7dc2e628'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:19.784947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8ace9428404ed4a6ef798e7dc2e628'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:19.808485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8ace9428404ed4a6ef798e7dc2e628'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:19.809243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25cf4bb3-e150-424e-93f3-bdeefe0acc42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8ace9428404ed4a6ef798e7dc2e628'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:19.858595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284399c-828a-47a3-b608-059965d5c745 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.858784 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 896/1786] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:19 2022] DELETE /identity/v3/users/9c8ace9428404ed4a6ef798e7dc2e628 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:19.858880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284399c-828a-47a3-b608-059965d5c745 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.859756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284399c-828a-47a3-b608-059965d5c745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.862563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:19.862964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:19.863250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:19.913173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:19.915479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:19.918633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e284399c-828a-47a3-b608-059965d5c745 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:19.926875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e284399c-828a-47a3-b608-059965d5c745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:20.027891 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 897/1787] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:20.137461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 891/1788] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:48:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:48:20.326265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:20.340035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:20.522273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:20.534905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad79ac5-fe8f-4b22-9760-4dc0c5ee78f7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:20.545804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:20.547202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:20.548228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:20.550378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:20.576994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:20.578509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:20.614622 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 898/1789] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:21.530424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284399c-828a-47a3-b608-059965d5c745 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:21.535236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e284399c-828a-47a3-b608-059965d5c745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:21.548643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:21.558081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284399c-828a-47a3-b608-059965d5c745 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:21.580543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:21.581174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:21.581457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:21.581796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:21.594619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:21.596463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:21.613134 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 892/1790] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:21.867589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:21.869375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:21.922942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c881c3-d4b3-424a-9714-2aff0c2e5e2f service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:21.953461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:21.954157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:21.954406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:21.963032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:21.971363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:22.380563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:22.387489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:22.411994 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 899/1791] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:21 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:22.725115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:22.730483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:22.808153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924187a-b47b-494d-8303-45f786377a9b service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:22.832289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:22.833689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:22.835698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:22.836043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] PATH_INFO: `/v3/users/b4859b9cd38c4aad92e0d54b9f4cbaaf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:22.841989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4859b9cd38c4aad92e0d54b9f4cbaaf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:22.846154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:23.012667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4859b9cd38c4aad92e0d54b9f4cbaaf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.050218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4859b9cd38c4aad92e0d54b9f4cbaaf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.059056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:23.065776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:23.076972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4859b9cd38c4aad92e0d54b9f4cbaaf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.103699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4859b9cd38c4aad92e0d54b9f4cbaaf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.104397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-091b56ee-b05f-44b6-aa1a-c91e787e056c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4859b9cd38c4aad92e0d54b9f4cbaaf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.114759 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 893/1792] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:22 2022] DELETE /identity/v3/users/b4859b9cd38c4aad92e0d54b9f4cbaaf => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:23.123486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:23.123782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:23.124098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:23.141790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d2fbe-8be2-49df-bfbc-587085fbfdf6 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:23.169468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:23.170627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:23.398197 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 894/1793] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:23.401562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:23.404795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:23.412570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:23.414896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:23.416339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:23.418534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] PATH_INFO: `/v3/users/2cb680a77e4549499bf8f2616686d147` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:23.422276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb680a77e4549499bf8f2616686d147)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:23.425452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:23.449076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f460-2701-43e0-8863-89fe6ebb47ea None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:23.465221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:23.466280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:23.467820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:23.470343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] PATH_INFO: `/v3/projects/db0a0cc5f17b4fdbab61b12a89ba84a8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:23.481079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0a0cc5f17b4fdbab61b12a89ba84a8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:23.484074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:23.573794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb680a77e4549499bf8f2616686d147'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.596927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb680a77e4549499bf8f2616686d147'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.597705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb680a77e4549499bf8f2616686d147'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.619783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb680a77e4549499bf8f2616686d147'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.620467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb680a77e4549499bf8f2616686d147'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.657251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0a0cc5f17b4fdbab61b12a89ba84a8', 'request_id': 'req-a6d1c0c4-eaba-4768-a360-e2564fb34159'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.677711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0a0cc5f17b4fdbab61b12a89ba84a8', 'request_id': 'req-a6d1c0c4-eaba-4768-a360-e2564fb34159'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.689560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:23.713393 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 900/1794] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:23 2022] DELETE /identity/v3/users/2cb680a77e4549499bf8f2616686d147 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:23.714214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0a0cc5f17b4fdbab61b12a89ba84a8', 'request_id': 'req-a6d1c0c4-eaba-4768-a360-e2564fb34159'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.716707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:23.722619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:23.755442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] The token cache is being invalidate because project db0a0cc5f17b4fdbab61b12a89ba84a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:23.756025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.756437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6d1c0c4-eaba-4768-a360-e2564fb34159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:23.761399 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 895/1795] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:23 2022] DELETE /identity/v3/projects/db0a0cc5f17b4fdbab61b12a89ba84a8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:23.761399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:23.761399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:23.761399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:23.763517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:23.852430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:23.868218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:23.908629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde3cbd-5f98-4eb1-9bb4-46c83174c08b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:23.932736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:23.935805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:23.936354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:23.936867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:23.939615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:24.166576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:24.170838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:24.188126 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 901/1796] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:24.189546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:24.191359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:24.210706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 896/1797] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:24.230433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b64d90-5b11-44e4-be9f-503770240915 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:24.237039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:24.238125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:24.238659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:24.239145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] PATH_INFO: `/v3/users/340a27eb1140487b806bbcdc232a204b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:24.240658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=340a27eb1140487b806bbcdc232a204b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:24.243994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:24.288903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:24.290678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:24.316570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17bae-67b3-4727-be15-faf98cd0f54f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:24.354272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340a27eb1140487b806bbcdc232a204b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:24.361130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:24.363848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:24.364494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:24.365397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:24.382668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340a27eb1140487b806bbcdc232a204b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:24.385992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340a27eb1140487b806bbcdc232a204b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:24.405622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340a27eb1140487b806bbcdc232a204b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:24.407497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340a27eb1140487b806bbcdc232a204b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:24.413125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:24.418149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:24.456054 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 897/1798] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:24.460499 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 902/1799] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:24 2022] DELETE /identity/v3/users/340a27eb1140487b806bbcdc232a204b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:25.116251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:25.119960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:25.140291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0d592-4630-4e68-a86e-dc9cad501e41 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:25.364743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:25.372489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:25.372836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:25.375699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] PATH_INFO: `/v3/projects/10ac4880864d4024bbba272c908d1b67` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:25.378000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ac4880864d4024bbba272c908d1b67)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:25.380087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:25.535848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ac4880864d4024bbba272c908d1b67', 'request_id': 'req-62d7130d-b90d-4c99-9445-5605acd7fb2d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:25.554443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ac4880864d4024bbba272c908d1b67', 'request_id': 'req-62d7130d-b90d-4c99-9445-5605acd7fb2d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:25.572050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ac4880864d4024bbba272c908d1b67', 'request_id': 'req-62d7130d-b90d-4c99-9445-5605acd7fb2d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:25.608689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] The token cache is being invalidate because project 10ac4880864d4024bbba272c908d1b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:25.609094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:25.632399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62d7130d-b90d-4c99-9445-5605acd7fb2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:25.639541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 898/1800] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:25 2022] DELETE /identity/v3/projects/10ac4880864d4024bbba272c908d1b67 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:25.643602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:25.645099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:25.668666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ac16f-ce0e-4c80-8742-50321d778fad admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:25.745762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:25.914018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:25.919791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:25.920346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:25.920820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] PATH_INFO: `/v3/users/c3744c54e14342c8b7f1cc9bdb370de7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:25.924889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3744c54e14342c8b7f1cc9bdb370de7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:25.929211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40397c-d664-4ad1-9412-335570773c15 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:25.930007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40397c-d664-4ad1-9412-335570773c15 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:25.931135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40397c-d664-4ad1-9412-335570773c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:25.953278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:25.956774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:25.999105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7c40397c-d664-4ad1-9412-335570773c15 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:26.000380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7c40397c-d664-4ad1-9412-335570773c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:26.060690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3744c54e14342c8b7f1cc9bdb370de7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:26.070227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3744c54e14342c8b7f1cc9bdb370de7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:26.072162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3744c54e14342c8b7f1cc9bdb370de7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:26.098477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3744c54e14342c8b7f1cc9bdb370de7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:26.099615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3744c54e14342c8b7f1cc9bdb370de7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:26.133969 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 903/1801] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:25 2022] DELETE /identity/v3/users/c3744c54e14342c8b7f1cc9bdb370de7 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:26.173122 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 899/1802] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:26.206625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:26.208264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:26.392804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcdcf1-75c1-4e84-8d18-f3609f13d4fb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:26.405112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:26.405855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:26.407111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:26.407410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:26.516831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:26.538682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:26.542369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:26.556554 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 904/1803] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:48:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:48:27.246240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:27.246765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:27.246863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:27.335106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] MFA Rules not processed for user `7ded5b10c6294acba430ea04b44fb72f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:27.341982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:27.368035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:27.370411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:27.615159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c5bec-784b-414c-b70a-9f6dfe3f1241 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:27.624918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:27.625576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:27.626103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:27.626393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] PATH_INFO: `/v3/projects/4727b4b68ccd44c8b2dd41aa67fd7d68` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:27.630086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4727b4b68ccd44c8b2dd41aa67fd7d68)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:27.633562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:27.721579 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 900/1804] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:27.723449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:27.725261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:27.761126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9e307-50f3-4e74-80b9-13a4f410c030 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:27.794510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4727b4b68ccd44c8b2dd41aa67fd7d68', 'request_id': 'req-89111254-5eeb-4ed1-bf4f-a1cd007f475f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:27.810023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4727b4b68ccd44c8b2dd41aa67fd7d68', 'request_id': 'req-89111254-5eeb-4ed1-bf4f-a1cd007f475f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:27.823853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4727b4b68ccd44c8b2dd41aa67fd7d68', 'request_id': 'req-89111254-5eeb-4ed1-bf4f-a1cd007f475f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:27.860978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] The token cache is being invalidate because project 4727b4b68ccd44c8b2dd41aa67fd7d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:27.860978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:27.866330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:27.874132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 905/1805] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:27 2022] DELETE /identity/v3/projects/4727b4b68ccd44c8b2dd41aa67fd7d68 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:27.878624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:27.884971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:27.948587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:27.950127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:27.950668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:27.951758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] PATH_INFO: `/v3/projects/137b68e3885041a6bac6f4a55d96cf92` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:27.954224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=137b68e3885041a6bac6f4a55d96cf92)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:27.958062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:28.096620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:28.129436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137b68e3885041a6bac6f4a55d96cf92', 'request_id': 'req-f845223c-73a2-40dd-beeb-84a370375622'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.136409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89111254-5eeb-4ed1-bf4f-a1cd007f475f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:28.147101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137b68e3885041a6bac6f4a55d96cf92', 'request_id': 'req-f845223c-73a2-40dd-beeb-84a370375622'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.185514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:28.186982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137b68e3885041a6bac6f4a55d96cf92', 'request_id': 'req-f845223c-73a2-40dd-beeb-84a370375622'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.235377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] The token cache is being invalidate because project 137b68e3885041a6bac6f4a55d96cf92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:28.235617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.250289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.255530 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 901/1806] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:27 2022] DELETE /identity/v3/projects/137b68e3885041a6bac6f4a55d96cf92 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:28.255530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:28.256877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:28.399862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cd336-7409-4bf6-9a93-06e2f84918e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:28.402081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cd336-7409-4bf6-9a93-06e2f84918e3 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.402409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cd336-7409-4bf6-9a93-06e2f84918e3 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.402722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cd336-7409-4bf6-9a93-06e2f84918e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.409636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-063cd336-7409-4bf6-9a93-06e2f84918e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:28.460862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845223c-73a2-40dd-beeb-84a370375622 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:28.479057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:28.479712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.480008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.480267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] PATH_INFO: `/v3/projects/57d5b4d739ff4018a4842a2c243a897d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.482019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d5b4d739ff4018a4842a2c243a897d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:28.483954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:28.600503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cd336-7409-4bf6-9a93-06e2f84918e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:28.603620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cd336-7409-4bf6-9a93-06e2f84918e3 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:28.615259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d5b4d739ff4018a4842a2c243a897d', 'request_id': 'req-d68d3754-e266-4f35-a9b0-deb8005c6f08'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.632791 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 906/1807] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 758 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:28.634672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.637853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.638332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.652466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d5b4d739ff4018a4842a2c243a897d', 'request_id': 'req-d68d3754-e266-4f35-a9b0-deb8005c6f08'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.689057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:28.690602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d5b4d739ff4018a4842a2c243a897d', 'request_id': 'req-d68d3754-e266-4f35-a9b0-deb8005c6f08'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.691730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:28.745902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] The token cache is being invalidate because project 57d5b4d739ff4018a4842a2c243a897d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:28.746952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.748620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d68d3754-e266-4f35-a9b0-deb8005c6f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:28.753256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 902/1808] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:28 2022] DELETE /identity/v3/projects/57d5b4d739ff4018a4842a2c243a897d => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:28.754604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:28.754894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:28.755175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:28.811857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:28.817029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:28.955493 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 907/1809] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:28.955493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:28.955493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:28.956519 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 903/1810] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 10 20:48:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:28.958481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:28.963769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:28.985325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:29.005668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a4c41-0d79-464b-92d2-990390c46d43 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:29.022511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:29.026144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.026434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.026717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] PATH_INFO: `/v3/users/4826a2055974462f9251416540a7212c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.031645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4826a2055974462f9251416540a7212c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:29.033666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:29.144809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:29.153844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4826a2055974462f9251416540a7212c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:29.167938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4826a2055974462f9251416540a7212c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:29.170099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4826a2055974462f9251416540a7212c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:29.185494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4826a2055974462f9251416540a7212c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:29.186209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-55503e73-c03c-437e-b5ce-bddb21fa6574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4826a2055974462f9251416540a7212c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:29.207752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0b3b2-19b4-4626-8826-a7dcd33ba646 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:29.225627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc0339-7de9-4d77-9bcd-c545885497cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:29.229518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc0339-7de9-4d77-9bcd-c545885497cd service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.230036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc0339-7de9-4d77-9bcd-c545885497cd service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.230466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc0339-7de9-4d77-9bcd-c545885497cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.232480 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 908/1811] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:28 2022] DELETE /identity/v3/users/4826a2055974462f9251416540a7212c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:29.233865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b809125-50ab-4f78-a9db-a9ad3d9d7091 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.234163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b809125-50ab-4f78-a9db-a9ad3d9d7091 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.234445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b809125-50ab-4f78-a9db-a9ad3d9d7091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.258597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2fdc0339-7de9-4d77-9bcd-c545885497cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:29.274891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9b809125-50ab-4f78-a9db-a9ad3d9d7091 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:29.276259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9b809125-50ab-4f78-a9db-a9ad3d9d7091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:29.447377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc0339-7de9-4d77-9bcd-c545885497cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:29.450967 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 909/1812] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:29.452348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc0339-7de9-4d77-9bcd-c545885497cd service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:29.455282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5e0d9-80e6-439d-ab05-eab39245fc38 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.458625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5e0d9-80e6-439d-ab05-eab39245fc38 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.458904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5e0d9-80e6-439d-ab05-eab39245fc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.468217 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 904/1813] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:28 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:29.470678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c24b-324e-41fe-9192-de82e05d5acb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.471277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c24b-324e-41fe-9192-de82e05d5acb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.471858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c24b-324e-41fe-9192-de82e05d5acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.513456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-44f5e0d9-80e6-439d-ab05-eab39245fc38 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:29.518240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-44f5e0d9-80e6-439d-ab05-eab39245fc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:29.520945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-47d5c24b-324e-41fe-9192-de82e05d5acb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:29.522182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-47d5c24b-324e-41fe-9192-de82e05d5acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:29.538996 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 910/1814] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:29.543938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5e0d9-80e6-439d-ab05-eab39245fc38 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:29.551988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 905/1815] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:29.565262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5c24b-324e-41fe-9192-de82e05d5acb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:29.565810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5e0d9-80e6-439d-ab05-eab39245fc38 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:29.580017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5c24b-324e-41fe-9192-de82e05d5acb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:29.580841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:29.581503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.581787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.582325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.582737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:29.583042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:29.588950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:29.591699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:29.592327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.592610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.592885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.598861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:29.599305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:29.605610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:29.632476 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 911/1816] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:29.634076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:29.650351 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 906/1817] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:29.654129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:29.662011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfe7d1-3e90-4ae2-a362-e32471503e6d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:29.677351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:29.678088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.678367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.678647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.679384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:29.681752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:29.702591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06329fe-8a59-4555-a579-ed4e338ac46f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:29.717293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:29.718105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.721836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.722139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.722807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:29.728958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:29.768398 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 912/1818] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:29 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:29.773118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:29.795655 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 907/1819] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:29 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:29.799661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:29.901025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:29.908942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3f5d9-130f-4742-8eee-147291a805ec admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:29.916951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:29.918217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:29.918778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:29.919256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:29.920247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:29.922620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:29.956984 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 908/1820] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:29 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:29.958829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:29.965858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:30.028156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b127a-f08f-4c15-9627-9b35f026c672 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.040510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.041522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.042136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.042630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.043476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.045667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.124367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 913/1821] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:29 2022] POST /identity/v3/users => generated 288 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:30.125353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:30.128775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5a2fb-dd48-4f56-abcb-9d7faaf03d76 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.150731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.151411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.151846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.152194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.156628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824896bd-cb33-4c53-a07f-1a9948d60d81 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.158382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:30.178310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.179176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.179661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.180977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.182705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.188113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.202798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.205113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:30.218612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.223059 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 914/1822] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:30.225183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:30.228658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:30.254915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55600170-6162-438d-b2c9-962e05968aa8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.263320 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 909/1823] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:30.265960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:30.266318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.267732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.268299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.268814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] PATH_INFO: `/v3/projects/6f0a69129cee4059b288fa530eb3c195` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.270877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f0a69129cee4059b288fa530eb3c195)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.273317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.319546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f60cf-7e66-41d7-85a2-ad5b29460b31 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.327786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.328960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.329581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.330608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.332090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.334454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.361795 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 910/1824] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:30.363098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:30.371718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f0a69129cee4059b288fa530eb3c195', 'request_id': 'req-bb963ee7-d575-43ab-a755-9d871b80756a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:30.381120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f13c2-963f-4bef-84f2-483a7f4c5b7a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.384279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f0a69129cee4059b288fa530eb3c195', 'request_id': 'req-bb963ee7-d575-43ab-a755-9d871b80756a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:30.403800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f0a69129cee4059b288fa530eb3c195', 'request_id': 'req-bb963ee7-d575-43ab-a755-9d871b80756a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:30.442415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] The token cache is being invalidate because project 6f0a69129cee4059b288fa530eb3c195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:30.448975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:30.450410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:30.462917 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 915/1825] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:30 2022] DELETE /identity/v3/projects/6f0a69129cee4059b288fa530eb3c195 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:30.466171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:30.473007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:30.623943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33157c-d74d-439c-a1bb-ce44951871a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.624583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33157c-d74d-439c-a1bb-ce44951871a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.624866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33157c-d74d-439c-a1bb-ce44951871a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.625142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33157c-d74d-439c-a1bb-ce44951871a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.625857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33157c-d74d-439c-a1bb-ce44951871a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.627670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33157c-d74d-439c-a1bb-ce44951871a3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.653312 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 911/1826] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:30.655785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff81a-d669-47c8-b331-47defeb4d630 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.656072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff81a-d669-47c8-b331-47defeb4d630 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.656455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff81a-d669-47c8-b331-47defeb4d630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.696416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb963ee7-d575-43ab-a755-9d871b80756a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.709202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.709810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.710152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.710600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.711684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.715517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.727284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3ddff81a-d669-47c8-b331-47defeb4d630 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:30.729024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3ddff81a-d669-47c8-b331-47defeb4d630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:30.736761 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 916/1827] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:30.742720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:30.746223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:30.752391 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 912/1828] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:30.755461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddff81a-d669-47c8-b331-47defeb4d630 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:30.781764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddff81a-d669-47c8-b331-47defeb4d630 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.806611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec216c-5dea-416e-bd55-6cd181a5f5d0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:30.820168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bce8d-7102-4d25-be39-58cca746cbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.821062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc5630-6ac8-425e-933f-005b7c6ddf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:30.821763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc5630-6ac8-425e-933f-005b7c6ddf94 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.822156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc5630-6ac8-425e-933f-005b7c6ddf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.822513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc5630-6ac8-425e-933f-005b7c6ddf94 admin admin] PATH_INFO: `/v3/projects/4f0c0e38e3914dab88bf49957bd85708/users/4d9dac03431042fdb0471dd37639e70d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.823879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bce8d-7102-4d25-be39-58cca746cbcb admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.824368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bce8d-7102-4d25-be39-58cca746cbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.824805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bce8d-7102-4d25-be39-58cca746cbcb admin admin] PATH_INFO: `/v3/projects/96b3741511bb4b6d9927841373aaffcf/users/de265dd4263f4c6cbac4f150b7a9d073/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.874697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc5630-6ac8-425e-933f-005b7c6ddf94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0c0e38e3914dab88bf49957bd85708, user_id=4d9dac03431042fdb0471dd37639e70d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.878287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc5630-6ac8-425e-933f-005b7c6ddf94 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.895608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bce8d-7102-4d25-be39-58cca746cbcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b3741511bb4b6d9927841373aaffcf, user_id=de265dd4263f4c6cbac4f150b7a9d073, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:30.909348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bce8d-7102-4d25-be39-58cca746cbcb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:30.921754 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 913/1829] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:30 2022] PUT /identity/v3/projects/4f0c0e38e3914dab88bf49957bd85708/users/4d9dac03431042fdb0471dd37639e70d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:30.923668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04364062-df3f-410a-9a83-eb6abd894012 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.923668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04364062-df3f-410a-9a83-eb6abd894012 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.923903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04364062-df3f-410a-9a83-eb6abd894012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:30.941527 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 917/1830] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:30 2022] PUT /identity/v3/projects/96b3741511bb4b6d9927841373aaffcf/users/de265dd4263f4c6cbac4f150b7a9d073/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:30.943382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb898379-021e-4297-b62a-2009f87adc52 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:30.943889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb898379-021e-4297-b62a-2009f87adc52 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:30.944416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb898379-021e-4297-b62a-2009f87adc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.023482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-eb898379-021e-4297-b62a-2009f87adc52 None None] MFA Rules not processed for user `4d9dac03431042fdb0471dd37639e70d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:31.025389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-04364062-df3f-410a-9a83-eb6abd894012 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:31.028145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eb898379-021e-4297-b62a-2009f87adc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:31.030031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-04364062-df3f-410a-9a83-eb6abd894012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:31.135081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:31.145473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:31.205743 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 914/1831] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:31.208024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54c88-5160-40a7-bee7-0da749fe1e44 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.209007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54c88-5160-40a7-bee7-0da749fe1e44 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.209905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54c88-5160-40a7-bee7-0da749fe1e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.250892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 918/1832] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:31.250892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb898379-021e-4297-b62a-2009f87adc52 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.257902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3be54c88-5160-40a7-bee7-0da749fe1e44 None None] MFA Rules not processed for user `de265dd4263f4c6cbac4f150b7a9d073`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:31.260336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3be54c88-5160-40a7-bee7-0da749fe1e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:31.274363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb898379-021e-4297-b62a-2009f87adc52 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.290344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.290983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.291493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.291920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.295285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:31.295485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:31.297136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:31.327718 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 919/1833] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:31.330099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.354197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00643b7d-c7a8-4bf2-9d8b-d8936dc3d044 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.370085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.371801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.372110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.372391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.375538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:31.379958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:31.453801 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 920/1834] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:31 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:31.458019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.485110 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 915/1835] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:31.487179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54c88-5160-40a7-bee7-0da749fe1e44 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.520561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54c88-5160-40a7-bee7-0da749fe1e44 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.632092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6508a-9823-470a-84f3-9e20fcc5fa22 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.649337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.650197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.650481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.650813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.651591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:31.658646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:31.709409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.712996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.713305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.713595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.714364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:31.724610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:31.773816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 916/1836] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:31 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:31.774968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.783005 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 921/1837] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:31 2022] POST /identity/v3/projects => generated 397 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:31.783920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:31.797939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc70f5-abfc-4fb3-b984-7c06b19d8b33 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.937943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.938751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.939435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.939805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:31.940873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:31.943361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:31.957161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:31.969095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b90b5-bc48-4d69-b19c-ee229e024c53 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:31.992255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:31.994757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:31.996186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:31.996581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.002816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.003002 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 917/1838] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:31 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:32.003457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.004425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.031525 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 922/1839] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:32.032777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.068649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e5c7-ac3a-493b-8ee3-7552a69c456c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.176600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:32.205883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e5579-38ac-4f90-8a69-aa0d22267917 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.218250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.219600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.220227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.221755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.223626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.226803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.290259 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 918/1840] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:32 2022] POST /identity/v3/users => generated 299 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:32.291478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.292265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.292553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.292927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.294576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.297332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.300521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.322462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e291-aaa8-406f-86a3-1fed118a8ae0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.346121 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 923/1841] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:32 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:32.347424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.347596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.348428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.348828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.349146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] PATH_INFO: `/v3/projects/8e3a52bb0e284901a617bc95fdeda3e6/users/e56a8d8d4a384d9fa32951e5ec4a07ba/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.359473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6503c-3ab8-4caf-8fd1-d46af0fa9e82 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.380620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3a52bb0e284901a617bc95fdeda3e6, user_id=e56a8d8d4a384d9fa32951e5ec4a07ba, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.381005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.382030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.382516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.383672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.384233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.384742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.386996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.408283 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 924/1842] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:32.409624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.429260 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 919/1843] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:32 2022] PUT /identity/v3/projects/8e3a52bb0e284901a617bc95fdeda3e6/users/e56a8d8d4a384d9fa32951e5ec4a07ba/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:32.432050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.440008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac32f4-eb42-4ad4-9ba5-c74308b9d8ce admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.616262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5660b-8c7e-4c9a-97b2-eb667ec847a6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.628300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.629205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.630168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.630612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.631536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.633613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.653584 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 920/1844] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:32.658475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.666608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.667460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.667836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.668209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.668985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.671889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.683868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4c48e-2f0c-46c1-875f-19828eb8bab4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.698669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.699667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.700501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.701806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.702819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.703298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 925/1845] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:32.704258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.707301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.740727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a29628-96f2-46d4-8462-ca29f5f4644e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.747713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 921/1846] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:32.749522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.787360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41f7b4-7199-412c-a24d-fe4c3aec4c75 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:32.801552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.810789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.811180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:32.811561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.812769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:32.814878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:32.815311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] PATH_INFO: `/v3/projects/89075fefc8ea47c78d4ff7364e12ffde/users/dd431fc2d9834bea8df807edb91771ee/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.815934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] PATH_INFO: `/v3/projects/e78431e6a1ff46b3996093ea6e535afc/users/90c1fadf4983407d9520460dc7d99301/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:32.859189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78431e6a1ff46b3996093ea6e535afc, user_id=90c1fadf4983407d9520460dc7d99301, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.870548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.874381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89075fefc8ea47c78d4ff7364e12ffde, user_id=dd431fc2d9834bea8df807edb91771ee, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:32.877441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:32.913196 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 922/1847] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:32 2022] PUT /identity/v3/projects/e78431e6a1ff46b3996093ea6e535afc/users/90c1fadf4983407d9520460dc7d99301/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:32.915386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.925085 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 926/1848] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:32 2022] PUT /identity/v3/projects/89075fefc8ea47c78d4ff7364e12ffde/users/dd431fc2d9834bea8df807edb91771ee/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:32.926138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:32.956732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:33.153506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43a841-45f8-44dc-8d37-8f1e91a89061 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:33.158436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38509b1-e8d7-40fa-983a-1a3dfb382dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:33.159068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38509b1-e8d7-40fa-983a-1a3dfb382dbf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.159350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38509b1-e8d7-40fa-983a-1a3dfb382dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.159771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38509b1-e8d7-40fa-983a-1a3dfb382dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.160433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38509b1-e8d7-40fa-983a-1a3dfb382dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:33.162068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38509b1-e8d7-40fa-983a-1a3dfb382dbf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:33.168267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa37ca-7ed6-47fa-b1e5-9c75b0dc8823 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:33.188568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:33.192220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.192361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 927/1849] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:33.192454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.192454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] PATH_INFO: `/v3/projects/8e3a52bb0e284901a617bc95fdeda3e6/users/e56a8d8d4a384d9fa32951e5ec4a07ba/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.192635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbbd64-6b0a-4d03-8311-47d6a9ba818c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.192635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbbd64-6b0a-4d03-8311-47d6a9ba818c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.192635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbbd64-6b0a-4d03-8311-47d6a9ba818c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.198916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3a52bb0e284901a617bc95fdeda3e6, user_id=e56a8d8d4a384d9fa32951e5ec4a07ba, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:33.201641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:33.254547 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 923/1850] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:32 2022] PUT /identity/v3/projects/8e3a52bb0e284901a617bc95fdeda3e6/users/e56a8d8d4a384d9fa32951e5ec4a07ba/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:33.255584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:33.267076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-49bbbd64-6b0a-4d03-8311-47d6a9ba818c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:33.268379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-49bbbd64-6b0a-4d03-8311-47d6a9ba818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:33.314934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:33.390360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aefeef-8212-4fc6-a660-5f4fa9c5c196 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:33.400622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:33.401303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.401582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.402086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] PATH_INFO: `/v3/projects/e78431e6a1ff46b3996093ea6e535afc/users/90c1fadf4983407d9520460dc7d99301/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.406847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78431e6a1ff46b3996093ea6e535afc, user_id=90c1fadf4983407d9520460dc7d99301, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:33.412150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:33.439859 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 928/1851] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:33.454246 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 924/1852] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:33 2022] PUT /identity/v3/projects/e78431e6a1ff46b3996093ea6e535afc/users/90c1fadf4983407d9520460dc7d99301/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:33.456570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:33.456997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32902cf-1c8d-449d-9ed7-1f381eba6d6e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.467682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32902cf-1c8d-449d-9ed7-1f381eba6d6e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.468586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32902cf-1c8d-449d-9ed7-1f381eba6d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.473382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:33.573785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d32902cf-1c8d-449d-9ed7-1f381eba6d6e None None] MFA Rules not processed for user `e56a8d8d4a384d9fa32951e5ec4a07ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:33.575116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d32902cf-1c8d-449d-9ed7-1f381eba6d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:33.657630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab29fee-f1b0-4696-9877-24adc9e068d9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:33.662476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:33.663096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.663373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.663642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.776893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:33.797258 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 929/1853] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:33.799339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad092674-7098-4b98-b367-62f934f6da96 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:33.799719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad092674-7098-4b98-b367-62f934f6da96 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:33.800101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad092674-7098-4b98-b367-62f934f6da96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:33.808536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:33.810559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:33.845020 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 925/1854] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:33.851601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:33.855757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:33.914978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ad092674-7098-4b98-b367-62f934f6da96 None None] MFA Rules not processed for user `90c1fadf4983407d9520460dc7d99301`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:33.915862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad092674-7098-4b98-b367-62f934f6da96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:34.040990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474340-608a-456c-b7fe-be7a00e9656d service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.076918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.080381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.080654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.080927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.101354 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 930/1855] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:34.101354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345574-9185-43f8-ae39-06972c0991db None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.101354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345574-9185-43f8-ae39-06972c0991db None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.101354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345574-9185-43f8-ae39-06972c0991db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.109166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.114696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.129106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-41345574-9185-43f8-ae39-06972c0991db None None] MFA Rules not processed for user `a020e015186b4c32950c2d4edd26374e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:34.130507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-41345574-9185-43f8-ae39-06972c0991db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:34.139807 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 926/1856] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:34.140702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.166147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5004ac2-5821-40d0-b1f2-ebd667d7b576 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.168077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:34.181599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.182299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.182597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.182885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.184055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:34.184055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.185596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.209365 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 927/1857] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:34.214370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.238992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9480d4-b601-412d-b129-cff7fac0d02b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.250089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.250676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.250863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.251084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.251730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.253670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.266661 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 931/1858] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:34.411631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 928/1859] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:34 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:48:34.416256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345574-9185-43f8-ae39-06972c0991db None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.446083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345574-9185-43f8-ae39-06972c0991db None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.656354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677e477-fd76-4786-b068-6498c69cab5f tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.657864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677e477-fd76-4786-b068-6498c69cab5f tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.660501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677e477-fd76-4786-b068-6498c69cab5f tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.660896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677e477-fd76-4786-b068-6498c69cab5f tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.662437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677e477-fd76-4786-b068-6498c69cab5f tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:34.662861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677e477-fd76-4786-b068-6498c69cab5f tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.666416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677e477-fd76-4786-b068-6498c69cab5f tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.702169 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 932/1860] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:34.752509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:34.919548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816bcf-1d3a-4762-9f1e-ee36d6316416 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:34.926235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:34.927206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:34.927534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:34.927921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:34.950469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:34.952636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:34.968183 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 929/1861] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:34.999814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eea4f4-ffaa-4116-ae0d-17d2c51bbe66 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.000339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eea4f4-ffaa-4116-ae0d-17d2c51bbe66 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.000339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eea4f4-ffaa-4116-ae0d-17d2c51bbe66 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.001870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 933/1862] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:35.010143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:35.023553 ncdv-master-1 devstack@keystone.service[93527]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 10 20:48:35.023958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:35.035340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2226-a940-4047-8583-ac3ec48d2ae9 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:35.051417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d0c9c-0698-46e7-ae2d-bc5388f44d92 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:35.052505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d0c9c-0698-46e7-ae2d-bc5388f44d92 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.055341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d0c9c-0698-46e7-ae2d-bc5388f44d92 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.056062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d0c9c-0698-46e7-ae2d-bc5388f44d92 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/64c9116ca73f41289a741b948b6b8a25` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.057857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d0c9c-0698-46e7-ae2d-bc5388f44d92 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=64c9116ca73f41289a741b948b6b8a25)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:35.060598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d0c9c-0698-46e7-ae2d-bc5388f44d92 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:35.063722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 930/1863] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:35 2022] GET /identity/v3/projects/64c9116ca73f41289a741b948b6b8a25 => generated 371 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:35.279052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d2850-415e-4cc7-876a-af99d50e6afe None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.279784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d2850-415e-4cc7-876a-af99d50e6afe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.281165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d2850-415e-4cc7-876a-af99d50e6afe None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.283046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 934/1864] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:35.290784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27012-745d-4e49-a566-b6856d69f0da None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.291005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27012-745d-4e49-a566-b6856d69f0da None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.291405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27012-745d-4e49-a566-b6856d69f0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.313389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8dc27012-745d-4e49-a566-b6856d69f0da None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:35.314659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8dc27012-745d-4e49-a566-b6856d69f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:35.333881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:35.381801 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 931/1865] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:35.394155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d2850-415e-4cc7-876a-af99d50e6afe None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:35.409629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d2850-415e-4cc7-876a-af99d50e6afe None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:35.415687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:35.418121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.418121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.418121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.437382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:35.439650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:35.455166 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 935/1866] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:35.677688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9468fee-0f71-4af9-bda3-b4ca59b3ebd6 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.678253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9468fee-0f71-4af9-bda3-b4ca59b3ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.678708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9468fee-0f71-4af9-bda3-b4ca59b3ebd6 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.680515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 932/1867] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:35.686668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:35.710562 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 10 20:48:35.712457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:35.747391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bae29-09c5-4850-976b-251022fd0c8a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:35.754348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:35.756052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:35.756692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:35.757141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/64c9116ca73f41289a741b948b6b8a25` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:35.759035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=64c9116ca73f41289a741b948b6b8a25)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:35.761660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:35.768514 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 936/1868] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:35 2022] GET /identity/v3/projects/64c9116ca73f41289a741b948b6b8a25 => generated 371 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:36.048204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57efa72-575d-4c47-ae1a-d565d06752c5 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:36.048837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57efa72-575d-4c47-ae1a-d565d06752c5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:36.049042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57efa72-575d-4c47-ae1a-d565d06752c5 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:36.050620 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 933/1869] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:48:36.060377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:36.082363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:36.093352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19fd2-5518-4b62-9a15-fd24b41d5dcf tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:36.125469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:36.127386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:36.129602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:36.130252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/64c9116ca73f41289a741b948b6b8a25` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:36.138777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57efa72-575d-4c47-ae1a-d565d06752c5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:36.160280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=64c9116ca73f41289a741b948b6b8a25)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:36.164863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:36.173189 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 937/1870] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:36 2022] GET /identity/v3/projects/64c9116ca73f41289a741b948b6b8a25 => generated 371 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:36.219642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:36.234785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1432080-c5b5-4bd0-ab10-bcfd3716558b tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:36.247289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:36.248447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:36.249002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:36.249402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/64c9116ca73f41289a741b948b6b8a25` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:36.250812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:delete_project(project_id=64c9116ca73f41289a741b948b6b8a25)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:36.252975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:36.300431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57efa72-575d-4c47-ae1a-d565d06752c5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:36.316154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:36.317815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:36.319403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:36.320001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] PATH_INFO: `/v3/users/4d9dac03431042fdb0471dd37639e70d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:36.322580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9dac03431042fdb0471dd37639e70d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:36.324946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:36.405885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c9116ca73f41289a741b948b6b8a25', 'request_id': 'req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.426203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9dac03431042fdb0471dd37639e70d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.436851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c9116ca73f41289a741b948b6b8a25', 'request_id': 'req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.450447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c9116ca73f41289a741b948b6b8a25', 'request_id': 'req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.461130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9dac03431042fdb0471dd37639e70d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.479051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9dac03431042fdb0471dd37639e70d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.491181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9dac03431042fdb0471dd37639e70d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.492966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d09499-a3f0-469d-b9e5-464f8ee0b8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9dac03431042fdb0471dd37639e70d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.509244 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 934/1871] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:36 2022] DELETE /identity/v3/users/4d9dac03431042fdb0471dd37639e70d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:36.511134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:36.511134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:36.511382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:36.549029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] The token cache is being invalidate because project 64c9116ca73f41289a741b948b6b8a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:36.549730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.551525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967dbb56-2b7b-43ee-8cdb-13dacf5b4b6d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:36.556168 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 938/1872] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:36 2022] DELETE /identity/v3/projects/64c9116ca73f41289a741b948b6b8a25 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:36.556728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:36.558177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:36.576683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:36.576973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:36.577253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:36.580113 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 939/1873] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:48:36.592851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:36.595620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:36.660304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:36.717181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:36.798901 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 935/1874] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:36.845689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:36.847640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:36.871473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:36.882241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:37.059206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20033006-10ab-4e6e-b2d5-9e32af461f2e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.066774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9400932-0276-40bc-88a0-fa9bc65724d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.067953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9400932-0276-40bc-88a0-fa9bc65724d2 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.071856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9400932-0276-40bc-88a0-fa9bc65724d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.073279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9400932-0276-40bc-88a0-fa9bc65724d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.094783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9400932-0276-40bc-88a0-fa9bc65724d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.094947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32281f5-4ed7-46c1-b353-56827def99fd None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.098409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9400932-0276-40bc-88a0-fa9bc65724d2 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.112264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.116725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.117540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.118098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/96b3741511bb4b6d9927841373aaffcf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.122784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=96b3741511bb4b6d9927841373aaffcf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.129508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.140771 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 940/1875] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:36 2022] GET /identity/v3/projects/96b3741511bb4b6d9927841373aaffcf => generated 375 bytes in 549 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:37.167633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:37.213193 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 936/1876] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:37.216726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f140e-f423-4ea0-afef-e8c94d9f3049 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.243139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.244035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.244614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.244987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.246035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:37.246463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.248501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.291353 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 941/1877] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:37.295669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:37.297135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada219-8405-4f10-97ee-80ec424689d8 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.297529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada219-8405-4f10-97ee-80ec424689d8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.298618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada219-8405-4f10-97ee-80ec424689d8 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.300735 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 937/1878] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:48:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:37.313324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db76e0d-0379-4036-bf1f-2656ab9c040d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.332050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.333155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.333648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.334542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.335168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.335454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.335736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.339494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.348066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.379236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:37.380762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:37.412475 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 942/1879] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:37 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:37.416243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:37.424983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 938/1880] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:48:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:37.448745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:37.450911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:37.482760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d0542-7b33-4e53-8cd4-54a47ad52b62 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.565972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e46392-baed-41fe-98c8-8b8bead0c1a6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.569104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.569725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.571630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.572109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.573462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a1e16-5cfb-42e6-9a0e-fe850139de91 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.574400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1e16-5cfb-42e6-9a0e-fe850139de91 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.574907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1e16-5cfb-42e6-9a0e-fe850139de91 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.575394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1e16-5cfb-42e6-9a0e-fe850139de91 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.576253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1e16-5cfb-42e6-9a0e-fe850139de91 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:37.576760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1e16-5cfb-42e6-9a0e-fe850139de91 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.579776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1e16-5cfb-42e6-9a0e-fe850139de91 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.583709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.585466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.598367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 943/1881] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:37.603937 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 939/1882] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:37.720038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9cf5-5d3c-40b3-9f4a-d71f9b892e31 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.720147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9cf5-5d3c-40b3-9f4a-d71f9b892e31 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.720637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f9cf5-5d3c-40b3-9f4a-d71f9b892e31 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.722219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 944/1883] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:37.727325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:37.739052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:37.750858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316d0b6-decc-4b36-8447-e371396f8270 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.756116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.756870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.757156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.757430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/900e5c0658564fdb9f109f03bd66a0e1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.759597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=900e5c0658564fdb9f109f03bd66a0e1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.761662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.766017 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 940/1884] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:37 2022] GET /identity/v3/projects/900e5c0658564fdb9f109f03bd66a0e1 => generated 371 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:37.861182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c4ee-6d7c-44fa-8af8-5c7124e841fa None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.861636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c4ee-6d7c-44fa-8af8-5c7124e841fa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.862260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c4ee-6d7c-44fa-8af8-5c7124e841fa None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.863775 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 945/1885] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:37.870081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:37.890050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:37.900619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711033f4-3a43-421d-abfe-d439609d6044 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:37.906134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:37.906811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:37.907758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:37.908054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/900e5c0658564fdb9f109f03bd66a0e1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:37.909373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=900e5c0658564fdb9f109f03bd66a0e1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:37.912303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:37.915432 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 941/1886] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:37 2022] GET /identity/v3/projects/900e5c0658564fdb9f109f03bd66a0e1 => generated 371 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:37.983113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c4ee-6d7c-44fa-8af8-5c7124e841fa None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.004303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c4ee-6d7c-44fa-8af8-5c7124e841fa None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.011222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.012710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.013630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.013630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.014258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:38.014673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.016853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.036954 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 946/1887] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:38.038792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.053052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455ee54-c92d-4148-bfa3-22278d9a21cd tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.060700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b17347-5d75-4b4b-8ee9-29739ac3c970 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.061384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17347-5d75-4b4b-8ee9-29739ac3c970 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.061602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17347-5d75-4b4b-8ee9-29739ac3c970 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.061947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b17347-5d75-4b4b-8ee9-29739ac3c970 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.062700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b17347-5d75-4b4b-8ee9-29739ac3c970 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.064658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b17347-5d75-4b4b-8ee9-29739ac3c970 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.103441 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 942/1888] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:38 2022] POST /identity/v3/users => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:38.104053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.112932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:38.127747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0035c6-7923-428c-bcaf-a35c6e3f46c2 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.135510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.136580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.136580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.136926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.137684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:38.138810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.141913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.204445 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 947/1889] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:38.222261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fe9c8-5159-4a25-bcf7-b4b9d9820594 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.222786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fe9c8-5159-4a25-bcf7-b4b9d9820594 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.223046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fe9c8-5159-4a25-bcf7-b4b9d9820594 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.224731 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 943/1890] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:38.234164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:38.254588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.265738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3729cf1-8b49-4644-93de-a4f23a9c8fe6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.273814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.274536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.274822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.275153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/900e5c0658564fdb9f109f03bd66a0e1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.278602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=900e5c0658564fdb9f109f03bd66a0e1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.281192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.282857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 948/1891] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:38 2022] GET /identity/v3/projects/900e5c0658564fdb9f109f03bd66a0e1 => generated 371 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:38.575367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84486bf1-1173-41d3-9332-1f254ca285f7 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.575367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84486bf1-1173-41d3-9332-1f254ca285f7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.575367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84486bf1-1173-41d3-9332-1f254ca285f7 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.575367 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 944/1892] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:38.591911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:38.614057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.637858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833ee7b-d42e-4419-81eb-5a24293cf29a tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.643515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.644165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.644454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.644737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/900e5c0658564fdb9f109f03bd66a0e1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.646069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=900e5c0658564fdb9f109f03bd66a0e1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.648171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.656239 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 949/1893] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:38 2022] GET /identity/v3/projects/900e5c0658564fdb9f109f03bd66a0e1 => generated 371 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:38.689936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84486bf1-1173-41d3-9332-1f254ca285f7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.716739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84486bf1-1173-41d3-9332-1f254ca285f7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.722586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.723412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.723802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.724194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/users/2600406ea73648b2acb1e759cf8ba7ce` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.744364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.748842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:38.777299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:delete_user(user_id=2600406ea73648b2acb1e759cf8ba7ce)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.780071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.817897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:38.872843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2600406ea73648b2acb1e759cf8ba7ce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:38.879435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683cc49-310e-4e93-8fd0-9c026f3b926d tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:38.885026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:38.885862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:38.886260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:38.886384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] PATH_INFO: `/v3/projects/4f0c0e38e3914dab88bf49957bd85708` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:38.887664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0c0e38e3914dab88bf49957bd85708)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:38.889962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:38.913584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2600406ea73648b2acb1e759cf8ba7ce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:38.935911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2600406ea73648b2acb1e759cf8ba7ce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:38.948212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2600406ea73648b2acb1e759cf8ba7ce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:38.949279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2600406ea73648b2acb1e759cf8ba7ce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:38.959570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0c0e38e3914dab88bf49957bd85708', 'request_id': 'req-13e96470-47a8-4f30-b590-362457574d78'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:38.968110 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 945/1894] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:38 2022] DELETE /identity/v3/users/2600406ea73648b2acb1e759cf8ba7ce => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:38.969139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0c0e38e3914dab88bf49957bd85708', 'request_id': 'req-13e96470-47a8-4f30-b590-362457574d78'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:38.971912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:38.979591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:38.991359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0c0e38e3914dab88bf49957bd85708', 'request_id': 'req-13e96470-47a8-4f30-b590-362457574d78'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.011818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] The token cache is being invalidate because project 4f0c0e38e3914dab88bf49957bd85708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:39.012230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.013089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.015519 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 950/1895] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:38 2022] DELETE /identity/v3/projects/4f0c0e38e3914dab88bf49957bd85708 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:39.022883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:39.025504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:39.081007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9901e00-6300-4d38-999e-88c2136c8449 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:39.086889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:39.087718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:39.088106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:39.088600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/900e5c0658564fdb9f109f03bd66a0e1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:39.090101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:delete_project(project_id=900e5c0658564fdb9f109f03bd66a0e1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:39.092485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:39.125283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e96470-47a8-4f30-b590-362457574d78 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:39.143555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:39.215772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900e5c0658564fdb9f109f03bd66a0e1', 'request_id': 'req-fb4844bf-4256-47b8-b058-08c08c375e66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.236286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900e5c0658564fdb9f109f03bd66a0e1', 'request_id': 'req-fb4844bf-4256-47b8-b058-08c08c375e66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.256241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900e5c0658564fdb9f109f03bd66a0e1', 'request_id': 'req-fb4844bf-4256-47b8-b058-08c08c375e66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.281069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] The token cache is being invalidate because project 900e5c0658564fdb9f109f03bd66a0e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:39.281488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.282163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb4844bf-4256-47b8-b058-08c08c375e66 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.286194 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 946/1896] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:38 2022] DELETE /identity/v3/projects/900e5c0658564fdb9f109f03bd66a0e1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:39.307778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:39.310203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:39.310601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:39.310980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] PATH_INFO: `/v3/users/e56a8d8d4a384d9fa32951e5ec4a07ba` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:39.311951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:39.312251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:39.312530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:39.313639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56a8d8d4a384d9fa32951e5ec4a07ba)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:39.314234 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 947/1897] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:39.321288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:39.325936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:39.328361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:39.384676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:39.432207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56a8d8d4a384d9fa32951e5ec4a07ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.440357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56a8d8d4a384d9fa32951e5ec4a07ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.442195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56a8d8d4a384d9fa32951e5ec4a07ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.453890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56a8d8d4a384d9fa32951e5ec4a07ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.454548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-098404c0-a91a-4e85-8e4b-2edc1acf5083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56a8d8d4a384d9fa32951e5ec4a07ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:39.484367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:39.486896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:39.489872 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 951/1898] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:39 2022] DELETE /identity/v3/users/e56a8d8d4a384d9fa32951e5ec4a07ba => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:39.610036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3a316-1ced-4f7e-9b93-8b6ff9986173 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:39.698335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:39.699262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:39.699890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:39.700179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/96b3741511bb4b6d9927841373aaffcf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:39.701889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=96b3741511bb4b6d9927841373aaffcf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:39.704080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:39.706341 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 948/1899] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:39 2022] GET /identity/v3/projects/96b3741511bb4b6d9927841373aaffcf => generated 375 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:48:39.740192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2319a-c66e-4eec-ab84-a11d3f587781 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:39.740713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2319a-c66e-4eec-ab84-a11d3f587781 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:39.741303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2319a-c66e-4eec-ab84-a11d3f587781 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:39.744273 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 952/1900] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:39.751767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:39.753852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:39.889390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:39.893169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:39.905602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:39.917449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869ecf5-51d3-4995-aae3-2e9b5b500ee6 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:39.926063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:39.927974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:39.928485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:39.928917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/96b3741511bb4b6d9927841373aaffcf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:39.931311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=96b3741511bb4b6d9927841373aaffcf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:39.934761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:39.950006 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 949/1901] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:39 2022] GET /identity/v3/projects/96b3741511bb4b6d9927841373aaffcf => generated 375 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:40.336584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cdaef-54fe-4dcf-8276-cee525d62086 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:40.337908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cdaef-54fe-4dcf-8276-cee525d62086 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:40.338660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cdaef-54fe-4dcf-8276-cee525d62086 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:40.341052 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 953/1902] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:40.347704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:48:40.372811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:40.382590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70657b4c-11b0-4db6-a930-ef564255eead tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:40.387746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3e675-0eca-47d9-89ca-03e8bfe49f68 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c1fadf4983407d9520460dc7d99301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e78431e6a1ff46b3996093ea6e535afc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:40.388394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3e675-0eca-47d9-89ca-03e8bfe49f68 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:40.388681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3e675-0eca-47d9-89ca-03e8bfe49f68 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:40.388962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3e675-0eca-47d9-89ca-03e8bfe49f68 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] PATH_INFO: `/v3/projects/96b3741511bb4b6d9927841373aaffcf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:40.390144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3e675-0eca-47d9-89ca-03e8bfe49f68 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorizing `identity:get_project(project_id=96b3741511bb4b6d9927841373aaffcf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:40.392851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3e675-0eca-47d9-89ca-03e8bfe49f68 tempest-QuotasAdminTestJSON-701082686 tempest-QuotasAdminTestJSON-701082686-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:40.394907 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 950/1903] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:48:40 2022] GET /identity/v3/projects/96b3741511bb4b6d9927841373aaffcf => generated 375 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:40.558267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cdaef-54fe-4dcf-8276-cee525d62086 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:40.560533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af3cdaef-54fe-4dcf-8276-cee525d62086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:40.742568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cdaef-54fe-4dcf-8276-cee525d62086 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:40.765856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:40.770502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:40.770798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:40.774757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] PATH_INFO: `/v3/projects/8e3a52bb0e284901a617bc95fdeda3e6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:40.779866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3a52bb0e284901a617bc95fdeda3e6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:40.782071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:40.866473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:40.893623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3a52bb0e284901a617bc95fdeda3e6', 'request_id': 'req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:40.901796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3a52bb0e284901a617bc95fdeda3e6', 'request_id': 'req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:40.912544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3a52bb0e284901a617bc95fdeda3e6', 'request_id': 'req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:40.927344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] The token cache is being invalidate because project 8e3a52bb0e284901a617bc95fdeda3e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:40.927678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:40.928780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:40.933924 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 954/1904] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:40 2022] DELETE /identity/v3/projects/8e3a52bb0e284901a617bc95fdeda3e6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:40.937321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10ad4-95bd-4b10-9947-f9d89e171ba3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:40.938185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10ad4-95bd-4b10-9947-f9d89e171ba3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:40.938681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10ad4-95bd-4b10-9947-f9d89e171ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:40.969136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:40.975576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:40.975725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-76c10ad4-95bd-4b10-9947-f9d89e171ba3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:40.975725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76c10ad4-95bd-4b10-9947-f9d89e171ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:41.074562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f88db-55a3-4782-a6b4-cc0baf3c7cbe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.082607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.083432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.083823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.084215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] PATH_INFO: `/v3/users/de265dd4263f4c6cbac4f150b7a9d073` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.086465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de265dd4263f4c6cbac4f150b7a9d073)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.088782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.099833 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 951/1905] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:41.107155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8866b36-c480-494f-877b-36004d650e85 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.107447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8866b36-c480-494f-877b-36004d650e85 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.107757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8866b36-c480-494f-877b-36004d650e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.143624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e8866b36-c480-494f-877b-36004d650e85 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:41.145250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e8866b36-c480-494f-877b-36004d650e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:41.156673 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 952/1906] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:48:41.161660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8866b36-c480-494f-877b-36004d650e85 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.176700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8866b36-c480-494f-877b-36004d650e85 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.186159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.189812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.190162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.190437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.191116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:41.191500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.193620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.237920 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 953/1907] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:41.245437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.250762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de265dd4263f4c6cbac4f150b7a9d073'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:41.260753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de265dd4263f4c6cbac4f150b7a9d073'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:41.261241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb5b19-0b2e-4b41-b402-254af0521c59 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.261868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de265dd4263f4c6cbac4f150b7a9d073'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:41.266922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.267539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.267817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.268091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.268720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.270905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.276433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de265dd4263f4c6cbac4f150b7a9d073'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:41.277319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7fac790-cdda-4a45-8fbe-e9d38f2338b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de265dd4263f4c6cbac4f150b7a9d073'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:41.320149 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 954/1909] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:41 2022] POST /identity/v3/projects => generated 403 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:41.320707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.320891 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 955/1909] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:40 2022] DELETE /identity/v3/users/de265dd4263f4c6cbac4f150b7a9d073 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:41.322497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.322960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.323389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.324640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:41.349549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:41.350818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:41.412170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:41.457915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb1e20-677c-48a5-a67a-fe6a11aff9c4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.459648 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 956/1910] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:41.472345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.472946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.473229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.473503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.474396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.480787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.485793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.487366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:41.534911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 955/1911] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:41 2022] POST /identity/v3/users => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:41.538718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.551990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b628e57-8423-4bbf-874f-0371743279e2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.560938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.561697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.562705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.566103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.567436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.569159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.589580 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 956/1912] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:41.596706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.613427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4f86b-42b7-464d-b931-e25ba452df94 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.619472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.620948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.621304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.621650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.622414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.625415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.630038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a58c-2800-4842-9ddb-f2baf7a04ef5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.641261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c219a08-a31a-4794-9a6d-2a7cc1faad4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.642296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c219a08-a31a-4794-9a6d-2a7cc1faad4a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.642576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c219a08-a31a-4794-9a6d-2a7cc1faad4a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.642847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c219a08-a31a-4794-9a6d-2a7cc1faad4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.654271 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 957/1913] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:41.658862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c219a08-a31a-4794-9a6d-2a7cc1faad4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.660090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:41.660789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c219a08-a31a-4794-9a6d-2a7cc1faad4a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.672433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f8b4c-f6a2-42c5-a98b-658a6102862f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:41.673300 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 957/1914] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:41.678407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:41.679077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.679523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.680137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] PATH_INFO: `/v3/projects/60e7e4ba813748518f453fb87b2f0e17/users/f6c7299bc6fd4163b360528217f0b679/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.699048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:41.708057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e7e4ba813748518f453fb87b2f0e17, user_id=f6c7299bc6fd4163b360528217f0b679, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:41.716252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:41.745877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:41.746375 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 958/1915] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:41 2022] PUT /identity/v3/projects/60e7e4ba813748518f453fb87b2f0e17/users/f6c7299bc6fd4163b360528217f0b679/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:41.746772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:41.747089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:41.782979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:41.791888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:41.932998 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 958/1916] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:41.944905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:42.040765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa7784-badf-4e59-8bde-5ecd6545099f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:42.044632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:42.045258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:42.045542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:42.045816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:42.058850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:42.061241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:42.078329 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 959/1917] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:42.129578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:42.131321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:42.146845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314303-e89c-48c3-9b95-ddc07f324a9f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:42.158160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:42.164232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:42.164693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:42.165045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] PATH_INFO: `/v3/projects/96b3741511bb4b6d9927841373aaffcf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:42.166776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b3741511bb4b6d9927841373aaffcf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:42.169162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:42.312043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b3741511bb4b6d9927841373aaffcf', 'request_id': 'req-a39b09f0-cee7-4298-aa20-fc794d82e3f1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.321674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b3741511bb4b6d9927841373aaffcf', 'request_id': 'req-a39b09f0-cee7-4298-aa20-fc794d82e3f1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.352429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b3741511bb4b6d9927841373aaffcf', 'request_id': 'req-a39b09f0-cee7-4298-aa20-fc794d82e3f1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.370323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] The token cache is being invalidate because project 96b3741511bb4b6d9927841373aaffcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:42.370682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.371715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a39b09f0-cee7-4298-aa20-fc794d82e3f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.376399 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 959/1918] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:42 2022] DELETE /identity/v3/projects/96b3741511bb4b6d9927841373aaffcf => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:42.377663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:42.397142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee57f3-e6b2-4fb0-94bb-4d0a5a3ad007 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:42.494001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:42.494664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:42.494997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:42.495355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] PATH_INFO: `/v3/users/90c1fadf4983407d9520460dc7d99301` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:42.497392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c1fadf4983407d9520460dc7d99301)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:42.499691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:42.582396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c1fadf4983407d9520460dc7d99301'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.600989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c1fadf4983407d9520460dc7d99301'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.612359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c1fadf4983407d9520460dc7d99301'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.624083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c1fadf4983407d9520460dc7d99301'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.624739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79a1dccc-ef41-4734-9e07-7c0aaa89d510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c1fadf4983407d9520460dc7d99301'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:42.649628 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 960/1919] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:42 2022] DELETE /identity/v3/users/90c1fadf4983407d9520460dc7d99301 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:43.839259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36cc9-8d49-4686-a565-4531004dadb4 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:43.839831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36cc9-8d49-4686-a565-4531004dadb4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:43.839912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36cc9-8d49-4686-a565-4531004dadb4 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:43.841555 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 960/1920] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:48:43.887589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43eb62-4deb-4481-8c7a-81b495a181d5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:43.888068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43eb62-4deb-4481-8c7a-81b495a181d5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:43.888171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43eb62-4deb-4481-8c7a-81b495a181d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:43.909197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7a43eb62-4deb-4481-8c7a-81b495a181d5 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:43.910533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a43eb62-4deb-4481-8c7a-81b495a181d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:43.920863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:44.030300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36cc9-8d49-4686-a565-4531004dadb4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.032064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-12c36cc9-8d49-4686-a565-4531004dadb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.042179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 961/1921] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:44.046140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:44.114974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36cc9-8d49-4686-a565-4531004dadb4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.120908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.121767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.122086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.122365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] PATH_INFO: `/v3/projects/e78431e6a1ff46b3996093ea6e535afc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.123515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78431e6a1ff46b3996093ea6e535afc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.125477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.176078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78431e6a1ff46b3996093ea6e535afc', 'request_id': 'req-ae8793f1-5166-4050-9837-f49abb2b8015'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:44.182867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78431e6a1ff46b3996093ea6e535afc', 'request_id': 'req-ae8793f1-5166-4050-9837-f49abb2b8015'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:44.191070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78431e6a1ff46b3996093ea6e535afc', 'request_id': 'req-ae8793f1-5166-4050-9837-f49abb2b8015'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:44.203297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] The token cache is being invalidate because project e78431e6a1ff46b3996093ea6e535afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:44.203606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:44.204283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:44.206147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 961/1922] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:44 2022] DELETE /identity/v3/projects/e78431e6a1ff46b3996093ea6e535afc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:48:44.213213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd93e1-d75e-42f0-91d9-c1a8bff09945 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.213502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd93e1-d75e-42f0-91d9-c1a8bff09945 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.213763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd93e1-d75e-42f0-91d9-c1a8bff09945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.223791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.225507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.250661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a9bd93e1-d75e-42f0-91d9-c1a8bff09945 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:44.252043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a9bd93e1-d75e-42f0-91d9-c1a8bff09945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.328783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8793f1-5166-4050-9837-f49abb2b8015 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.329986 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 962/1923] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:44.338227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4578e9-57d2-4b66-b220-6192afd4fdb7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.338928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4578e9-57d2-4b66-b220-6192afd4fdb7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.339225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4578e9-57d2-4b66-b220-6192afd4fdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.346467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.347098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.347380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.347653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.349808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.365049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5d4578e9-57d2-4b66-b220-6192afd4fdb7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:44.366535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.366806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5d4578e9-57d2-4b66-b220-6192afd4fdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.369516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.381140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 963/1924] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:44.382265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4578e9-57d2-4b66-b220-6192afd4fdb7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.385681 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 962/1925] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:44.397078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4578e9-57d2-4b66-b220-6192afd4fdb7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.402971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac7e19-2b45-4157-b594-8411f505b758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.403572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7e19-2b45-4157-b594-8411f505b758 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.403860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7e19-2b45-4157-b594-8411f505b758 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.404126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac7e19-2b45-4157-b594-8411f505b758 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.404774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac7e19-2b45-4157-b594-8411f505b758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:48:44.405567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac7e19-2b45-4157-b594-8411f505b758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.408311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac7e19-2b45-4157-b594-8411f505b758 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.424430 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 964/1926] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:48:44.428739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.444693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79031a1f-d8cb-4335-aca1-2e6b8d4a3bad service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.453683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.454337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.454615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.454889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.455534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.457554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.485242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:44.501157 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 963/1927] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/projects => generated 395 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:48:44.505288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.505576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.506602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.508850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.547134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] MFA Rules not processed for user `dd431fc2d9834bea8df807edb91771ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:44.548354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.591533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2fe66-48c1-40ef-a6e2-9c9e8d8f29bc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.603867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.604510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.604795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.605072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.605852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.612522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.661585 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 964/1928] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:44.666316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.678089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ffc3d-d10c-4030-9af6-9ff41bfc456b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.684713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.685267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.685558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.685864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.686524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.688448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.693813 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 965/1929] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:44.699971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.704047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.711961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 965/1930] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:48:44.715541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.732125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7f269-b43c-41d1-b127-61418f1c106a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.738568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.739552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.740046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.740644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.741729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.743754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714ab29-889c-47ea-9f7d-3a407356908f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.745372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.757237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.757954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.758223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.759358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.759358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.761118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.775734 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 966/1931] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:44.777228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.790051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be779-fdf1-460c-8ab9-4633c5c1ffdc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:44.796727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c85b2-700d-405d-bd42-633f259e3cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:44.797896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c85b2-700d-405d-bd42-633f259e3cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.798391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c85b2-700d-405d-bd42-633f259e3cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.798953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c85b2-700d-405d-bd42-633f259e3cbb admin admin] PATH_INFO: `/v3/projects/d9204ddaf90a4685a4286c8ebfd76433/users/ffba8e54e6f14960b7f57c5e19d3cfae/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.811600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:44.831164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c85b2-700d-405d-bd42-633f259e3cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9204ddaf90a4685a4286c8ebfd76433, user_id=ffba8e54e6f14960b7f57c5e19d3cfae, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:44.833862 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 966/1932] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/projects => generated 397 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:44.834992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:44.838617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c85b2-700d-405d-bd42-633f259e3cbb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:44.860379 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 967/1933] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:44 2022] PUT /identity/v3/projects/d9204ddaf90a4685a4286c8ebfd76433/users/ffba8e54e6f14960b7f57c5e19d3cfae/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:44.884148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b38ae-f310-4c4c-b5a8-fe59c94d8bdb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:44.884847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b38ae-f310-4c4c-b5a8-fe59c94d8bdb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:44.885316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b38ae-f310-4c4c-b5a8-fe59c94d8bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:44.936418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-358b38ae-f310-4c4c-b5a8-fe59c94d8bdb None None] MFA Rules not processed for user `ffba8e54e6f14960b7f57c5e19d3cfae`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:44.938234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-358b38ae-f310-4c4c-b5a8-fe59c94d8bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:44.982078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62b7e8-7efc-4c2b-ac2f-29ca14cce76a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.078718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.080086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.080909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.082261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.083516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.087611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.112378 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 968/1934] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:45.114537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b38ae-f310-4c4c-b5a8-fe59c94d8bdb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.125374 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 967/1935] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:44 2022] POST /identity/v3/users => generated 305 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:45.129320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b38ae-f310-4c4c-b5a8-fe59c94d8bdb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.129446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.153520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.154128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ffd03-68ff-4646-b783-d83632ac18f9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.155023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.155419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.155749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.160505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.168301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.168907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.170099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.170740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.171072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.172626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.177477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.204318 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 968/1936] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:45.210494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.247955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e87ad3-d1e8-4fb5-9614-250d53cb0e4a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.249301 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 969/1937] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:45 2022] POST /identity/v3/projects => generated 395 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:45.250253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.277919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:45.395160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d17c68-d2d5-4eaa-843b-11898beea2c6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.397396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.398693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.398990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.399323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] PATH_INFO: `/v3/projects/c9ea92bd4cef4840b6e96b68f28099de/users/4defc9e17db743d7b79e732a0f695ee6/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.403489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.404196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.404403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.404683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.405327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.408160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.417254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ea92bd4cef4840b6e96b68f28099de, user_id=4defc9e17db743d7b79e732a0f695ee6, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.420351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.442775 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 970/1938] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:45 2022] POST /identity/v3/users => generated 304 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:45.443947 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 969/1939] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:45 2022] PUT /identity/v3/projects/c9ea92bd4cef4840b6e96b68f28099de/users/4defc9e17db743d7b79e732a0f695ee6/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:45.444639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.451338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.457069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:45.461768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a95f-db9d-4ffc-93d1-c9ccdf615e86 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.544674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81754252-f436-4c48-a383-f98e0d0e615c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.552576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.556173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.556587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.557536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.558662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.560842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.562363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.563084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.563467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.565978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.566883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.568570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.590722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 971/1940] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:45.593981 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 970/1941] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:45.595237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.596554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.607226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600599c-5b76-48cc-b928-d6b2f7955563 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.612721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.613478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.614185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.614545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] PATH_INFO: `/v3/projects/e4fdbb588d1a4dbeae77a55d6710d030/users/0ea7f93584594749bb70e39d45cf44b3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.632369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fdbb588d1a4dbeae77a55d6710d030, user_id=0ea7f93584594749bb70e39d45cf44b3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.636927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.641151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aaa29-3ea8-4281-b31c-74ff210b0b90 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.648658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.649515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.650004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.650285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] PATH_INFO: `/v3/projects/c9ea92bd4cef4840b6e96b68f28099de/users/4defc9e17db743d7b79e732a0f695ee6/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.653613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ea92bd4cef4840b6e96b68f28099de, user_id=4defc9e17db743d7b79e732a0f695ee6, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.656698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.668269 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 971/1942] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:45 2022] PUT /identity/v3/projects/e4fdbb588d1a4dbeae77a55d6710d030/users/0ea7f93584594749bb70e39d45cf44b3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:45.672567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.681572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 972/1943] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:45 2022] PUT /identity/v3/projects/c9ea92bd4cef4840b6e96b68f28099de/users/4defc9e17db743d7b79e732a0f695ee6/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:45.794101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44cd3b-936e-4648-9875-d841ba6def30 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.915053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93901c4b-9671-4742-a23a-37b58788e80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.916180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901c4b-9671-4742-a23a-37b58788e80c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.916604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901c4b-9671-4742-a23a-37b58788e80c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.917017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93901c4b-9671-4742-a23a-37b58788e80c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.918956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93901c4b-9671-4742-a23a-37b58788e80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.920652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93901c4b-9671-4742-a23a-37b58788e80c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:45.940108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 972/1944] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:45.941167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:45.958803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c37c3-439d-4f9e-8890-199ebaea8635 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:45.985137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5fa61-2f26-4527-bf55-44aeca586ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:45.985849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fa61-2f26-4527-bf55-44aeca586ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:45.986128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fa61-2f26-4527-bf55-44aeca586ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:45.986397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5fa61-2f26-4527-bf55-44aeca586ac4 admin admin] PATH_INFO: `/v3/projects/e4fdbb588d1a4dbeae77a55d6710d030/users/0ea7f93584594749bb70e39d45cf44b3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:45.989419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5fa61-2f26-4527-bf55-44aeca586ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fdbb588d1a4dbeae77a55d6710d030, user_id=0ea7f93584594749bb70e39d45cf44b3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:45.993637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5fa61-2f26-4527-bf55-44aeca586ac4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:46.025031 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 973/1945] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:45 2022] PUT /identity/v3/projects/e4fdbb588d1a4dbeae77a55d6710d030/users/0ea7f93584594749bb70e39d45cf44b3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:46.027788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1537c91-d930-4983-a528-2cc5d397ba54 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:46.028238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1537c91-d930-4983-a528-2cc5d397ba54 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:46.028658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1537c91-d930-4983-a528-2cc5d397ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:46.123930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d1537c91-d930-4983-a528-2cc5d397ba54 None None] MFA Rules not processed for user `0ea7f93584594749bb70e39d45cf44b3`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:46.127102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d1537c91-d930-4983-a528-2cc5d397ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:46.145219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:46.266300 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 973/1946] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:46.280664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c5d6-3b5f-4859-be93-fddf2c667fac None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:46.281111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c5d6-3b5f-4859-be93-fddf2c667fac None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:46.281352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c5d6-3b5f-4859-be93-fddf2c667fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:46.319871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b296c5d6-3b5f-4859-be93-fddf2c667fac None None] MFA Rules not processed for user `ffba8e54e6f14960b7f57c5e19d3cfae`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:46.323673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b296c5d6-3b5f-4859-be93-fddf2c667fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:46.488276 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 974/1947] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:46.588465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1537c91-d930-4983-a528-2cc5d397ba54 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:46.734883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1537c91-d930-4983-a528-2cc5d397ba54 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:46.741668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec7f52-2158-471c-80f6-7a7ca2d0b299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:46.745523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7f52-2158-471c-80f6-7a7ca2d0b299 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:46.745523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7f52-2158-471c-80f6-7a7ca2d0b299 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:46.745523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7f52-2158-471c-80f6-7a7ca2d0b299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:46.763569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec7f52-2158-471c-80f6-7a7ca2d0b299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:46.765262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec7f52-2158-471c-80f6-7a7ca2d0b299 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:46.792975 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 974/1948] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:46 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:50.221416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69bd9a-3867-4bcd-92c3-24d74dc4da6c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:50.222341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69bd9a-3867-4bcd-92c3-24d74dc4da6c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:50.222761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69bd9a-3867-4bcd-92c3-24d74dc4da6c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:50.224469 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 975/1949] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:48:50.233388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:50.234218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:50.234677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:50.270830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:50.272403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:50.424616 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 975/1950] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:50.460382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69bd9a-3867-4bcd-92c3-24d74dc4da6c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:50.462767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c69bd9a-3867-4bcd-92c3-24d74dc4da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:50.504058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69bd9a-3867-4bcd-92c3-24d74dc4da6c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:50.517416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810d37a-2f4e-47d7-b122-bbd5a79a3874 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:50.518095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810d37a-2f4e-47d7-b122-bbd5a79a3874 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:50.518377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810d37a-2f4e-47d7-b122-bbd5a79a3874 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:50.518646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810d37a-2f4e-47d7-b122-bbd5a79a3874 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:50.540734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810d37a-2f4e-47d7-b122-bbd5a79a3874 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:50.542390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810d37a-2f4e-47d7-b122-bbd5a79a3874 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:50.548799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:50.572986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 976/1951] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:55.193104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:55.194624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:55.275402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:55.354401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b96664-f9eb-4741-bded-07726d4bd4d2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:55.376483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:55.377437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:55.377691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:55.378020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] PATH_INFO: `/v3/users/ae0c0bfa95c94ed7a8b31e8a9715675d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:55.382488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0c0bfa95c94ed7a8b31e8a9715675d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:55.384620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:55.479654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0c0bfa95c94ed7a8b31e8a9715675d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:55.506465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0c0bfa95c94ed7a8b31e8a9715675d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:55.523783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0c0bfa95c94ed7a8b31e8a9715675d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:55.547086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0c0bfa95c94ed7a8b31e8a9715675d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:55.547670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0c0bfa95c94ed7a8b31e8a9715675d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:55.562787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 976/1952] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:55 2022] DELETE /identity/v3/users/ae0c0bfa95c94ed7a8b31e8a9715675d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:55.564227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9095bf0-678d-4de1-9ab7-ff534c755a73 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:55.564726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9095bf0-678d-4de1-9ab7-ff534c755a73 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:55.564820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9095bf0-678d-4de1-9ab7-ff534c755a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:55.605360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f9095bf0-678d-4de1-9ab7-ff534c755a73 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:55.606914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f9095bf0-678d-4de1-9ab7-ff534c755a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:55.638488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:55.640088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:55.795991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 977/1953] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:55.797800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:55.798146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:55.798403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:55.820842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8dae3-65b3-47fa-99ba-c5483af095b9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:55.845113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffe2fb-d178-4390-9be3-c3c13b7f0d95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:55.846733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffe2fb-d178-4390-9be3-c3c13b7f0d95 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:55.847741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffe2fb-d178-4390-9be3-c3c13b7f0d95 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:55.848205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffe2fb-d178-4390-9be3-c3c13b7f0d95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:55.852467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-21ffe2fb-d178-4390-9be3-c3c13b7f0d95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:55.863034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] MFA Rules not processed for user `f6c7299bc6fd4163b360528217f0b679`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:55.864162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:55.943563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:56.005375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffe2fb-d178-4390-9be3-c3c13b7f0d95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.009654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffe2fb-d178-4390-9be3-c3c13b7f0d95 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.018141 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 978/1954] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:48:56.022610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.023816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:56.040136 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 977/1955] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:56.041667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd3747-866f-4397-b630-d242705e9d2f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.042131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd3747-866f-4397-b630-d242705e9d2f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.042557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd3747-866f-4397-b630-d242705e9d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.105415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c3fd3747-866f-4397-b630-d242705e9d2f None None] MFA Rules not processed for user `4defc9e17db743d7b79e732a0f695ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:56.106790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c3fd3747-866f-4397-b630-d242705e9d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:56.184031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2802de-6a5e-4fd6-848b-6b70f403d767 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.194982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.195618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.195904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.196181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.214848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.216317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.241028 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 979/1956] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:56.242156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.247009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:56.253666 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 978/1957] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:56.264723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8930e-a2ee-45ab-b25f-6607599abb69 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.265185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8930e-a2ee-45ab-b25f-6607599abb69 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.265613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8930e-a2ee-45ab-b25f-6607599abb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.271482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31ab02-9397-4001-bcfb-15c215869a3c service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.289713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81688e-62d6-4de0-b43d-afb7e7deb773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.290392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81688e-62d6-4de0-b43d-afb7e7deb773 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.290679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81688e-62d6-4de0-b43d-afb7e7deb773 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.290957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81688e-62d6-4de0-b43d-afb7e7deb773 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.291603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81688e-62d6-4de0-b43d-afb7e7deb773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.293533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81688e-62d6-4de0-b43d-afb7e7deb773 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.301064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-31f8930e-a2ee-45ab-b25f-6607599abb69 None None] MFA Rules not processed for user `dd431fc2d9834bea8df807edb91771ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:56.305302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-31f8930e-a2ee-45ab-b25f-6607599abb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:56.367052 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 980/1958] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:48:56 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:56.368699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.371214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.373337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.417119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:56.420169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:56.436088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:56.458282 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 979/1959] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:56.458282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8930e-a2ee-45ab-b25f-6607599abb69 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.483609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8930e-a2ee-45ab-b25f-6607599abb69 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.549511 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 981/1960] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:56.563395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.565240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:56.583700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec2a1-e5cc-44dc-b3ee-593a8bdc9212 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.611727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.612891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.614826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.615828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.640994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.641815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.653852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.654489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.654776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.655051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.655700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.658049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.666205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 982/1961] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:48:56.684700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:56.707462 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 980/1962] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:48:56 2022] POST /identity/v3/users => generated 309 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:48:56.711841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.743869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef92de-6e69-4dc5-b937-ab8662d8eb74 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.760752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.766304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.766695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.768254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.769032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.775908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.814459 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 983/1963] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:56.818414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:56.838157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4c278-1746-4764-9bd6-3b9c7867b744 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:56.848887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:56.852648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:56.853116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:56.853748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] PATH_INFO: `/v3/projects/e9832c54f08946e699d60a980cc621e5/users/b77e361aa6cb477083688130e08db6c0/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:56.923864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9832c54f08946e699d60a980cc621e5, user_id=b77e361aa6cb477083688130e08db6c0, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:56.926854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:56.986617 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 981/1964] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:56 2022] PUT /identity/v3/projects/e9832c54f08946e699d60a980cc621e5/users/b77e361aa6cb477083688130e08db6c0/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:56.991599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:57.008680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49b4cf-0480-4d88-943b-7769e8395bc8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:57.057267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:57.059343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:57.238023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:57.239015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:57.239311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:57.239587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:57.240242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:57.242036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:57.276172 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 984/1965] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:48:57.277381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:57.281808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:57.287064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8c4b1-cd06-46d6-a370-00323a80f1cc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:57.309342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:57.311565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:57.311859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:57.312131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:57.316401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:57.325206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a884fa-44dd-49cf-b295-6182348c50c9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:57.343211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:57.343874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:57.344155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:57.344424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] PATH_INFO: `/v3/projects/b887509639cb4e708a501572e5c6aee8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:57.351639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b887509639cb4e708a501572e5c6aee8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:57.358555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:57.537311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:57.543748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:57.551923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b887509639cb4e708a501572e5c6aee8', 'request_id': 'req-e3c4cdc6-1963-4e49-88ad-fd18375ab462'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:57.591012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b887509639cb4e708a501572e5c6aee8', 'request_id': 'req-e3c4cdc6-1963-4e49-88ad-fd18375ab462'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:57.595637 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 982/1966] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:57.597530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:57.613434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b887509639cb4e708a501572e5c6aee8', 'request_id': 'req-e3c4cdc6-1963-4e49-88ad-fd18375ab462'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:57.632537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38dd92-dbca-486a-8382-26fb50ae279b service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:57.638995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] The token cache is being invalidate because project b887509639cb4e708a501572e5c6aee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:48:57.654547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:57.655395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:57.659472 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 985/1967] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:57 2022] DELETE /identity/v3/projects/b887509639cb4e708a501572e5c6aee8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:57.663092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:57.667365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:57.721761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:57.793593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c76c8-b4c5-45f2-806e-b67b249139ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:57.794407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c76c8-b4c5-45f2-806e-b67b249139ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:57.794902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c76c8-b4c5-45f2-806e-b67b249139ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:57.795483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c76c8-b4c5-45f2-806e-b67b249139ad admin admin] PATH_INFO: `/v3/projects/e9832c54f08946e699d60a980cc621e5/users/b77e361aa6cb477083688130e08db6c0/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:57.799152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c76c8-b4c5-45f2-806e-b67b249139ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9832c54f08946e699d60a980cc621e5, user_id=b77e361aa6cb477083688130e08db6c0, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:57.806330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c76c8-b4c5-45f2-806e-b67b249139ad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:57.851948 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 983/1968] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:48:57 2022] PUT /identity/v3/projects/e9832c54f08946e699d60a980cc621e5/users/b77e361aa6cb477083688130e08db6c0/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:57.854279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4cdc6-1963-4e49-88ad-fd18375ab462 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:57.995942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:57.995942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:57.995942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:58.019880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:58.052394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] MFA Rules not processed for user `5aab292e925940568e9e5db8b6910f45`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:48:58.054104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:58.067633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:58.068991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:58.069437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:58.070491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] PATH_INFO: `/v3/users/89ed19cfebd84df6b1935e8fcf7dfa90` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:58.072292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89ed19cfebd84df6b1935e8fcf7dfa90)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:58.074413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:58.209390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89ed19cfebd84df6b1935e8fcf7dfa90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:58.227687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89ed19cfebd84df6b1935e8fcf7dfa90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:58.228855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89ed19cfebd84df6b1935e8fcf7dfa90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:58.240941 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 984/1969] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:48:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:48:58.251421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89ed19cfebd84df6b1935e8fcf7dfa90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:58.256496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89ed19cfebd84df6b1935e8fcf7dfa90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:58.306490 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 986/1970] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:48:57 2022] DELETE /identity/v3/users/89ed19cfebd84df6b1935e8fcf7dfa90 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:48:59.430596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:59.432933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:59.564974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739e6c-d5df-44f2-a70a-c19cfb1e11af None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:59.574872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:48:59.577074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:59.691947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c8b3f-f655-4deb-b7b7-5136c626db92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:59.695675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c8b3f-f655-4deb-b7b7-5136c626db92 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:59.697117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c8b3f-f655-4deb-b7b7-5136c626db92 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:59.697520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c8b3f-f655-4deb-b7b7-5136c626db92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:59.703468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9a0c8b3f-f655-4deb-b7b7-5136c626db92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:48:59.717779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:59.746293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b3184-779a-47d2-ba3d-6f66b7801ab8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:48:59.752110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:48:59.757311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:48:59.757877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:48:59.758487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] PATH_INFO: `/v3/projects/05112a99b5054250bb739e5632cc77ff` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:48:59.761105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05112a99b5054250bb739e5632cc77ff)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:59.763981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:59.848994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:48:59.882537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c8b3f-f655-4deb-b7b7-5136c626db92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:48:59.886971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c8b3f-f655-4deb-b7b7-5136c626db92 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:48:59.891631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05112a99b5054250bb739e5632cc77ff', 'request_id': 'req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:59.925349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 985/1971] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:48:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:48:59.927807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05112a99b5054250bb739e5632cc77ff', 'request_id': 'req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:48:59.971604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05112a99b5054250bb739e5632cc77ff', 'request_id': 'req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:00.001688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] The token cache is being invalidate because project 05112a99b5054250bb739e5632cc77ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:00.002209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:00.003424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9f103675-fb4d-4ecf-93a0-7fda6fbdab25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:00.005352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 987/1972] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:48:59 2022] DELETE /identity/v3/projects/05112a99b5054250bb739e5632cc77ff => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:00.010088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f975e6e-7a68-431f-b4e4-0621ddbbe5dd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.010835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f975e6e-7a68-431f-b4e4-0621ddbbe5dd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.011461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f975e6e-7a68-431f-b4e4-0621ddbbe5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.065755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4f975e6e-7a68-431f-b4e4-0621ddbbe5dd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:00.074120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4f975e6e-7a68-431f-b4e4-0621ddbbe5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:00.190499 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 986/1973] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:00.199964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c4d46-9b1d-4b8c-858b-473190a64a8c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.200317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c4d46-9b1d-4b8c-858b-473190a64a8c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.200686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c4d46-9b1d-4b8c-858b-473190a64a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.246205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f47c4d46-9b1d-4b8c-858b-473190a64a8c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:00.247595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f47c4d46-9b1d-4b8c-858b-473190a64a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:00.283269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f975e6e-7a68-431f-b4e4-0621ddbbe5dd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.293952 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 988/1974] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:00.296358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f975e6e-7a68-431f-b4e4-0621ddbbe5dd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.310599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.312716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.313005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.313382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.321678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:00.322222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.323903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.351295 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 987/1975] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:00.353447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c4d46-9b1d-4b8c-858b-473190a64a8c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.364937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c4d46-9b1d-4b8c-858b-473190a64a8c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.373813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.374667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.375341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.376528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.377347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.379964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.446914 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 989/1976] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:00 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:00.447900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.461231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b293f-1329-4d08-b9ce-4de06c713dd8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.672960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.673666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.673988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.674270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.674911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.677091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.736233 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 988/1977] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:00 2022] POST /identity/v3/users => generated 291 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:00.737705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.757707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c099e-5093-4982-bb0b-c57402a57630 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.774402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.775047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.775581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.775859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.776976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.781700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.826381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.843607 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 990/1978] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:00.850878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a12227-75bd-4602-990f-f93d6aae22e8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.864898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a8e13-40ff-4b8e-9689-98d18ab49af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.865526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8e13-40ff-4b8e-9689-98d18ab49af5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.865855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8e13-40ff-4b8e-9689-98d18ab49af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.866142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8e13-40ff-4b8e-9689-98d18ab49af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:00.867416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a8e13-40ff-4b8e-9689-98d18ab49af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:00.869154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a8e13-40ff-4b8e-9689-98d18ab49af5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:00.906291 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 989/1979] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:00.907404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:00.924108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fb5ff-1270-47ff-a63f-6506b18ec8c2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:00.936672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:00.937508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:00.937885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:00.938087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] PATH_INFO: `/v3/projects/87b2a9c063724073b3336b6acbb92e82/users/45e53cb30e6f41ad935e71c26ce703ed/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:01.003012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b2a9c063724073b3336b6acbb92e82, user_id=45e53cb30e6f41ad935e71c26ce703ed, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:01.012322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:01.056392 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 991/1980] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:00 2022] PUT /identity/v3/projects/87b2a9c063724073b3336b6acbb92e82/users/45e53cb30e6f41ad935e71c26ce703ed/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:01.058074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:01.058365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:01.058641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:01.119898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] MFA Rules not processed for user `45e53cb30e6f41ad935e71c26ce703ed`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:01.121334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:01.225370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:01.285009 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 990/1981] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:01.302929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:01.304831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:01.470319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220e5ef-e296-4401-90f3-63f52083c9ec admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:01.477200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7b758-d869-42b4-926b-4ed7842979c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:01.478284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7b758-d869-42b4-926b-4ed7842979c2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:01.478913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7b758-d869-42b4-926b-4ed7842979c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:01.479500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7b758-d869-42b4-926b-4ed7842979c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:01.484065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:01.486362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:01.536165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7b758-d869-42b4-926b-4ed7842979c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:01.539096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7b758-d869-42b4-926b-4ed7842979c2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:01.564200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 992/1982] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:01.622056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b126200-92b7-4108-b00e-117cb9cbc264 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:01.630151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:01.631403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:01.632737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:01.632737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] PATH_INFO: `/v3/users/736fcd7811e341468d7080da18b011ea` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:01.634370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=736fcd7811e341468d7080da18b011ea)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:01.637338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:01.755796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736fcd7811e341468d7080da18b011ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:01.809711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736fcd7811e341468d7080da18b011ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:01.821125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736fcd7811e341468d7080da18b011ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:01.833046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736fcd7811e341468d7080da18b011ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:01.834041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736fcd7811e341468d7080da18b011ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:01.855722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 991/1983] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:01 2022] DELETE /identity/v3/users/736fcd7811e341468d7080da18b011ea => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:01.857469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:01.858802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:01.859236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:01.892280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:01.893809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:01.899560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:02.047009 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 993/1984] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:02.165211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:02.166975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:02.331931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47d867-9983-475a-bd05-473ea5704088 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:02.352290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37571d45-f32c-445d-9ef7-746a1d47dc3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:02.353092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37571d45-f32c-445d-9ef7-746a1d47dc3c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:02.353381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37571d45-f32c-445d-9ef7-746a1d47dc3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:02.353658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37571d45-f32c-445d-9ef7-746a1d47dc3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:02.375377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37571d45-f32c-445d-9ef7-746a1d47dc3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:02.377075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37571d45-f32c-445d-9ef7-746a1d47dc3c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:02.405325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:02.417090 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 992/1985] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:02.736163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:02.737679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:02.765702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeeb65-30a6-4a0c-9473-a2ddf68867d3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:02.784021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:02.784678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:02.784962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:02.785244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] PATH_INFO: `/v3/users/45e53cb30e6f41ad935e71c26ce703ed` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:02.786609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e53cb30e6f41ad935e71c26ce703ed)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:02.794923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:02.915738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e53cb30e6f41ad935e71c26ce703ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:02.931971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e53cb30e6f41ad935e71c26ce703ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:02.932975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e53cb30e6f41ad935e71c26ce703ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:02.954658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e53cb30e6f41ad935e71c26ce703ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:02.955543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e53cb30e6f41ad935e71c26ce703ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:02.996184 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 994/1986] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:02 2022] DELETE /identity/v3/users/45e53cb30e6f41ad935e71c26ce703ed => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:03.000840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.001327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.001534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.064185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:03.067372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:03.267900 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 993/1987] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:03.425980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:03.427750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:03.500012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eb7b4-ce11-460f-bed5-ff6015694a0e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:03.504613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:03.506239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:03.506628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:03.507034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:03.518489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:03.520563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:03.531555 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 995/1988] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:04.033792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:04.038489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:04.089429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c995185-6bdc-480f-89f9-1783b332c82a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:04.093567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:04.095798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:04.110246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:04.113774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:04.114322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:04.116185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] PATH_INFO: `/v3/users/ea55e45cd1b94a41baed5bb35ce324f8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:04.120321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649a2a9-085b-4263-a7fd-a84a4d1b87ba service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:04.122153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea55e45cd1b94a41baed5bb35ce324f8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:04.124124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:04.136587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:04.137647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:04.145445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:04.147083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] PATH_INFO: `/v3/projects/5f6c275f77df48e196553dbc20db8ed9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:04.154661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f6c275f77df48e196553dbc20db8ed9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:04.156912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:04.233456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:04.267626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea55e45cd1b94a41baed5bb35ce324f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.311541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f6c275f77df48e196553dbc20db8ed9', 'request_id': 'req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.325733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea55e45cd1b94a41baed5bb35ce324f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.330450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f6c275f77df48e196553dbc20db8ed9', 'request_id': 'req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.347763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f6c275f77df48e196553dbc20db8ed9', 'request_id': 'req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.364347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] The token cache is being invalidate because project 5f6c275f77df48e196553dbc20db8ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:04.364471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.367514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.373046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 996/1989] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:04 2022] DELETE /identity/v3/projects/5f6c275f77df48e196553dbc20db8ed9 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:04.376734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:04.380816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea55e45cd1b94a41baed5bb35ce324f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.386161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:04.397022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea55e45cd1b94a41baed5bb35ce324f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.398379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec13c197-c4de-4b15-9c63-e4fc8e7d14d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea55e45cd1b94a41baed5bb35ce324f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.412791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 994/1990] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:04 2022] DELETE /identity/v3/users/ea55e45cd1b94a41baed5bb35ce324f8 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:04.417600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:04.419346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:04.419896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:04.472026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:04.479410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:04.532220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdee1c-003a-4f10-89d2-f9693bfa4f50 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:04.575673 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 995/1991] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:04.635303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:04.636290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:04.659770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:04.670402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:04.671628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:04.672112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:04.672592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] PATH_INFO: `/v3/users/1e75f8b01c4a4939b7f8c2086e3974e5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:04.675075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e75f8b01c4a4939b7f8c2086e3974e5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:04.678164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:04.777472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614492-0ef2-4e03-8d29-b4402882676d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:04.787181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:04.790736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:04.791613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:04.791916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:04.794313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e75f8b01c4a4939b7f8c2086e3974e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.813816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e75f8b01c4a4939b7f8c2086e3974e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.817400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e75f8b01c4a4939b7f8c2086e3974e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.820918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:04.823836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:04.824653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:04.837726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e75f8b01c4a4939b7f8c2086e3974e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.838998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e75f8b01c4a4939b7f8c2086e3974e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:04.858948 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 996/1992] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:04.875587 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 997/1993] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:04 2022] DELETE /identity/v3/users/1e75f8b01c4a4939b7f8c2086e3974e5 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:06.395004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:06.396672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:06.425355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cdd3e-59f0-41c3-b5c9-18a20e6fb95f service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:06.584577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:06.585574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:06.585906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:06.586186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] PATH_INFO: `/v3/projects/64caaddd046d4282a6c5dccf553eb5ea` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:06.591266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64caaddd046d4282a6c5dccf553eb5ea)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:06.600177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:06.693519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64caaddd046d4282a6c5dccf553eb5ea', 'request_id': 'req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:06.706228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64caaddd046d4282a6c5dccf553eb5ea', 'request_id': 'req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:06.727504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64caaddd046d4282a6c5dccf553eb5ea', 'request_id': 'req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:06.756624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] The token cache is being invalidate because project 64caaddd046d4282a6c5dccf553eb5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:06.757609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:06.765916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:06.783308 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 997/1994] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:06 2022] DELETE /identity/v3/projects/64caaddd046d4282a6c5dccf553eb5ea => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:06.787926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:06.792887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:06.823181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a4781-9f19-4ef2-bed0-0f3a22393a8a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:06.828055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:06.829628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:06.935374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:06.935919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:06.936408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:06.937425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] PATH_INFO: `/v3/users/f794e66e441b4e489ea613b164ed6b29` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:06.952958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:06.958569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f794e66e441b4e489ea613b164ed6b29)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:06.971245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:06.988702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24352be9-2550-4ee3-8bcf-e7e566b7a0c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.014341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:07.015002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.015287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.015564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] PATH_INFO: `/v3/projects/c8dfd36f32b9446d9fcb8551c253e3c9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.026756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8dfd36f32b9446d9fcb8551c253e3c9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:07.028839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:07.083984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f794e66e441b4e489ea613b164ed6b29'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.101696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f794e66e441b4e489ea613b164ed6b29'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.104705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f794e66e441b4e489ea613b164ed6b29'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.117570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f794e66e441b4e489ea613b164ed6b29'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.119067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f794e66e441b4e489ea613b164ed6b29'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.127616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8dfd36f32b9446d9fcb8551c253e3c9', 'request_id': 'req-8597d9ff-37ff-4886-a25e-d79b8428693b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.142827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8dfd36f32b9446d9fcb8551c253e3c9', 'request_id': 'req-8597d9ff-37ff-4886-a25e-d79b8428693b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.147967 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 998/1995] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:06 2022] DELETE /identity/v3/users/f794e66e441b4e489ea613b164ed6b29 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:07.148972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:07.150242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8dfd36f32b9446d9fcb8551c253e3c9', 'request_id': 'req-8597d9ff-37ff-4886-a25e-d79b8428693b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.150735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:07.167518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] The token cache is being invalidate because project c8dfd36f32b9446d9fcb8551c253e3c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:07.167902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.168636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8597d9ff-37ff-4886-a25e-d79b8428693b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.171830 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 998/1996] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:06 2022] DELETE /identity/v3/projects/c8dfd36f32b9446d9fcb8551c253e3c9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:07.177362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29d76-2c6b-40b4-8366-658dc639d06d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.177978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29d76-2c6b-40b4-8366-658dc639d06d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.178411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d29d76-2c6b-40b4-8366-658dc639d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.205142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-04d29d76-2c6b-40b4-8366-658dc639d06d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:07.206839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-04d29d76-2c6b-40b4-8366-658dc639d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:07.228258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:07.234445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b5c35-e599-4853-8be1-b0fc62d6aaa4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.243837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:07.244063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.244785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.244785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] PATH_INFO: `/v3/projects/87b2a9c063724073b3336b6acbb92e82` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.250704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b2a9c063724073b3336b6acbb92e82)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:07.254065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:07.327187 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 999/1997] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:07.334202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b15118-2e51-4b4c-8ebe-8f0542cdac67 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.334962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b15118-2e51-4b4c-8ebe-8f0542cdac67 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.336156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b15118-2e51-4b4c-8ebe-8f0542cdac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.350260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b2a9c063724073b3336b6acbb92e82', 'request_id': 'req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.371149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b2a9c063724073b3336b6acbb92e82', 'request_id': 'req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.383672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-76b15118-2e51-4b4c-8ebe-8f0542cdac67 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:07.385863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b2a9c063724073b3336b6acbb92e82', 'request_id': 'req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.391126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76b15118-2e51-4b4c-8ebe-8f0542cdac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:07.409578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] The token cache is being invalidate because project 87b2a9c063724073b3336b6acbb92e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:07.412749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.414503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba6faa71-eacf-4ea2-9044-5e720ee62dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:07.422993 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 999/1998] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:07 2022] DELETE /identity/v3/projects/87b2a9c063724073b3336b6acbb92e82 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:07.424706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955673bd-ee1a-4527-b3bb-782849385349 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.425150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955673bd-ee1a-4527-b3bb-782849385349 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.425433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955673bd-ee1a-4527-b3bb-782849385349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.465664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-955673bd-ee1a-4527-b3bb-782849385349 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:07.466960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-955673bd-ee1a-4527-b3bb-782849385349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:07.548629 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1000/1999] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:07.553032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b15118-2e51-4b4c-8ebe-8f0542cdac67 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:07.575863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b15118-2e51-4b4c-8ebe-8f0542cdac67 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.587677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:07.587677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.587967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.588284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.588902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:07.589207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:07.591403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:07.625485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1001/2000] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:07.628217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:07.629567 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1000/2001] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:07.639155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e88e-b90d-493a-a075-2af9a4d2de21 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.639509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e88e-b90d-493a-a075-2af9a4d2de21 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.639704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e88e-b90d-493a-a075-2af9a4d2de21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.663023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:07.676708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e9421-9cbe-4780-93b0-33776e24a1f3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.684095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:07.684814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.685477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.685477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.686145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:07.686930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5058e88e-b90d-493a-a075-2af9a4d2de21 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:07.688288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:07.688415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5058e88e-b90d-493a-a075-2af9a4d2de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:07.702462 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1001/2002] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:07.703564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058e88e-b90d-493a-a075-2af9a4d2de21 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:07.722499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058e88e-b90d-493a-a075-2af9a4d2de21 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.726624 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1002/2003] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:07 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:07.730703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:07.742117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:07.743034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.743321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.743597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.744263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:07.744626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:07.758396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:07.778777 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1002/2004] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:07.791997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:07.832566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61e9c9-7fd3-4b27-b1cf-2bceaa1aa295 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.871775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0643705-607a-44aa-8dde-8ad016456a3c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.885438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:07.886356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.887092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.887092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.887846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:07.889805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:07.924608 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1003/2005] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:07 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:07.929404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:07.942958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef18b9-2767-4653-a053-2043c1fd28ad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:07.954185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:07.955418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:07.955961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:07.956493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:07.958436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:07.962513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:07.981673 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1004/2006] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:07.985273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.004881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6d6b2-eed1-457b-93ab-dfda42cb941b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.011508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.012481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.013154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.013651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.015151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.017749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.036759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.037164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.037459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.039299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1005/2007] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:08.042471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.043230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.043813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.045190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.060010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8ec62-4890-4912-b4ff-3a9962bbcde0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.075757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebf770-6992-45b8-bd6d-ab3398fa5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.076331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf770-6992-45b8-bd6d-ab3398fa5473 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.077135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf770-6992-45b8-bd6d-ab3398fa5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.077135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf770-6992-45b8-bd6d-ab3398fa5473 admin admin] PATH_INFO: `/v3/projects/247a7ee43ba84bcc9152a6bce650b52e/users/e754ce3854f2458b98dc0438028ebc54/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.085695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:08.114732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebf770-6992-45b8-bd6d-ab3398fa5473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=247a7ee43ba84bcc9152a6bce650b52e, user_id=e754ce3854f2458b98dc0438028ebc54, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.124044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebf770-6992-45b8-bd6d-ab3398fa5473 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.140278 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1003/2008] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:07 2022] POST /identity/v3/projects => generated 373 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:08.141228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.150243 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1006/2009] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:08 2022] PUT /identity/v3/projects/247a7ee43ba84bcc9152a6bce650b52e/users/e754ce3854f2458b98dc0438028ebc54/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:08.161644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b170c-6d64-48c8-8535-c46254fa9757 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.161644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b170c-6d64-48c8-8535-c46254fa9757 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.161644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b170c-6d64-48c8-8535-c46254fa9757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.236867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-823b170c-6d64-48c8-8535-c46254fa9757 None None] MFA Rules not processed for user `e754ce3854f2458b98dc0438028ebc54`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:08.238185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-823b170c-6d64-48c8-8535-c46254fa9757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:08.292931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b0670-9d6c-464c-b6be-3163e03477ed admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.382879 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1007/2010] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:08.383950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b170c-6d64-48c8-8535-c46254fa9757 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.422480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:08.423968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.434984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.435279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.435568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.442310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.449988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.488036 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1004/2011] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:08 2022] POST /identity/v3/users => generated 287 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:08.489593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.502064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc259-3463-40ae-ac82-ed29fd6312b9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.516164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.516395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b170c-6d64-48c8-8535-c46254fa9757 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.516855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.517133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.517426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.518232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.519694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.521464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.523511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.523853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.524196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.525308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.527523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.551723 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1005/2012] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:08.555056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.573443 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1008/2013] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:08 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:08.574773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.582916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfc614-896a-472c-b1ce-c03332629f4b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.682277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.688215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f624f2-4c72-4ed9-a38e-fc5220e81871 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.694057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.694057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.694985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.696456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.697143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.697584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.697870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.698777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.700085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.701292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.702233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.730811 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1006/2014] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:08.732206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.743828 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1009/2015] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:08 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:08.744647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.751927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2849f75-122f-41c5-bff7-19ccf7686da6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.758790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94650b7f-4c24-44d1-8266-d0b135d48100 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.768166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.768855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.769186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.769500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.770140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.771821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.775634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34733cb7-da77-47d4-ae9a-331f8056d4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:08.776842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733cb7-da77-47d4-ae9a-331f8056d4d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.777316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733cb7-da77-47d4-ae9a-331f8056d4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.777759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34733cb7-da77-47d4-ae9a-331f8056d4d0 admin admin] PATH_INFO: `/v3/projects/781b9701e20f4fbc9f6a9e1894e7ed10/users/088d1e931600487f8ace059ac5b32e0f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.811591 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1010/2016] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:08.812644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:08.813392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:08.821040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34733cb7-da77-47d4-ae9a-331f8056d4d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781b9701e20f4fbc9f6a9e1894e7ed10, user_id=088d1e931600487f8ace059ac5b32e0f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:08.825053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34733cb7-da77-47d4-ae9a-331f8056d4d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:08.841671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b5e81-02eb-4aa0-b082-8f8ce3d8b5d7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:08.854142 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1007/2017] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:08 2022] PUT /identity/v3/projects/781b9701e20f4fbc9f6a9e1894e7ed10/users/088d1e931600487f8ace059ac5b32e0f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:08.876494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d29e6-213c-4c58-a398-8466cd7398da None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:08.878186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d29e6-213c-4c58-a398-8466cd7398da None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:08.878713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d29e6-213c-4c58-a398-8466cd7398da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:08.949393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5e2d29e6-213c-4c58-a398-8466cd7398da None None] MFA Rules not processed for user `088d1e931600487f8ace059ac5b32e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:08.953183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e2d29e6-213c-4c58-a398-8466cd7398da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:09.002971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:09.007456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.008030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.008511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] PATH_INFO: `/v3/projects/f7520a7e2bdc4a10baa5ed56bae12769/users/133ba8746ac34c5bbc9877db8b385ddc/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.044772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7520a7e2bdc4a10baa5ed56bae12769, user_id=133ba8746ac34c5bbc9877db8b385ddc, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:09.052026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:09.087728 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1008/2018] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:09.095174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d29e6-213c-4c58-a398-8466cd7398da None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:09.096438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1011/2019] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:08 2022] PUT /identity/v3/projects/f7520a7e2bdc4a10baa5ed56bae12769/users/133ba8746ac34c5bbc9877db8b385ddc/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:09.097415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:09.102375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:09.133118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d29e6-213c-4c58-a398-8466cd7398da None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:09.249423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1e629-4396-45f0-9174-01f1708023d3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:09.261191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:09.262088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.262309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.262540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.263193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:09.265123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:09.278674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:09.279488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.279941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.280654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.284602 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1012/2020] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:09.284602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:09.285278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:09.287014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:09.288808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:09.304960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c7d40-8869-47f3-b5b5-d5cbc0b440bc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:09.312567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:09.313364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.313753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.314162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] PATH_INFO: `/v3/projects/3a825f49753c4baaa6661e82f2c30df7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.315614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a825f49753c4baaa6661e82f2c30df7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:09.319370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:09.341774 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1009/2021] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:09 2022] POST /identity/v3/projects => generated 373 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:09.342514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:09.417406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a825f49753c4baaa6661e82f2c30df7', 'request_id': 'req-4431509c-4b7c-48d5-93e6-eca62a19a50b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:09.428730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a825f49753c4baaa6661e82f2c30df7', 'request_id': 'req-4431509c-4b7c-48d5-93e6-eca62a19a50b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:09.451339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a825f49753c4baaa6661e82f2c30df7', 'request_id': 'req-4431509c-4b7c-48d5-93e6-eca62a19a50b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:09.495897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] The token cache is being invalidate because project 3a825f49753c4baaa6661e82f2c30df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:09.496260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:09.497026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:09.498946 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1013/2022] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:09 2022] DELETE /identity/v3/projects/3a825f49753c4baaa6661e82f2c30df7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:09.499694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:09.501491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:09.579879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300798-572e-4340-ac2c-7b01ef6c8a6b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:09.598027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:09.658964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431509c-4b7c-48d5-93e6-eca62a19a50b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:09.677168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb953f-9591-44a4-88b4-58e15151ffd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:09.678354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb953f-9591-44a4-88b4-58e15151ffd3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.680437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb953f-9591-44a4-88b4-58e15151ffd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.681740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb953f-9591-44a4-88b4-58e15151ffd3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.692023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb953f-9591-44a4-88b4-58e15151ffd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:09.707783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb953f-9591-44a4-88b4-58e15151ffd3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:09.751993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:09.752821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.754678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.754953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] PATH_INFO: `/v3/projects/f7520a7e2bdc4a10baa5ed56bae12769/users/133ba8746ac34c5bbc9877db8b385ddc/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.759583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7520a7e2bdc4a10baa5ed56bae12769, user_id=133ba8746ac34c5bbc9877db8b385ddc, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:09.761959 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1014/2023] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:09 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:09.769729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8ead4-93fa-4942-9443-c2c02a2ba2db None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.770387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:09.770780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8ead4-93fa-4942-9443-c2c02a2ba2db None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.771219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8ead4-93fa-4942-9443-c2c02a2ba2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.819849 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1010/2024] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:09 2022] PUT /identity/v3/projects/f7520a7e2bdc4a10baa5ed56bae12769/users/133ba8746ac34c5bbc9877db8b385ddc/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:09.820990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:09.851597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-42c8ead4-93fa-4942-9443-c2c02a2ba2db None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:09.853624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-42c8ead4-93fa-4942-9443-c2c02a2ba2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:09.867916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:09.973385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbc611-8b04-4668-abf2-00cc25c09969 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:09.979625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b9503-40ad-4699-9edd-f9837e7b2686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:09.980504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b9503-40ad-4699-9edd-f9837e7b2686 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:09.981602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b9503-40ad-4699-9edd-f9837e7b2686 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:09.982003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b9503-40ad-4699-9edd-f9837e7b2686 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:09.983535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b9503-40ad-4699-9edd-f9837e7b2686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:09.986724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b9503-40ad-4699-9edd-f9837e7b2686 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.011382 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1011/2025] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:10.011382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a0cee-e6e2-485a-a1b6-1b01626a8194 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.011382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a0cee-e6e2-485a-a1b6-1b01626a8194 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.011382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a0cee-e6e2-485a-a1b6-1b01626a8194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.019253 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1015/2026] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:10.020442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8ead4-93fa-4942-9443-c2c02a2ba2db None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:10.034510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8ead4-93fa-4942-9443-c2c02a2ba2db None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:10.041354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1e362-57a1-4a42-ac3d-60bf0f1f748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:10.042527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1e362-57a1-4a42-ac3d-60bf0f1f748f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.043021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1e362-57a1-4a42-ac3d-60bf0f1f748f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.043480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1e362-57a1-4a42-ac3d-60bf0f1f748f admin admin] PATH_INFO: `/v3/projects/18dbf9e8023e42c8a747ba1efc2e6def/users/2def79099da04db7a6672771b71591c8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.071619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8e7a0cee-e6e2-485a-a1b6-1b01626a8194 None None] MFA Rules not processed for user `133ba8746ac34c5bbc9877db8b385ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:10.074125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e7a0cee-e6e2-485a-a1b6-1b01626a8194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:10.089109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1e362-57a1-4a42-ac3d-60bf0f1f748f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18dbf9e8023e42c8a747ba1efc2e6def, user_id=2def79099da04db7a6672771b71591c8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:10.094526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1e362-57a1-4a42-ac3d-60bf0f1f748f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.137361 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1016/2027] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:10 2022] PUT /identity/v3/projects/18dbf9e8023e42c8a747ba1efc2e6def/users/2def79099da04db7a6672771b71591c8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:10.143331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.145005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.145670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.199767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:10.201316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:10.202040 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1012/2028] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:10.203162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a0cee-e6e2-485a-a1b6-1b01626a8194 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:10.220222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a0cee-e6e2-485a-a1b6-1b01626a8194 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:10.273633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:10.349087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:10.349495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.350322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.350322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.351606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:10.353617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.376062 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1017/2029] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:10.377084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:10.382781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:10.387186 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1013/2030] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:10.388141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:10.408285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e837e36-6449-469f-9287-9fac4b245706 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:10.421374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:10.422554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.422849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.423124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.423781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:10.424083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:10.431109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.471374 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1014/2031] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:10.472317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:10.484710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd473b-4ac9-44c2-8649-851866db604f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:10.495562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:10.496248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.498187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.498537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] PATH_INFO: `/v3/projects/18dbf9e8023e42c8a747ba1efc2e6def/users/2def79099da04db7a6672771b71591c8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.504042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18dbf9e8023e42c8a747ba1efc2e6def, user_id=2def79099da04db7a6672771b71591c8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:10.510909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.511196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e924d5-7562-4b1c-961b-8b8f4e4f3715 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:10.522004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79c0bd-47fa-42b8-8423-beb073cb3814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:10.523595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79c0bd-47fa-42b8-8423-beb073cb3814 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.523786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79c0bd-47fa-42b8-8423-beb073cb3814 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.524049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79c0bd-47fa-42b8-8423-beb073cb3814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.526121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bd79c0bd-47fa-42b8-8423-beb073cb3814 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:10.530859 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1015/2032] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:10 2022] PUT /identity/v3/projects/18dbf9e8023e42c8a747ba1efc2e6def/users/2def79099da04db7a6672771b71591c8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:10.531861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:10.572036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:10.665132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86263d7-76fa-43f6-b559-514856ff4b84 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:10.679703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79c0bd-47fa-42b8-8423-beb073cb3814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:10.681343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79c0bd-47fa-42b8-8423-beb073cb3814 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.697197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:10.699293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.699590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.699868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.700519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:10.702532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.714817 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1018/2033] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:10.717684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e05b3-08e5-4542-baf5-2188588baad7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.718083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e05b3-08e5-4542-baf5-2188588baad7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.718327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e05b3-08e5-4542-baf5-2188588baad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.780575 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1016/2034] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:10 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:10.782090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:10.786994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:10.791806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-404e05b3-08e5-4542-baf5-2188588baad7 None None] MFA Rules not processed for user `2def79099da04db7a6672771b71591c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:10.793288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-404e05b3-08e5-4542-baf5-2188588baad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:10.910066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071dd48-272e-4f71-aa3a-3fd5e0e7c4d9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:10.916321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:10.917038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:10.917330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:10.917612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] PATH_INFO: `/v3/users/e754ce3854f2458b98dc0438028ebc54` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:10.919402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e754ce3854f2458b98dc0438028ebc54)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:10.921654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:10.960849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:10.995627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e754ce3854f2458b98dc0438028ebc54'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:11.003001 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1019/2035] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:11.003661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e05b3-08e5-4542-baf5-2188588baad7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.005763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e754ce3854f2458b98dc0438028ebc54'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:11.007459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e754ce3854f2458b98dc0438028ebc54'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:11.025310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e754ce3854f2458b98dc0438028ebc54'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:11.026854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e05b3-08e5-4542-baf5-2188588baad7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:11.028507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0514aeb4-9206-4c72-903a-a6a7f6f94d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e754ce3854f2458b98dc0438028ebc54'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:11.036220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:11.036921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.037206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.037480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.038613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:11.040890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:11.084166 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1017/2036] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:10 2022] DELETE /identity/v3/users/e754ce3854f2458b98dc0438028ebc54 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:11.088461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1020/2037] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:11 2022] POST /identity/v3/users => generated 294 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:11.099867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.100688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed35609-513f-401e-8922-3b77905c50fa None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.100688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed35609-513f-401e-8922-3b77905c50fa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.102976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.108408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed35609-513f-401e-8922-3b77905c50fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.129574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f4e66-bc72-47cd-b4e6-3c4d1b8fc8d5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:11.142637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fed35609-513f-401e-8922-3b77905c50fa None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:11.144356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fed35609-513f-401e-8922-3b77905c50fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.239032 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1018/2038] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:11.240268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed35609-513f-401e-8922-3b77905c50fa None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.249472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fed35609-513f-401e-8922-3b77905c50fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.261526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:11.262861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.267366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.267772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.268825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:11.271980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:11.311605 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1021/2039] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:11.315254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.317925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.336548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed35609-513f-401e-8922-3b77905c50fa None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:11.342919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:11.343553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.343836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.344109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.346338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.410973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:11.423320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:11.480922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:11.482619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:11.485043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa796-1cda-4bf6-b071-50e05dd16d77 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:11.502756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:11.503389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.503670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.503946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.513206 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1019/2040] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:11.517355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.526786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:11.528245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:11.543131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5df62e-b4f5-4c99-b88f-0e9ac4ce8445 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:11.557108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:11.570417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.570417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.570417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.570417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:11.571006 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1022/2041] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:11.571086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:11.592057 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1020/2042] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:11.596845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.610103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc369df4-2232-46d5-a575-49c3b42d2175 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:11.616408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ef26b-13b8-4e34-aced-50aee8e1aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:11.617311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ef26b-13b8-4e34-aced-50aee8e1aecb admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.618613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ef26b-13b8-4e34-aced-50aee8e1aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.619219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ef26b-13b8-4e34-aced-50aee8e1aecb admin admin] PATH_INFO: `/v3/projects/5cc9f18c8fd44fca98cda3832b60b26d/users/365336e9f59e4b65ae0a05554776b0ff/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.641331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ef26b-13b8-4e34-aced-50aee8e1aecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc9f18c8fd44fca98cda3832b60b26d, user_id=365336e9f59e4b65ae0a05554776b0ff, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:11.644521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.646226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ef26b-13b8-4e34-aced-50aee8e1aecb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:11.646769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.672807 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1023/2043] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:11 2022] PUT /identity/v3/projects/5cc9f18c8fd44fca98cda3832b60b26d/users/365336e9f59e4b65ae0a05554776b0ff/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:11.683595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7996-f4da-4d6f-bfe6-72c980f5bac9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.684557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7996-f4da-4d6f-bfe6-72c980f5bac9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.684987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7996-f4da-4d6f-bfe6-72c980f5bac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.691022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e29b6-2fb9-44c2-be9b-3a8ff9d2da14 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:11.746507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9e4c7996-f4da-4d6f-bfe6-72c980f5bac9 None None] MFA Rules not processed for user `365336e9f59e4b65ae0a05554776b0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:11.748226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9e4c7996-f4da-4d6f-bfe6-72c980f5bac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.798753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:11.800040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:11.800636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:11.801015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:11.803497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:11.896413 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1024/2044] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:11.898380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c7996-f4da-4d6f-bfe6-72c980f5bac9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:11.903114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:11.905050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:11.924191 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1021/2045] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:12.013849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c7996-f4da-4d6f-bfe6-72c980f5bac9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:12.020257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:12.021054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.021461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.021852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.023393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:12.026608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:12.081746 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1025/2046] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:11 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:12.086241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:12.100637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57aa8f6-cec4-4e05-b917-38e80590044d service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:12.218948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:12.219935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.220295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.220641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.221352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:12.225177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:12.235694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:12.260381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1022/2047] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:12 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:12.265022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:12.285765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a4b5b-9a6a-46e7-9bb3-d9fffc22b2b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:12.298183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:12.298819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.299103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.299379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.300088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:12.301692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:12.332301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:12.338435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:12.371302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1026/2048] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:12.375106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299725f-8acb-4a1b-bb13-ed7c3bf894b3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:12.376652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:12.382690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:12.384285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.384792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.385274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] PATH_INFO: `/v3/users/088d1e931600487f8ace059ac5b32e0f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.387179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=088d1e931600487f8ace059ac5b32e0f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:12.391642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fccfd-bc73-4c66-a2d8-9882b76eecab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:12.396352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:12.403525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:12.404505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.405309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.409237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] PATH_INFO: `/v3/projects/2600c77488cb4c7882998dc76799b9fc/users/0f668ef345144fc695d1fc0fffe1cb3e/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.422389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:12.440805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2600c77488cb4c7882998dc76799b9fc, user_id=0f668ef345144fc695d1fc0fffe1cb3e, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:12.444998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:12.471086 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1027/2049] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:12 2022] PUT /identity/v3/projects/2600c77488cb4c7882998dc76799b9fc/users/0f668ef345144fc695d1fc0fffe1cb3e/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:12.473495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:12.519837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088d1e931600487f8ace059ac5b32e0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.534524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088d1e931600487f8ace059ac5b32e0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.535452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088d1e931600487f8ace059ac5b32e0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.549648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088d1e931600487f8ace059ac5b32e0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.551662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f41154e-3940-414e-90fd-52a3fd4d0128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088d1e931600487f8ace059ac5b32e0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.607682 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1023/2050] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:12 2022] DELETE /identity/v3/users/088d1e931600487f8ace059ac5b32e0f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:12.610216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.610692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.611204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.642113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:12.643642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634cf82-77d8-4c07-bf16-3de244d0b066 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:12.643874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:12.786639 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1024/2051] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:12.790182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:12.790281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:12.790709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:12.790789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.790998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.791266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.796371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:12.805492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:12.813718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb431f8-bf45-4a63-a639-26c54c915520 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:12.813718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:12.813718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:12.813718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:12.814040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] PATH_INFO: `/v3/projects/247a7ee43ba84bcc9152a6bce650b52e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:12.828503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] RBAC: Authorizing `identity:delete_project(project_id=247a7ee43ba84bcc9152a6bce650b52e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:12.830650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:12.831748 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1028/2052] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:12.832863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:12.835688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:12.941083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247a7ee43ba84bcc9152a6bce650b52e', 'request_id': 'req-ac212855-b923-41c6-8ce0-4df6a850becd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.953099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247a7ee43ba84bcc9152a6bce650b52e', 'request_id': 'req-ac212855-b923-41c6-8ce0-4df6a850becd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.958478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=5,keystone:SELECT=81,keystone:DELETE=14 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:12.973187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247a7ee43ba84bcc9152a6bce650b52e', 'request_id': 'req-ac212855-b923-41c6-8ce0-4df6a850becd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:12.994226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb8bf1-2653-4532-a08a-6ba0040d74eb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:13.011107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] The token cache is being invalidate because project 247a7ee43ba84bcc9152a6bce650b52e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:13.011691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:13.013110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:13.015931 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1025/2053] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:12 2022] DELETE /identity/v3/projects/247a7ee43ba84bcc9152a6bce650b52e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:13.016953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:13.018499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:13.087715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:13.108479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60aca3b-704a-4709-8eea-cb691cdfbdd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:13.113315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aca3b-704a-4709-8eea-cb691cdfbdd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:13.113745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aca3b-704a-4709-8eea-cb691cdfbdd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:13.114061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aca3b-704a-4709-8eea-cb691cdfbdd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:13.116472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a60aca3b-704a-4709-8eea-cb691cdfbdd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:13.124605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212855-b923-41c6-8ce0-4df6a850becd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:13.137880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:13.137880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:13.137880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:13.137880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] PATH_INFO: `/v3/projects/2600c77488cb4c7882998dc76799b9fc/users/0f668ef345144fc695d1fc0fffe1cb3e/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:13.138328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2600c77488cb4c7882998dc76799b9fc, user_id=0f668ef345144fc695d1fc0fffe1cb3e, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:13.139742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:13.151729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60aca3b-704a-4709-8eea-cb691cdfbdd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:13.153432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60aca3b-704a-4709-8eea-cb691cdfbdd2 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:13.153813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1026/2054] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:13 2022] PUT /identity/v3/projects/2600c77488cb4c7882998dc76799b9fc/users/0f668ef345144fc695d1fc0fffe1cb3e/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:13.163906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:13.166513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:13.197279 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1029/2055] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:13.200044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45c22-a39a-4f2d-9099-b6f977172330 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:13.200338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45c22-a39a-4f2d-9099-b6f977172330 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:13.202412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf45c22-a39a-4f2d-9099-b6f977172330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:13.280485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cbf45c22-a39a-4f2d-9099-b6f977172330 None None] MFA Rules not processed for user `0f668ef345144fc695d1fc0fffe1cb3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:13.281737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cbf45c22-a39a-4f2d-9099-b6f977172330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:13.350414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d31cc-eb0b-4894-96c5-b82f94183b28 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:13.359808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:13.360604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:13.360894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:13.361366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] PATH_INFO: `/v3/users/133ba8746ac34c5bbc9877db8b385ddc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:13.362668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=133ba8746ac34c5bbc9877db8b385ddc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:13.364902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:13.469968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1030/2056] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:13.483225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133ba8746ac34c5bbc9877db8b385ddc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:13.485781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:13.489548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:13.489548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:13.518721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133ba8746ac34c5bbc9877db8b385ddc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:13.520120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133ba8746ac34c5bbc9877db8b385ddc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:13.521197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] MFA Rules not processed for user `365336e9f59e4b65ae0a05554776b0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:13.529769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:13.534886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133ba8746ac34c5bbc9877db8b385ddc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:13.535908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133ba8746ac34c5bbc9877db8b385ddc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:13.589629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:13.599604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1027/2057] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:13 2022] DELETE /identity/v3/users/133ba8746ac34c5bbc9877db8b385ddc => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:13.646649 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1031/2058] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:13.666178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:13.668124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:13.772919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7ebf-9b12-4fcf-9a40-569a3cd949ac admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:13.779042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:13.780013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:13.780129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:13.780464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:13.782536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:13.933976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:13.936206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:13.953098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1028/2059] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:14.238941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:14.240524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:14.264022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba948dae-3630-407c-9372-bacfc5d88be0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:14.396145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:14.396934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:14.397205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:14.398308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] PATH_INFO: `/v3/projects/781b9701e20f4fbc9f6a9e1894e7ed10` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:14.399574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=781b9701e20f4fbc9f6a9e1894e7ed10)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:14.401498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:14.444525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:14.446714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:14.450710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:14.471124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf39ba-2578-48cf-9440-a9e1c5efbf58 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:14.486488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781b9701e20f4fbc9f6a9e1894e7ed10', 'request_id': 'req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.499051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781b9701e20f4fbc9f6a9e1894e7ed10', 'request_id': 'req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.509482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781b9701e20f4fbc9f6a9e1894e7ed10', 'request_id': 'req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.527968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] The token cache is being invalidate because project 781b9701e20f4fbc9f6a9e1894e7ed10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:14.528344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.529074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.531230 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1032/2060] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:14 2022] DELETE /identity/v3/projects/781b9701e20f4fbc9f6a9e1894e7ed10 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:14.534816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:14.536707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:14.586388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:14.587415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:14.587763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:14.588203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] PATH_INFO: `/v3/projects/f7520a7e2bdc4a10baa5ed56bae12769` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:14.589416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7520a7e2bdc4a10baa5ed56bae12769)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:14.591509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:14.642093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad3c6f-0df7-4b83-a30d-dadc9bedbc45 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:14.648151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:14.659382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7520a7e2bdc4a10baa5ed56bae12769', 'request_id': 'req-67e3b334-f956-4051-b352-4127af81fcd0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.669335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7520a7e2bdc4a10baa5ed56bae12769', 'request_id': 'req-67e3b334-f956-4051-b352-4127af81fcd0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.682284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7520a7e2bdc4a10baa5ed56bae12769', 'request_id': 'req-67e3b334-f956-4051-b352-4127af81fcd0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.696287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] The token cache is being invalidate because project f7520a7e2bdc4a10baa5ed56bae12769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:14.696753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.697473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-67e3b334-f956-4051-b352-4127af81fcd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.699642 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1029/2061] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:14 2022] DELETE /identity/v3/projects/f7520a7e2bdc4a10baa5ed56bae12769 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:14.705090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210616ee-d5be-4c7a-9154-97c80febfa0b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:14.706343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210616ee-d5be-4c7a-9154-97c80febfa0b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:14.706900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210616ee-d5be-4c7a-9154-97c80febfa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:14.738443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-210616ee-d5be-4c7a-9154-97c80febfa0b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:14.740407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-210616ee-d5be-4c7a-9154-97c80febfa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:14.745429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:14.746802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:14.747174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:14.747526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] PATH_INFO: `/v3/users/2def79099da04db7a6672771b71591c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:14.748918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2def79099da04db7a6672771b71591c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:14.751572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1030/2062] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:14.751720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:14.760758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3421384-8315-4b36-bd5a-e9b5dcd80d02 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:14.761320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3421384-8315-4b36-bd5a-e9b5dcd80d02 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:14.761779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3421384-8315-4b36-bd5a-e9b5dcd80d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:14.790215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b3421384-8315-4b36-bd5a-e9b5dcd80d02 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:14.791374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b3421384-8315-4b36-bd5a-e9b5dcd80d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:14.813467 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1031/2063] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:14.818841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3421384-8315-4b36-bd5a-e9b5dcd80d02 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:14.834811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3421384-8315-4b36-bd5a-e9b5dcd80d02 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:14.839461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2def79099da04db7a6672771b71591c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.840655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:14.841270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:14.841549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:14.841819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:14.842509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:14.842792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:14.844624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:14.856425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:14.861901 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1032/2064] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:49:14.862990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2def79099da04db7a6672771b71591c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.864020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:14.873732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2def79099da04db7a6672771b71591c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.883730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2def79099da04db7a6672771b71591c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.884091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2def79099da04db7a6672771b71591c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:14.888414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d06a6f-2889-4b87-a883-973973340007 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:14.892458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1033/2065] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:14 2022] DELETE /identity/v3/users/2def79099da04db7a6672771b71591c8 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:14.899131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:14.899904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:14.900245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:14.900582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:14.901283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:14.903339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:14.945488 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1033/2066] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:14 2022] POST /identity/v3/projects => generated 393 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:14.948000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:14.949543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:15.055342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34699143-7e59-4653-bbc1-040b77fdb596 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:15.134025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:15.134829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:15.135212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:15.135613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:15.136717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:15.138969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:15.166019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1034/2067] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:14 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:15.168340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:15.183712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfef305-8408-4316-abd3-33cc4412a60a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:15.189261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c19a2c-8402-49d9-a33f-5cc05b282016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:15.191201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c19a2c-8402-49d9-a33f-5cc05b282016 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:15.191634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c19a2c-8402-49d9-a33f-5cc05b282016 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:15.192060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c19a2c-8402-49d9-a33f-5cc05b282016 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:15.193047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c19a2c-8402-49d9-a33f-5cc05b282016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:15.196051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c19a2c-8402-49d9-a33f-5cc05b282016 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:15.212107 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1034/2068] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:15.213383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:15.228461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73070bec-d90a-4bee-9075-b5c20dcca814 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:15.251958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:15.255512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:15.256505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:15.278985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117fee1-976f-4c2f-bfb8-ecb325505446 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:15.279095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117fee1-976f-4c2f-bfb8-ecb325505446 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:15.280207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117fee1-976f-4c2f-bfb8-ecb325505446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:15.289305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:15.290717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:15.294808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:15.314516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1035/2069] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:15.319976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:15.345237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0449f76-7106-46a7-9e0a-8ee9c40fcd00 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:15.359448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb775d-032f-4171-9635-ba988a943aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:15.360127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb775d-032f-4171-9635-ba988a943aee admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:15.361760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb775d-032f-4171-9635-ba988a943aee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:15.362135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb775d-032f-4171-9635-ba988a943aee admin admin] PATH_INFO: `/v3/projects/465cb86fae594c0d97c43f087a2db0b5/users/811d70e726fd4aebb9a475d0feb1153d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:15.373814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e117fee1-976f-4c2f-bfb8-ecb325505446 None None] MFA Rules not processed for user `b77e361aa6cb477083688130e08db6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:15.386134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e117fee1-976f-4c2f-bfb8-ecb325505446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:15.391340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb775d-032f-4171-9635-ba988a943aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=465cb86fae594c0d97c43f087a2db0b5, user_id=811d70e726fd4aebb9a475d0feb1153d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:15.395057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb775d-032f-4171-9635-ba988a943aee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:15.423835 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1036/2070] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:15 2022] PUT /identity/v3/projects/465cb86fae594c0d97c43f087a2db0b5/users/811d70e726fd4aebb9a475d0feb1153d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:15.440633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6964973-9b82-49b5-b3c3-8d17888174fb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:15.441598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6964973-9b82-49b5-b3c3-8d17888174fb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:15.442072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6964973-9b82-49b5-b3c3-8d17888174fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:15.523040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e6964973-9b82-49b5-b3c3-8d17888174fb None None] MFA Rules not processed for user `811d70e726fd4aebb9a475d0feb1153d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:15.528047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e6964973-9b82-49b5-b3c3-8d17888174fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:15.621497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:15.652140 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1035/2071] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:15.726399 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1037/2072] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:15.732166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117fee1-976f-4c2f-bfb8-ecb325505446 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:15.760374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117fee1-976f-4c2f-bfb8-ecb325505446 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:15.880615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:15.912547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:15.914642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:15.915033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:15.915728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:15.917092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:15.919426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:15.973517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1036/2073] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:15 2022] POST /identity/v3/projects => generated 395 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:15.979417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6964973-9b82-49b5-b3c3-8d17888174fb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:16.024547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:16.026735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6964973-9b82-49b5-b3c3-8d17888174fb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:16.029057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:16.226648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:16.229993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.230284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.230570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:16.235323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:16.248465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:16.313590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d2f0f-fcaa-4f3c-85fe-60a56f7ea410 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:16.325134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:16.325643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.325981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.326283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] PATH_INFO: `/v3/projects/18dbf9e8023e42c8a747ba1efc2e6def` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:16.335959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18dbf9e8023e42c8a747ba1efc2e6def)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:16.340759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:16.360915 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1038/2074] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:15 2022] POST /identity/v3/users => generated 304 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:16.365607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:16.426166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ff306-9168-46e3-91db-0668be98d6a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:16.457106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:16.459374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.462056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.463521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:16.468499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:16.469059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18dbf9e8023e42c8a747ba1efc2e6def', 'request_id': 'req-3ecb5775-0ca8-45d1-bf82-b35074f0b227'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:16.478325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:16.485141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18dbf9e8023e42c8a747ba1efc2e6def', 'request_id': 'req-3ecb5775-0ca8-45d1-bf82-b35074f0b227'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:16.511140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18dbf9e8023e42c8a747ba1efc2e6def', 'request_id': 'req-3ecb5775-0ca8-45d1-bf82-b35074f0b227'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:16.539027 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1039/2075] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:16.541710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:16.568466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] The token cache is being invalidate because project 18dbf9e8023e42c8a747ba1efc2e6def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:16.573911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:16.574504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ae9de-d743-452c-867c-c22754b47fe4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:16.575392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3ecb5775-0ca8-45d1-bf82-b35074f0b227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:16.585429 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1037/2076] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:16 2022] DELETE /identity/v3/projects/18dbf9e8023e42c8a747ba1efc2e6def => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:16.595913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.596385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.598052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:16.634955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:16.636609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:16.720577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300aeac-4590-4000-b6b9-f08e2d3e1345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:16.721466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300aeac-4590-4000-b6b9-f08e2d3e1345 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.722072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300aeac-4590-4000-b6b9-f08e2d3e1345 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.722416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300aeac-4590-4000-b6b9-f08e2d3e1345 admin admin] PATH_INFO: `/v3/projects/55523014277a49d4b3f455ac5f7becf8/users/224d1f0c976e45018d6c24c6d4e6744c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:16.746347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300aeac-4590-4000-b6b9-f08e2d3e1345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55523014277a49d4b3f455ac5f7becf8, user_id=224d1f0c976e45018d6c24c6d4e6744c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:16.752355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300aeac-4590-4000-b6b9-f08e2d3e1345 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:16.773575 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1040/2077] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:16 2022] PUT /identity/v3/projects/55523014277a49d4b3f455ac5f7becf8/users/224d1f0c976e45018d6c24c6d4e6744c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:16.773883 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1038/2078] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:16.777230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:16.778881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:16.780966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c2c5a-2eec-4baa-a916-1475582b7e7b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.781599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c2c5a-2eec-4baa-a916-1475582b7e7b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.781986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c2c5a-2eec-4baa-a916-1475582b7e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:16.798466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a18fb-e817-4464-b44c-752a4e526e71 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:16.806090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:16.811671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-036c2c5a-2eec-4baa-a916-1475582b7e7b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:16.813053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-036c2c5a-2eec-4baa-a916-1475582b7e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:16.879124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:16.928248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:16.930115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.930494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.930772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:16.931426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:16.938380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:16.971457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1039/2079] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:16.976910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:16.991728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bda809-18b6-47e0-91cb-923a90b59c63 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:16.997581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866aaae-5290-442d-97e9-64c21f8befcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:16.998298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866aaae-5290-442d-97e9-64c21f8befcb admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:16.998582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866aaae-5290-442d-97e9-64c21f8befcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:16.998999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866aaae-5290-442d-97e9-64c21f8befcb admin admin] PATH_INFO: `/v3/projects/55523014277a49d4b3f455ac5f7becf8/users/224d1f0c976e45018d6c24c6d4e6744c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.001422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866aaae-5290-442d-97e9-64c21f8befcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=55523014277a49d4b3f455ac5f7becf8, user_id=224d1f0c976e45018d6c24c6d4e6744c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.004271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866aaae-5290-442d-97e9-64c21f8befcb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.012541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1041/2080] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:17.015197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c2c5a-2eec-4baa-a916-1475582b7e7b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.023883 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1040/2081] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:16 2022] PUT /identity/v3/projects/55523014277a49d4b3f455ac5f7becf8/users/224d1f0c976e45018d6c24c6d4e6744c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:17.035273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece6fc-9650-4ff2-8b42-07a9b787dacc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.035670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece6fc-9650-4ff2-8b42-07a9b787dacc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.047796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece6fc-9650-4ff2-8b42-07a9b787dacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.058303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c2c5a-2eec-4baa-a916-1475582b7e7b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.120971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d8ece6fc-9650-4ff2-8b42-07a9b787dacc None None] MFA Rules not processed for user `224d1f0c976e45018d6c24c6d4e6744c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:17.122703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d8ece6fc-9650-4ff2-8b42-07a9b787dacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:17.193277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.194102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.195065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.195564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.196469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:17.196782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.198697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.219108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1042/2082] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:17.222661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.236639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91608d41-5382-4952-ab42-7827d95383c4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.244876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.245682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.246102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.246636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.248100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.251777 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1041/2083] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:17.251918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.254169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece6fc-9650-4ff2-8b42-07a9b787dacc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.283024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece6fc-9650-4ff2-8b42-07a9b787dacc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.292305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.293954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.294298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.294709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.295544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.298305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.299002 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1043/2084] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:17 2022] POST /identity/v3/projects => generated 389 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:17.302996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.354693 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1042/2085] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:17 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:17.358437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.412310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9c1ed-856d-4ea8-bdab-8b259d3fbb8f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.485490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:17.504481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6f55d-7b8b-4762-9a28-d07e57380935 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.505860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:17.516758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.517859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.518415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.518909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.519746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.522316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.549952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.551819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.552180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.554341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.555897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.560335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.566046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1043/2086] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:17 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:17.571043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.600104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf797-f72f-4dc7-b2e1-365a726d35fa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.615342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.616612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.617016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.617499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.618254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.619855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.621058 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1044/2087] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:17 2022] POST /identity/v3/users => generated 295 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:17.625590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.638053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2b8f8-a603-4879-8a27-f202e4135147 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.650423 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1044/2088] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:17.651621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.660740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.661372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.661657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.662019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.663405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.667533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.673937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6cb07-8491-4332-b341-5c0706e7dc7f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.680930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.681546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.681908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.682187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.682838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.685217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.723520 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1045/2089] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:17.726110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.736823 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1045/2090] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:17.740997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.763826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fe832-2dd3-4014-90d2-6f1c493dff38 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.764539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca468bfb-bce1-4132-900b-fa8152712256 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.785664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.786431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.786673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.786960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.787257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.787725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.788458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.788850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.789272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] PATH_INFO: `/v3/projects/5fe10f8794e44897a983f3164f81e0bf/users/bccfcffe6dd94af48b53a57104d4f003/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:17.798912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.808903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe10f8794e44897a983f3164f81e0bf, user_id=bccfcffe6dd94af48b53a57104d4f003, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:17.812382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:17.825651 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1046/2091] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:17.826799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.830595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:17.841857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1046/2092] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:17 2022] PUT /identity/v3/projects/5fe10f8794e44897a983f3164f81e0bf/users/bccfcffe6dd94af48b53a57104d4f003/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:17.846067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:17.873095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b04c4-fd1c-4ce0-ad33-4b6cd0ce0426 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.966263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f46ced-455d-454a-98c4-a5d1c34afe2d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:17.982322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bbc7b-236e-426f-80fa-22e0a3da3093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:17.983029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bbc7b-236e-426f-80fa-22e0a3da3093 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:17.983637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bbc7b-236e-426f-80fa-22e0a3da3093 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:17.983928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bbc7b-236e-426f-80fa-22e0a3da3093 admin admin] PATH_INFO: `/v3/projects/96346a9cc8fc416fb0bdc356308280ab/users/a80a5386c470486e836eadbda8957dcd/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.019889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bbc7b-236e-426f-80fa-22e0a3da3093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96346a9cc8fc416fb0bdc356308280ab, user_id=a80a5386c470486e836eadbda8957dcd, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:18.023125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bbc7b-236e-426f-80fa-22e0a3da3093 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:18.051125 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1047/2093] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:17 2022] PUT /identity/v3/projects/96346a9cc8fc416fb0bdc356308280ab/users/a80a5386c470486e836eadbda8957dcd/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:18.053271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:18.055539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.055978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.057919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] PATH_INFO: `/v3/users/ffba8e54e6f14960b7f57c5e19d3cfae` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.059200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbddc5-b6b6-484d-91fc-bd7978f5f9c4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.060502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffba8e54e6f14960b7f57c5e19d3cfae)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:18.061258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbddc5-b6b6-484d-91fc-bd7978f5f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.061681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbddc5-b6b6-484d-91fc-bd7978f5f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.067104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:18.144518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c9dbddc5-b6b6-484d-91fc-bd7978f5f9c4 None None] MFA Rules not processed for user `bccfcffe6dd94af48b53a57104d4f003`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:18.163456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c9dbddc5-b6b6-484d-91fc-bd7978f5f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:18.193741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffba8e54e6f14960b7f57c5e19d3cfae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:18.217119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffba8e54e6f14960b7f57c5e19d3cfae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:18.231513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffba8e54e6f14960b7f57c5e19d3cfae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:18.244507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffba8e54e6f14960b7f57c5e19d3cfae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:18.245290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76428ca0-3df1-42ff-856a-1d39407135da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffba8e54e6f14960b7f57c5e19d3cfae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:18.256095 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1047/2094] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:17 2022] DELETE /identity/v3/users/ffba8e54e6f14960b7f57c5e19d3cfae => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:18.260927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.263522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.264475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.353526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] MFA Rules not processed for user `a80a5386c470486e836eadbda8957dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:18.357891 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1048/2095] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:18.358613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:18.358712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3172-1408-4551-990c-927934153256 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.358712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3172-1408-4551-990c-927934153256 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.358712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3172-1408-4551-990c-927934153256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.398437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-53af3172-1408-4551-990c-927934153256 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:18.401860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-53af3172-1408-4551-990c-927934153256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:18.407711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:18.427858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:18.527436 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1048/2096] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:18.527436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:18.527436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:18.529434 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1049/2097] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:18.530138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af3172-1408-4551-990c-927934153256 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:18.534297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-53af3172-1408-4551-990c-927934153256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:18.552208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af3172-1408-4551-990c-927934153256 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:18.558375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:18.559410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.559756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.560061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.560771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:18.563622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:18.608175 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1050/2098] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:18 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:18.608175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:18.619099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:18.704614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe90881-cd6b-43ef-9961-8bae57cd03c4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:18.711002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:18.713302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.713996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.714551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.722744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da95f02-a75a-43a0-b3e4-cf5d88abcfa9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:18.803278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:18.806385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:18.822087 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1051/2099] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:18.822980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:18.837071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b4f64-7c72-4d3b-bf33-d677502cdbe3 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:18.844307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:18.845442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.845848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.854197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.856385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:18.859767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:18.887341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:18.889076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.890875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.891280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.897992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:18.901063 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1052/2100] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:18 2022] POST /identity/v3/users => generated 300 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:18.905784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:18.926930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8eb1-7eec-495a-9ffd-69402eeccc52 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:18.939910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:18.943400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:18.943840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:18.945094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:18.946456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:18.952519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:18.982993 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1053/2101] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:18.987070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:19.012907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31353f-619f-4697-bde1-81afafaea44f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:19.024968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:19.027096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:19.028014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:19.029617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] PATH_INFO: `/v3/projects/b407799c2326425692dfc1c7a521762a/users/d8d290b0936a4b6cab5a340b3231d1de/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:19.038531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:19.061712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:19.070873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:19.075710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:19.076760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b407799c2326425692dfc1c7a521762a, user_id=d8d290b0936a4b6cab5a340b3231d1de, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:19.084388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:19.101487 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1049/2102] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:18 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:19.109326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:19.112478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:19.117253 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1054/2103] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:18 2022] PUT /identity/v3/projects/b407799c2326425692dfc1c7a521762a/users/d8d290b0936a4b6cab5a340b3231d1de/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:19.119261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:19.164949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe1076-3620-410f-9aa5-b337cb483e34 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:19.360838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca25dcd-73b9-4964-b044-af405dc3b93c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:19.373783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:19.374318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:19.374547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:19.374824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] PATH_INFO: `/v3/projects/d9204ddaf90a4685a4286c8ebfd76433` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:19.377296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a1487-4352-49d9-af9d-aa3027e17138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:19.377476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a1487-4352-49d9-af9d-aa3027e17138 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:19.377742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a1487-4352-49d9-af9d-aa3027e17138 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:19.378040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a1487-4352-49d9-af9d-aa3027e17138 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:19.378740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a1487-4352-49d9-af9d-aa3027e17138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:19.380501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a1487-4352-49d9-af9d-aa3027e17138 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:19.383837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9204ddaf90a4685a4286c8ebfd76433)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:19.385775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:19.414074 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1055/2104] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:19.417686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31941ba5-62a3-478b-ba24-9cb133a71368 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:19.421266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31941ba5-62a3-478b-ba24-9cb133a71368 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:19.421571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31941ba5-62a3-478b-ba24-9cb133a71368 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:19.423110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1056/2105] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:49:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:19.424603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31941ba5-62a3-478b-ba24-9cb133a71368 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:19.455778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31941ba5-62a3-478b-ba24-9cb133a71368 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:19.469054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:19.470396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:19.471117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:19.485630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] PATH_INFO: `/v3/projects/b407799c2326425692dfc1c7a521762a/users/d8d290b0936a4b6cab5a340b3231d1de/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:19.490491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b407799c2326425692dfc1c7a521762a, user_id=d8d290b0936a4b6cab5a340b3231d1de, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:19.497570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:19.550492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9204ddaf90a4685a4286c8ebfd76433', 'request_id': 'req-3218fbee-e566-4903-9d6d-a5a539f2ea72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:19.551050 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1057/2106] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:19 2022] PUT /identity/v3/projects/b407799c2326425692dfc1c7a521762a/users/d8d290b0936a4b6cab5a340b3231d1de/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:19.556053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:49:19.586032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9204ddaf90a4685a4286c8ebfd76433', 'request_id': 'req-3218fbee-e566-4903-9d6d-a5a539f2ea72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:19.602655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9204ddaf90a4685a4286c8ebfd76433', 'request_id': 'req-3218fbee-e566-4903-9d6d-a5a539f2ea72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:19.626002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] The token cache is being invalidate because project d9204ddaf90a4685a4286c8ebfd76433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:19.626673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:19.627576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3218fbee-e566-4903-9d6d-a5a539f2ea72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:19.629464 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1050/2107] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:19 2022] DELETE /identity/v3/projects/d9204ddaf90a4685a4286c8ebfd76433 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:19.630930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:19.631454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:19.631987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:19.686671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] MFA Rules not processed for user `d8d290b0936a4b6cab5a340b3231d1de`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:19.688568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:19.706986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:19.850140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1051/2108] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:19.850140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:19.851341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:19.872637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:19.896017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0aac4-7198-43ea-9411-27dfc29a75d2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:19.913056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224d1f0c976e45018d6c24c6d4e6744c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55523014277a49d4b3f455ac5f7becf8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:19.915574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:19.915574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:19.915574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] PATH_INFO: `/v3/projects/465cb86fae594c0d97c43f087a2db0b5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:19.917192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorizing `identity:get_project(project_id=465cb86fae594c0d97c43f087a2db0b5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:19.919461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:19.921547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1058/2109] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:49:19 2022] GET /identity/v3/projects/465cb86fae594c0d97c43f087a2db0b5 => generated 393 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:19.996394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b0220-8ee7-48ca-9502-0f700c3cf74a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:20.004540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:20.005211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:20.005491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:20.005765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] PATH_INFO: `/v3/users/0ea7f93584594749bb70e39d45cf44b3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:20.009600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea7f93584594749bb70e39d45cf44b3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:20.011675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:20.099948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea7f93584594749bb70e39d45cf44b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.152949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea7f93584594749bb70e39d45cf44b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.170895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea7f93584594749bb70e39d45cf44b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.186478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea7f93584594749bb70e39d45cf44b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.187421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8288252b-073a-4ae4-8f52-4a25dfa5fbd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea7f93584594749bb70e39d45cf44b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.206648 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1052/2110] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:19 2022] DELETE /identity/v3/users/0ea7f93584594749bb70e39d45cf44b3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:20.370134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:20.373473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:20.393001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:20.403272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b19d4-ad37-43e8-bbeb-467cf8a64f7c tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:20.513917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:20.515313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:20.515612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:20.518845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:20.521703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:20.640695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:20.642738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:20.658759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1059/2111] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:20.728393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:20.729692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c804439-d728-4c94-83c9-55c0f3018dca None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:20.730197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c804439-d728-4c94-83c9-55c0f3018dca None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:20.730479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c804439-d728-4c94-83c9-55c0f3018dca None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:20.731849 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1053/2112] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:49:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:20.732619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:20.770230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c804439-d728-4c94-83c9-55c0f3018dca None None] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:49:20.793643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c804439-d728-4c94-83c9-55c0f3018dca None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:20.795095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3c804439-d728-4c94-83c9-55c0f3018dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:20.854038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eabfce-765f-419e-8772-5113d40053c3 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:20.859986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:20.860634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:20.861311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:20.861596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] PATH_INFO: `/v3/projects/e4fdbb588d1a4dbeae77a55d6710d030` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:20.862791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fdbb588d1a4dbeae77a55d6710d030)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:20.866594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:20.899908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c804439-d728-4c94-83c9-55c0f3018dca None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:20.907528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224d1f0c976e45018d6c24c6d4e6744c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55523014277a49d4b3f455ac5f7becf8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:20.908604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:20.909129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:20.909610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] PATH_INFO: `/v3/projects/465cb86fae594c0d97c43f087a2db0b5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:20.911476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorizing `identity:get_project(project_id=465cb86fae594c0d97c43f087a2db0b5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:20.915334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:20.926075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1054/2113] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:49:20 2022] GET /identity/v3/projects/465cb86fae594c0d97c43f087a2db0b5 => generated 393 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:49:20.945762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:20.957624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fdbb588d1a4dbeae77a55d6710d030', 'request_id': 'req-b1998c28-07dc-4dad-84c6-1377696ea82e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.974438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fdbb588d1a4dbeae77a55d6710d030', 'request_id': 'req-b1998c28-07dc-4dad-84c6-1377696ea82e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.989012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fdbb588d1a4dbeae77a55d6710d030', 'request_id': 'req-b1998c28-07dc-4dad-84c6-1377696ea82e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:20.990442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.009667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff95bb-91c2-4595-8814-332a95c097ba tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.012925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] The token cache is being invalidate because project e4fdbb588d1a4dbeae77a55d6710d030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:21.013528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:21.014965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1998c28-07dc-4dad-84c6-1377696ea82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:21.016851 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1060/2114] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:20 2022] DELETE /identity/v3/projects/e4fdbb588d1a4dbeae77a55d6710d030 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:21.024631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3d555-75d0-4e0b-b392-8c91fc397918 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.026428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3d555-75d0-4e0b-b392-8c91fc397918 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.026918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3d555-75d0-4e0b-b392-8c91fc397918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.055219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:21.062917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9cf3d555-75d0-4e0b-b392-8c91fc397918 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:21.064793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9cf3d555-75d0-4e0b-b392-8c91fc397918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:21.132870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c43e5-9a9e-4678-a51c-e09d3c6851d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.134887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c43e5-9a9e-4678-a51c-e09d3c6851d2 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.136103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c43e5-9a9e-4678-a51c-e09d3c6851d2 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.136754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c43e5-9a9e-4678-a51c-e09d3c6851d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.140406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-de7c43e5-9a9e-4678-a51c-e09d3c6851d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:21.160975 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1061/2115] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:21.171424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eecf74-4eaf-4bc9-a329-00c3b259df1f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.171731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eecf74-4eaf-4bc9-a329-00c3b259df1f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.172001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eecf74-4eaf-4bc9-a329-00c3b259df1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.200490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-28eecf74-4eaf-4bc9-a329-00c3b259df1f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:21.202419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-28eecf74-4eaf-4bc9-a329-00c3b259df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:21.214052 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1062/2116] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:21.218096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eecf74-4eaf-4bc9-a329-00c3b259df1f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.230887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eecf74-4eaf-4bc9-a329-00c3b259df1f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.238536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c43e5-9a9e-4678-a51c-e09d3c6851d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.240185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c43e5-9a9e-4678-a51c-e09d3c6851d2 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.249404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.250078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.250369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.250646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.251470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:21.251579 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1055/2117] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:21.251782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.252903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.253235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.253490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.253602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.255603 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1056/2118] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:21.269478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:49:21.270752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:21.276439 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1063/2119] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:21.280662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.291671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9717a2c-4125-48f2-ba9b-0d52585cb300 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.295316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.296438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:21.300701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.303947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.305558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.305558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.307065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.312850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.373381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1064/2120] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:21 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:21.377048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.458289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb699c5-5ea6-45e4-81e3-0a95717eb75a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.548236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02aac-e0b6-4311-8acc-869feee07e86 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.554219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.554947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.555082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224d1f0c976e45018d6c24c6d4e6744c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55523014277a49d4b3f455ac5f7becf8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.555234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.555514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.555980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.556290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.556433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.556827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] PATH_INFO: `/v3/projects/465cb86fae594c0d97c43f087a2db0b5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.558278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.560190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorizing `identity:get_project(project_id=465cb86fae594c0d97c43f087a2db0b5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.562427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.573772 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1057/2121] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:49:21 2022] GET /identity/v3/projects/465cb86fae594c0d97c43f087a2db0b5 => generated 393 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:49:21.616442 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1065/2122] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:21 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:21.618304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.647172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9e19a-1489-4507-b3a0-5ca5505704e8 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.652288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.652909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.653196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.654807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.655659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.657857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.664937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:21.674551 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1058/2123] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:21.675118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.691930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eab713-4ee6-4db2-ae2e-1bd97cfaca1a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.719017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abf642-a950-42a5-b5b4-5b9beda32ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.719017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abf642-a950-42a5-b5b4-5b9beda32ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.719017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abf642-a950-42a5-b5b4-5b9beda32ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.719328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abf642-a950-42a5-b5b4-5b9beda32ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.720275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abf642-a950-42a5-b5b4-5b9beda32ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.728974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abf642-a950-42a5-b5b4-5b9beda32ee9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.766795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:21.776147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1066/2124] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:21.794055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94170615-de48-4a5b-a4d4-2be3f3aeddb8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:21.795761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:21.796606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.796909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.797332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] PATH_INFO: `/v3/projects/cda0d7766e3e4ddfa7471f9c9196745a/users/9232c6fb834f4f68a260f5e10212f891/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.831565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda0d7766e3e4ddfa7471f9c9196745a, user_id=9232c6fb834f4f68a260f5e10212f891, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:21.835384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:21.864912 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1059/2125] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:21 2022] PUT /identity/v3/projects/cda0d7766e3e4ddfa7471f9c9196745a/users/9232c6fb834f4f68a260f5e10212f891/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:21.875717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:21.876010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:21.876293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:21.889858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:21.962102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] MFA Rules not processed for user `9232c6fb834f4f68a260f5e10212f891`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:21.962102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:22.108603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:22.121037 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1067/2126] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:22.240855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:22.243488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:22.267627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f337c90-c5cf-4dac-a3a0-39f77e87faf2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:22.273166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:22.274308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:22.274734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:22.275109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:22.275826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:22.277863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:22.311277 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1060/2127] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:22 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:22.316328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:22.395831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b299-5838-4a16-a2de-a8014419626e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:22.461540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf5b0-427f-4074-8ee5-241e0eeef377 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:22.472829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:22.473564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:22.474400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:22.474400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:22.475116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:22.480327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:22.498440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:22.499074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:22.499360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:22.499660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:22.503738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:22.529067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1061/2128] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:22 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:22.530030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:22.531901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:22.550088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d9014-7b4b-417a-b2ae-86b182d4c03c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:22.555471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:22.558226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:22.559247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:22.559787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:22.560204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] PATH_INFO: `/v3/users/a80a5386c470486e836eadbda8957dcd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:22.562083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80a5386c470486e836eadbda8957dcd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:22.565396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:22.598244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:22.600398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:22.623078 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1068/2129] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:22.625720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:22.661257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:22.665393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80a5386c470486e836eadbda8957dcd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:22.694555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80a5386c470486e836eadbda8957dcd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:22.700203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9a0c4-e9f0-450e-8348-4379ef9b3ace service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:22.712300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:22.712917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80a5386c470486e836eadbda8957dcd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:22.713303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:22.713716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:22.714122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:22.715505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:22.717338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:22.732978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80a5386c470486e836eadbda8957dcd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:22.733756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6fbb4730-90e7-4758-9674-d65497d581d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80a5386c470486e836eadbda8957dcd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:22.738449 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1069/2130] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:22.739055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:22.742483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:22.744832 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1062/2131] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:22 2022] DELETE /identity/v3/users/a80a5386c470486e836eadbda8957dcd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:22.748955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455c70-d40e-4203-a122-8191d741b973 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:22.749249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455c70-d40e-4203-a122-8191d741b973 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:22.749741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455c70-d40e-4203-a122-8191d741b973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:22.766040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6eca9-2140-48fa-8cdf-107dbe891d2c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:22.814024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-87455c70-d40e-4203-a122-8191d741b973 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:22.818012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87455c70-d40e-4203-a122-8191d741b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:22.976583 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1063/2132] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:22.978912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:22.979652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:22.979927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:22.980236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] PATH_INFO: `/v3/projects/b2fd49585fde4c078fff9ded78b036a1/users/ed52a55d09dc4e2b89faf896307312ff/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.005842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fd49585fde4c078fff9ded78b036a1, user_id=ed52a55d09dc4e2b89faf896307312ff, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:23.008352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455c70-d40e-4203-a122-8191d741b973 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:23.010449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87455c70-d40e-4203-a122-8191d741b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:23.013604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:23.039717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1070/2133] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:22 2022] PUT /identity/v3/projects/b2fd49585fde4c078fff9ded78b036a1/users/ed52a55d09dc4e2b89faf896307312ff/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:23.045961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:23.149971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455c70-d40e-4203-a122-8191d741b973 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:23.186255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f2fdd-a91b-4fe6-9bf6-64e195a85c7e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:23.193730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:23.194900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.195165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.195517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.196246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:23.198099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:23.225361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1071/2134] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:23.226605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:23.252865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67c12c-55e5-4eec-bf37-1e29e6edd8f7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:23.266579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:23.267585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.267732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782ec26-c0ce-4baa-a978-fab9bef8a9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:23.267887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.268145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.268339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782ec26-c0ce-4baa-a978-fab9bef8a9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.268691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782ec26-c0ce-4baa-a978-fab9bef8a9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.268877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782ec26-c0ce-4baa-a978-fab9bef8a9b5 admin admin] PATH_INFO: `/v3/projects/b2fd49585fde4c078fff9ded78b036a1/users/ed52a55d09dc4e2b89faf896307312ff/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.270973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782ec26-c0ce-4baa-a978-fab9bef8a9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fd49585fde4c078fff9ded78b036a1, user_id=ed52a55d09dc4e2b89faf896307312ff, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:23.274731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782ec26-c0ce-4baa-a978-fab9bef8a9b5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:23.284881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:23.287058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:23.313345 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1072/2135] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:23 2022] PUT /identity/v3/projects/b2fd49585fde4c078fff9ded78b036a1/users/ed52a55d09dc4e2b89faf896307312ff/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:23.313467 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1064/2136] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:23.314653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4cdf-11dd-4e87-96a9-8e99fe26e939 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.315006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4cdf-11dd-4e87-96a9-8e99fe26e939 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.315315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4cdf-11dd-4e87-96a9-8e99fe26e939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.401916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:23.403813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:23.421296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-798b4cdf-11dd-4e87-96a9-8e99fe26e939 None None] MFA Rules not processed for user `ed52a55d09dc4e2b89faf896307312ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:23.422576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-798b4cdf-11dd-4e87-96a9-8e99fe26e939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:23.442720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:23.461508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:23.577278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04095a-5c25-446e-9e7b-23e08b58331b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:23.586447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1073/2137] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:23.600221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff8b6d-a9ad-46aa-9994-3060e5e86ee9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.600575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff8b6d-a9ad-46aa-9994-3060e5e86ee9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.600797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff8b6d-a9ad-46aa-9994-3060e5e86ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.630261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-aaff8b6d-a9ad-46aa-9994-3060e5e86ee9 None None] MFA Rules not processed for user `9232c6fb834f4f68a260f5e10212f891`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:23.632029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aaff8b6d-a9ad-46aa-9994-3060e5e86ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:23.691707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:23.692327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.692611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.692889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.695528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:23.721951 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1074/2138] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:23.721951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a097-54de-47a8-853e-309746db70e4 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.721951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a097-54de-47a8-853e-309746db70e4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.721951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a097-54de-47a8-853e-309746db70e4 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.726684 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1075/2139] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:49:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:23.727524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a097-54de-47a8-853e-309746db70e4 None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:49:23.729215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6771a097-54de-47a8-853e-309746db70e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:23.758456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a097-54de-47a8-853e-309746db70e4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:23.759797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6771a097-54de-47a8-853e-309746db70e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:23.796493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:23.798205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:23.810226 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1065/2140] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:23.854310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a097-54de-47a8-853e-309746db70e4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:23.859702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533992d9-e8d0-4e57-a719-dfddcd2fc744 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224d1f0c976e45018d6c24c6d4e6744c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55523014277a49d4b3f455ac5f7becf8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:23.860538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533992d9-e8d0-4e57-a719-dfddcd2fc744 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:23.860900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533992d9-e8d0-4e57-a719-dfddcd2fc744 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:23.861271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533992d9-e8d0-4e57-a719-dfddcd2fc744 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] PATH_INFO: `/v3/projects/465cb86fae594c0d97c43f087a2db0b5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:23.862539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533992d9-e8d0-4e57-a719-dfddcd2fc744 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorizing `identity:get_project(project_id=465cb86fae594c0d97c43f087a2db0b5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:23.865475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533992d9-e8d0-4e57-a719-dfddcd2fc744 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:23.869367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1076/2141] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:49:23 2022] GET /identity/v3/projects/465cb86fae594c0d97c43f087a2db0b5 => generated 393 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:24.040730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:24.059936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d5d1b-3ff7-4067-bd6d-40fabf2a79cd service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:24.066012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db456f-45db-478f-b8ee-183f39f817bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:24.066636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db456f-45db-478f-b8ee-183f39f817bf service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:24.066923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db456f-45db-478f-b8ee-183f39f817bf service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:24.067193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db456f-45db-478f-b8ee-183f39f817bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:24.069557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8db456f-45db-478f-b8ee-183f39f817bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:24.081053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db456f-45db-478f-b8ee-183f39f817bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:24.082735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db456f-45db-478f-b8ee-183f39f817bf service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:24.093035 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1066/2142] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:24.180092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7155740-3c3d-45a2-936e-74cf65ce9815 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:24.180515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7155740-3c3d-45a2-936e-74cf65ce9815 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:24.180833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7155740-3c3d-45a2-936e-74cf65ce9815 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:24.182795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1077/2143] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:24.193323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:24.194695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:24.194934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:24.225489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:24.226617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:24.280219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:24.365079 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1067/2144] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:49:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:24.440088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7155740-3c3d-45a2-936e-74cf65ce9815 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:24.446360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c7155740-3c3d-45a2-936e-74cf65ce9815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:24.479548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7155740-3c3d-45a2-936e-74cf65ce9815 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:24.486702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f9652-792d-4ecf-ba6e-1cac72959671 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:24.487433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f9652-792d-4ecf-ba6e-1cac72959671 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:24.487670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f9652-792d-4ecf-ba6e-1cac72959671 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:24.488572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f9652-792d-4ecf-ba6e-1cac72959671 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:24.501479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f9652-792d-4ecf-ba6e-1cac72959671 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:24.503801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f9652-792d-4ecf-ba6e-1cac72959671 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:24.520593 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1078/2145] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:24.652867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:24.655524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:24.823952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a015354-028b-45ca-9dc1-4a50e1a215d8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:24.830878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:24.831698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:24.832050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:24.832520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] PATH_INFO: `/v3/projects/96346a9cc8fc416fb0bdc356308280ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:24.833906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96346a9cc8fc416fb0bdc356308280ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:24.837491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:24.915333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6876-e703-4440-ba2c-d800b1b0763c None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:24.915999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6876-e703-4440-ba2c-d800b1b0763c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:24.916429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6876-e703-4440-ba2c-d800b1b0763c None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:24.919719 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1079/2146] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:49:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:24.929575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6876-e703-4440-ba2c-d800b1b0763c None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:49:24.938021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96346a9cc8fc416fb0bdc356308280ab', 'request_id': 'req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:24.945384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96346a9cc8fc416fb0bdc356308280ab', 'request_id': 'req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:24.952928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96346a9cc8fc416fb0bdc356308280ab', 'request_id': 'req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:24.980489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] The token cache is being invalidate because project 96346a9cc8fc416fb0bdc356308280ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:24.980798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:24.983730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:24.985678 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1068/2147] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:24 2022] DELETE /identity/v3/projects/96346a9cc8fc416fb0bdc356308280ab => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:24.989350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:24.991259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:25.041090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:25.087682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6876-e703-4440-ba2c-d800b1b0763c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:25.099883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8e4bc-9e47-4e0c-9e78-f276386f39f7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:25.106994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:25.108236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:25.108850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:25.109134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] PATH_INFO: `/v3/users/d8d290b0936a4b6cab5a340b3231d1de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:25.110504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d290b0936a4b6cab5a340b3231d1de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:25.114954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:25.124047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:25.182685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6876-e703-4440-ba2c-d800b1b0763c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:25.185602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d290b0936a4b6cab5a340b3231d1de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:25.189347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcbfea-2c3a-4efe-863b-a413a3fea7c2 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224d1f0c976e45018d6c24c6d4e6744c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55523014277a49d4b3f455ac5f7becf8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:25.190291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcbfea-2c3a-4efe-863b-a413a3fea7c2 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:25.190943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcbfea-2c3a-4efe-863b-a413a3fea7c2 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:25.191424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcbfea-2c3a-4efe-863b-a413a3fea7c2 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] PATH_INFO: `/v3/projects/465cb86fae594c0d97c43f087a2db0b5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:25.192780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcbfea-2c3a-4efe-863b-a413a3fea7c2 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorizing `identity:get_project(project_id=465cb86fae594c0d97c43f087a2db0b5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:25.197082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcbfea-2c3a-4efe-863b-a413a3fea7c2 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:25.199396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1080/2148] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:49:24 2022] GET /identity/v3/projects/465cb86fae594c0d97c43f087a2db0b5 => generated 393 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:25.211782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d290b0936a4b6cab5a340b3231d1de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:25.230831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d290b0936a4b6cab5a340b3231d1de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:25.244808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d290b0936a4b6cab5a340b3231d1de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:25.245781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d290b0936a4b6cab5a340b3231d1de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:25.253616 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1069/2149] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:24 2022] DELETE /identity/v3/users/d8d290b0936a4b6cab5a340b3231d1de => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:25.490585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962e92b-4bf3-4b49-a84c-0b96681efee7 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:25.491140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962e92b-4bf3-4b49-a84c-0b96681efee7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:25.491364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962e92b-4bf3-4b49-a84c-0b96681efee7 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:25.493794 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1081/2150] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:25.500875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:49:25.502358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:25.608517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:25.609973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:25.705322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c798e-674b-4639-a30d-8cb5f2fb0796 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:25.709370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962e92b-4bf3-4b49-a84c-0b96681efee7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:25.712721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224d1f0c976e45018d6c24c6d4e6744c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55523014277a49d4b3f455ac5f7becf8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:25.713423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:25.713898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:25.714305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] PATH_INFO: `/v3/projects/5fe10f8794e44897a983f3164f81e0bf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:25.715515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorizing `identity:get_project(project_id=5fe10f8794e44897a983f3164f81e0bf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:25.715849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f962e92b-4bf3-4b49-a84c-0b96681efee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:25.717879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:25.719625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1070/2151] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:49:25 2022] GET /identity/v3/projects/5fe10f8794e44897a983f3164f81e0bf => generated 393 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:25.738250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:25.745874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:25.800105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:25.862701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962e92b-4bf3-4b49-a84c-0b96681efee7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:25.869079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c946e-57e3-4b93-9454-ba098f08b041 tempest-FlavorsAccessNegativeTestJSON-2055522332 tempest-FlavorsAccessNegativeTestJSON-2055522332-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:25.869378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac49749-3f55-416d-b784-02f12c6de6fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:25.870060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac49749-3f55-416d-b784-02f12c6de6fb service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:25.870416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac49749-3f55-416d-b784-02f12c6de6fb service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:25.870757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac49749-3f55-416d-b784-02f12c6de6fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:25.872788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6ac49749-3f55-416d-b784-02f12c6de6fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:25.880370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:25.881907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:25.882393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:25.882938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] PATH_INFO: `/v3/projects/b407799c2326425692dfc1c7a521762a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:25.884238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b407799c2326425692dfc1c7a521762a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:25.886469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:25.889031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac49749-3f55-416d-b784-02f12c6de6fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:25.891094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac49749-3f55-416d-b784-02f12c6de6fb service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:25.925611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1082/2152] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:25.960611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b407799c2326425692dfc1c7a521762a', 'request_id': 'req-8224fa86-969b-4dbb-89d9-fb44c57d9e85'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:25.969015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b407799c2326425692dfc1c7a521762a', 'request_id': 'req-8224fa86-969b-4dbb-89d9-fb44c57d9e85'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:25.979255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b407799c2326425692dfc1c7a521762a', 'request_id': 'req-8224fa86-969b-4dbb-89d9-fb44c57d9e85'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.000755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] The token cache is being invalidate because project b407799c2326425692dfc1c7a521762a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:26.001115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.002311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8224fa86-969b-4dbb-89d9-fb44c57d9e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.004409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1071/2153] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:25 2022] DELETE /identity/v3/projects/b407799c2326425692dfc1c7a521762a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:26.010655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc7ab-c3f2-4013-b4ff-5095d633d9c6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.011661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc7ab-c3f2-4013-b4ff-5095d633d9c6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.011661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc7ab-c3f2-4013-b4ff-5095d633d9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.037371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9f6bc7ab-c3f2-4013-b4ff-5095d633d9c6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:26.039774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9f6bc7ab-c3f2-4013-b4ff-5095d633d9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:26.129359 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1083/2154] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:26.138775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c16323-7dbf-4b77-8233-da3656aae127 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.139299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c16323-7dbf-4b77-8233-da3656aae127 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.139299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c16323-7dbf-4b77-8233-da3656aae127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.165745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-02c16323-7dbf-4b77-8233-da3656aae127 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:26.167044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-02c16323-7dbf-4b77-8233-da3656aae127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:26.181611 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1072/2155] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:26.182962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bc7ab-c3f2-4013-b4ff-5095d633d9c6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.193865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bc7ab-c3f2-4013-b4ff-5095d633d9c6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.212500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.213086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.213350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.213623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.214473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:26.214786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:26.216462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:26.219453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:26.237715 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1084/2156] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:26.241998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c16323-7dbf-4b77-8233-da3656aae127 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.258650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c16323-7dbf-4b77-8233-da3656aae127 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.266824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.267509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.267858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.268237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.268938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:26.271092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:26.293641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.295494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:26.320848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d5d61-1aaf-46fc-b1f0-1cc2578d33de admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.321482 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1073/2157] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:26 2022] POST /identity/v3/projects => generated 411 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:26.322727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.426075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926fdf9-13c9-4947-8474-bdd0a888834b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.431979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.432296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.432831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.432992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.433102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.433388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] PATH_INFO: `/v3/users/811d70e726fd4aebb9a475d0feb1153d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.433492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.433758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.434547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:26.435443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=811d70e726fd4aebb9a475d0feb1153d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:26.436961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:26.437602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:26.474411 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1074/2158] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:26 2022] POST /identity/v3/users => generated 306 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:26.477490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.501294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b69655-4b30-4ab2-a631-3e8754a31e11 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.507057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811d70e726fd4aebb9a475d0feb1153d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.509506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.510172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.510465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.510739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.511624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:26.513989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:26.523123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811d70e726fd4aebb9a475d0feb1153d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.526245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811d70e726fd4aebb9a475d0feb1153d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.531576 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1075/2159] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:26.535894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.536503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811d70e726fd4aebb9a475d0feb1153d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.537310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d0ad39e8-55c6-4b3d-a17a-2871576c2e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811d70e726fd4aebb9a475d0feb1153d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:26.549410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:26.558338 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1085/2160] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:26 2022] DELETE /identity/v3/users/811d70e726fd4aebb9a475d0feb1153d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:26.561099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:26.562797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.563053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.563452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.567589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbee7c-fe23-4d6d-8565-8c08127c5b05 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.585183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:26.588655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:26.671141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.672377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.672848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.673308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.674982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:26.677070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:26.683495 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1086/2161] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:26.699605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.699605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:26.700461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1076/2162] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:26.701628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:26.718357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b80d6-63fb-460f-a321-6ad93e8c3880 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.722864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:26.752891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b626e6e-f12f-4280-ac91-ea9b1802f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.753578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b626e6e-f12f-4280-ac91-ea9b1802f18c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.755690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b626e6e-f12f-4280-ac91-ea9b1802f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.755955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b626e6e-f12f-4280-ac91-ea9b1802f18c admin admin] PATH_INFO: `/v3/projects/47b0e78e2b4c4d179016abee029d4f57/users/b38cb38ebf554156bac4eba0ad92ab95/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.805101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b626e6e-f12f-4280-ac91-ea9b1802f18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b0e78e2b4c4d179016abee029d4f57, user_id=b38cb38ebf554156bac4eba0ad92ab95, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:26.815763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b626e6e-f12f-4280-ac91-ea9b1802f18c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:26.855128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1077/2163] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:26 2022] PUT /identity/v3/projects/47b0e78e2b4c4d179016abee029d4f57/users/b38cb38ebf554156bac4eba0ad92ab95/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:26.857028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc5870-1c32-4b5b-aed8-70de581edb9e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.859339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc5870-1c32-4b5b-aed8-70de581edb9e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.859787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc5870-1c32-4b5b-aed8-70de581edb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:26.868726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea5894-5399-43a9-9436-a7caf945b620 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:26.914930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4acc5870-1c32-4b5b-aed8-70de581edb9e None None] MFA Rules not processed for user `b38cb38ebf554156bac4eba0ad92ab95`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:26.916763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4acc5870-1c32-4b5b-aed8-70de581edb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:26.974132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:26.975144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:26.975144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:26.975915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.092490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1078/2164] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:27.096464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc5870-1c32-4b5b-aed8-70de581edb9e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.114503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.116878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:27.137873 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1087/2165] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:27.227081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc5870-1c32-4b5b-aed8-70de581edb9e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:27.239440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:27.240156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.240550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.241506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.242386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.245978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:27.278977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:27.333420 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1079/2166] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:27 2022] POST /identity/v3/projects => generated 411 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:27.338123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.354517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48db06a-b73d-43e3-ac34-c7802a457b24 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:27.509441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:27.510103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.510374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.510651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.511295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.513250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:27.560818 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1088/2167] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:27 2022] POST /identity/v3/users => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:27.563852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.582138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.585061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb46622-736c-4a84-bb3b-4381ae3d7912 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:27.586847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:27.608514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:27.609147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.609429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.609997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.610467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f8dfd-939c-4a5f-a0dc-500261d580e5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:27.610638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.613273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:27.618095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:27.618888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.619269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.619638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] PATH_INFO: `/v3/projects/465cb86fae594c0d97c43f087a2db0b5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.622021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] RBAC: Authorizing `identity:delete_project(project_id=465cb86fae594c0d97c43f087a2db0b5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.633630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:27.637329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:27.642505 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1080/2168] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:27.649024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.677212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0b32-268c-4218-a5a6-2402eb0ca6db admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:27.689670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a9a16-daae-453c-8b21-60689bdae687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:27.690386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a9a16-daae-453c-8b21-60689bdae687 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.690669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a9a16-daae-453c-8b21-60689bdae687 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.693370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a9a16-daae-453c-8b21-60689bdae687 admin admin] PATH_INFO: `/v3/projects/17323cb776504d8a95ca8e8e4cc0c027/users/c48a6852b33241abb96d0d1eef56ab01/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.741751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a9a16-daae-453c-8b21-60689bdae687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17323cb776504d8a95ca8e8e4cc0c027, user_id=c48a6852b33241abb96d0d1eef56ab01, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.751900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a9a16-daae-453c-8b21-60689bdae687 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:27.791141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465cb86fae594c0d97c43f087a2db0b5', 'request_id': 'req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:27.795944 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1081/2169] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:27 2022] PUT /identity/v3/projects/17323cb776504d8a95ca8e8e4cc0c027/users/c48a6852b33241abb96d0d1eef56ab01/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:27.797588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf4cea-a9eb-4dc0-8a6e-f5e8b1f50f70 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.801056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf4cea-a9eb-4dc0-8a6e-f5e8b1f50f70 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.801365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf4cea-a9eb-4dc0-8a6e-f5e8b1f50f70 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.804674 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1082/2170] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:49:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:49:27.806840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf4cea-a9eb-4dc0-8a6e-f5e8b1f50f70 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.807302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465cb86fae594c0d97c43f087a2db0b5', 'request_id': 'req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:27.821668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465cb86fae594c0d97c43f087a2db0b5', 'request_id': 'req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:27.843044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] The token cache is being invalidate because project 465cb86fae594c0d97c43f087a2db0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:27.843354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:27.844113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff438d6e-7ea3-4bca-bcc1-3df0696f21be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:27.849140 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1089/2171] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:27 2022] DELETE /identity/v3/projects/465cb86fae594c0d97c43f087a2db0b5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:27.849140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.849140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.849140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.871783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:27.873072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:27.942893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf4cea-a9eb-4dc0-8a6e-f5e8b1f50f70 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:27.951540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:27.952458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.952746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.953028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.954174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.956092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:27.968450 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1083/2172] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:27.969373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.971436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:27.985501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b643b2a-5218-4a0a-9e5e-93415c6ce929 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:27.988203 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1090/2173] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:27.990093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:27.991279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:27.992153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:27.992338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:27.992438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:27.992703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] PATH_INFO: `/v3/users/224d1f0c976e45018d6c24c6d4e6744c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:27.993916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224d1f0c976e45018d6c24c6d4e6744c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:27.995878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:28.008306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b283c-4199-4149-8f65-6eb77cc0ebe0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:28.017258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:28.018319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:28.018319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:28.018536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] PATH_INFO: `/v3/projects/17323cb776504d8a95ca8e8e4cc0c027/users/c48a6852b33241abb96d0d1eef56ab01/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:28.020727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17323cb776504d8a95ca8e8e4cc0c027, user_id=c48a6852b33241abb96d0d1eef56ab01, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:28.024564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:28.037328 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1091/2174] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:27 2022] PUT /identity/v3/projects/17323cb776504d8a95ca8e8e4cc0c027/users/c48a6852b33241abb96d0d1eef56ab01/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:28.038635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:28.041109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:28.095534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224d1f0c976e45018d6c24c6d4e6744c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:28.119409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224d1f0c976e45018d6c24c6d4e6744c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:28.128750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224d1f0c976e45018d6c24c6d4e6744c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:28.137709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a173-fca5-4d98-98ec-e5e29c81eb19 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:28.140406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224d1f0c976e45018d6c24c6d4e6744c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:28.141428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09511682-0231-4fe6-b629-4a7d452f5c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224d1f0c976e45018d6c24c6d4e6744c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:28.151127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:28.151772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:28.152059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:28.152488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:28.154325 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1084/2175] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:27 2022] DELETE /identity/v3/users/224d1f0c976e45018d6c24c6d4e6744c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:28.156394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6b6dd-810d-4f87-ad9a-27e713c22c6c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:28.156696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6b6dd-810d-4f87-ad9a-27e713c22c6c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:28.156981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6b6dd-810d-4f87-ad9a-27e713c22c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:28.163392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:28.216208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bff6b6dd-810d-4f87-ad9a-27e713c22c6c None None] MFA Rules not processed for user `c48a6852b33241abb96d0d1eef56ab01`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:28.218680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bff6b6dd-810d-4f87-ad9a-27e713c22c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:28.234205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:28.245918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:28.348278 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1085/2176] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:28.359396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:28.360882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:28.383088 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1092/2177] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:28.404466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:28.405053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:28.405578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:28.430018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:28.431635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:28.501542 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1086/2178] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:28.530864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:28.532518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:28.547963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604332e-ffc3-4f0d-a585-9070570483f6 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:28.552070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:28.552688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:28.552975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:28.553801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:28.562202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:28.564011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:28.573259 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1093/2179] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:28.847320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:28.849906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:28.950393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16707cf0-fea3-4ef8-bbbf-898774f6b853 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:28.975331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:28.976082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:28.976494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:28.976997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:28.990634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:29.001111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:29.030395 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1087/2180] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:29.177713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:29.216793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5a07a-504e-42f8-b466-f6a6a897b406 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:29.234379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:29.235609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:29.235963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:29.236309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:29.239079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:29.332972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:29.335048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:29.387145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:29.391897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:29.451187 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1094/2181] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:29 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:29.503629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:29.536158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b9231-141f-4474-99f3-33d8039f77a1 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:29.542552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:29.543282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:29.543568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:29.543840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] PATH_INFO: `/v3/projects/55523014277a49d4b3f455ac5f7becf8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:29.548188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55523014277a49d4b3f455ac5f7becf8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:29.550306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:29.660486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55523014277a49d4b3f455ac5f7becf8', 'request_id': 'req-119f805f-c763-4abb-9127-3c4785b9ae95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.674896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55523014277a49d4b3f455ac5f7becf8', 'request_id': 'req-119f805f-c763-4abb-9127-3c4785b9ae95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.684516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55523014277a49d4b3f455ac5f7becf8', 'request_id': 'req-119f805f-c763-4abb-9127-3c4785b9ae95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.703661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] The token cache is being invalidate because project 55523014277a49d4b3f455ac5f7becf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:29.704105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.704897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.726204 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1088/2182] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:29 2022] DELETE /identity/v3/projects/55523014277a49d4b3f455ac5f7becf8 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:29.727495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:29.739640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30478875-49f6-4f0a-aaa5-a4d266c5f213 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:29.754529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:29.756930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:29.828982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:29.829710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:29.830033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:29.830312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] PATH_INFO: `/v3/users/bccfcffe6dd94af48b53a57104d4f003` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:29.831585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccfcffe6dd94af48b53a57104d4f003)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:29.833491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:29.847998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f805f-c763-4abb-9127-3c4785b9ae95 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:29.867883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:29.868779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:29.873790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:29.874179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] PATH_INFO: `/v3/users/b38cb38ebf554156bac4eba0ad92ab95` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:29.875736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38cb38ebf554156bac4eba0ad92ab95)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:29.887228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:29.895989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:29.909548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccfcffe6dd94af48b53a57104d4f003'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.924217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccfcffe6dd94af48b53a57104d4f003'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.924994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccfcffe6dd94af48b53a57104d4f003'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.940255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccfcffe6dd94af48b53a57104d4f003'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.941083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a79594e7-ce87-4388-bda8-8783a0b0fbb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccfcffe6dd94af48b53a57104d4f003'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:29.982069 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1095/2183] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:29 2022] DELETE /identity/v3/users/bccfcffe6dd94af48b53a57104d4f003 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:29.986671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38cb38ebf554156bac4eba0ad92ab95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.006785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38cb38ebf554156bac4eba0ad92ab95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.016397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38cb38ebf554156bac4eba0ad92ab95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.019650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:30.029182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38cb38ebf554156bac4eba0ad92ab95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.030169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38cb38ebf554156bac4eba0ad92ab95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.046694 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1089/2184] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:29 2022] DELETE /identity/v3/users/b38cb38ebf554156bac4eba0ad92ab95 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:30.051373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:30.051751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:30.052104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:30.096908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:30.099767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:30.259602 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1096/2185] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:30.266473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:30.267988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:30.365355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248b6a5-bab6-456c-82d5-9c1df8e9d216 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:30.370291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80f5d3-f8f0-4866-9243-46993239b5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:30.376913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80f5d3-f8f0-4866-9243-46993239b5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:30.377207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80f5d3-f8f0-4866-9243-46993239b5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:30.377488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80f5d3-f8f0-4866-9243-46993239b5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:30.411972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80f5d3-f8f0-4866-9243-46993239b5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:30.413466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80f5d3-f8f0-4866-9243-46993239b5a3 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:30.432974 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1090/2186] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:30.476320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:30.477784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:30.490041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f5979-1d88-4c78-ae7d-91bbb7bfb204 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:30.495130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:30.495763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:30.496235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:30.496509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] PATH_INFO: `/v3/projects/5fe10f8794e44897a983f3164f81e0bf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:30.497812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe10f8794e44897a983f3164f81e0bf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:30.499756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:30.589571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe10f8794e44897a983f3164f81e0bf', 'request_id': 'req-6e0aecde-7cd5-4c8e-9252-fb0381a15508'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.596343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe10f8794e44897a983f3164f81e0bf', 'request_id': 'req-6e0aecde-7cd5-4c8e-9252-fb0381a15508'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.603966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe10f8794e44897a983f3164f81e0bf', 'request_id': 'req-6e0aecde-7cd5-4c8e-9252-fb0381a15508'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.617893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] The token cache is being invalidate because project 5fe10f8794e44897a983f3164f81e0bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:30.618197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.618936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6e0aecde-7cd5-4c8e-9252-fb0381a15508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:30.621088 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1097/2187] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:30 2022] DELETE /identity/v3/projects/5fe10f8794e44897a983f3164f81e0bf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:30.629272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf11eb3-bb8d-414e-99cc-f9c8907d0b14 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:30.629554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf11eb3-bb8d-414e-99cc-f9c8907d0b14 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:30.629946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf11eb3-bb8d-414e-99cc-f9c8907d0b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:30.657124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5bf11eb3-bb8d-414e-99cc-f9c8907d0b14 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:30.658905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5bf11eb3-bb8d-414e-99cc-f9c8907d0b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:30.742031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1091/2188] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:30.751314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926a666-7fff-4a3e-8384-34ebb6c62bf0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:30.751748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926a666-7fff-4a3e-8384-34ebb6c62bf0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:30.751936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926a666-7fff-4a3e-8384-34ebb6c62bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:30.775333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b926a666-7fff-4a3e-8384-34ebb6c62bf0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:30.776731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b926a666-7fff-4a3e-8384-34ebb6c62bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:30.785003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1098/2189] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:30.790316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf11eb3-bb8d-414e-99cc-f9c8907d0b14 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:30.801767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf11eb3-bb8d-414e-99cc-f9c8907d0b14 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:30.807154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:30.807773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:30.808058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:30.808333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:30.809022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:30.809330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:30.811006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:30.829734 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1092/2190] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:30.834452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926a666-7fff-4a3e-8384-34ebb6c62bf0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:30.845859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926a666-7fff-4a3e-8384-34ebb6c62bf0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:30.851026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:30.852062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:30.852479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:30.852754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:30.853402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:30.855353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:30.906698 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1099/2191] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:30 2022] POST /identity/v3/projects => generated 393 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:30.910649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:30.922374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91ade6-79fe-434f-86ce-0751c54653e9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.021503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.022212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.022657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.022937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.023589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.025524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.061286 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1093/2192] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:30 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:31.065258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.082691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561e675-5c8f-4438-ae22-ed62e42849d4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.094542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.095433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.095710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.095988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.096723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.099057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.104804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:31.113718 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1100/2193] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:31.116274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.125142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.130978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c5de4-a293-4ef2-a080-213a898945db admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.132158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.147926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.149394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.149678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.150118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.150782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.154842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a01ad-7e22-42bb-bac1-bac24b12a5a7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.155385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.164204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.164915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.165262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.165645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] PATH_INFO: `/v3/users/f6c7299bc6fd4163b360528217f0b679` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.167817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c7299bc6fd4163b360528217f0b679)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.170268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:31.171379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.183335 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1094/2194] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:31.184634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.190953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.209060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584a31a-cc2a-41ec-a964-c291d6b2ec60 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.217311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.219062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.219420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.219761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] PATH_INFO: `/v3/projects/47b0e78e2b4c4d179016abee029d4f57` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.221871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b0e78e2b4c4d179016abee029d4f57)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.224363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.266729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c7299bc6fd4163b360528217f0b679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.280805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c7299bc6fd4163b360528217f0b679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.281814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c7299bc6fd4163b360528217f0b679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.293010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c7299bc6fd4163b360528217f0b679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.294314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c7299bc6fd4163b360528217f0b679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.295903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b0e78e2b4c4d179016abee029d4f57', 'request_id': 'req-18236df7-b981-4fbd-a807-26cce89c860c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.306190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b0e78e2b4c4d179016abee029d4f57', 'request_id': 'req-18236df7-b981-4fbd-a807-26cce89c860c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.318329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b0e78e2b4c4d179016abee029d4f57', 'request_id': 'req-18236df7-b981-4fbd-a807-26cce89c860c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.325871 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1101/2195] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:31 2022] DELETE /identity/v3/users/f6c7299bc6fd4163b360528217f0b679 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:31.327219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.332657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.349220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] The token cache is being invalidate because project 47b0e78e2b4c4d179016abee029d4f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:31.349573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.350982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.352857 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1095/2196] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:31 2022] DELETE /identity/v3/projects/47b0e78e2b4c4d179016abee029d4f57 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:31.358500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.360645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.445262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23187cfa-5803-484c-91ae-2492e9f00042 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.450429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e64db-db8c-46c0-b43e-d54d04a74b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.451169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e64db-db8c-46c0-b43e-d54d04a74b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.451390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e64db-db8c-46c0-b43e-d54d04a74b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.451668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e64db-db8c-46c0-b43e-d54d04a74b06 admin admin] PATH_INFO: `/v3/projects/9df7d4075cce40afb8851d0d51cd57ea/users/0e8d642f1f244a3d9abac7529fe94a0b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.482351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e64db-db8c-46c0-b43e-d54d04a74b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df7d4075cce40afb8851d0d51cd57ea, user_id=0e8d642f1f244a3d9abac7529fe94a0b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.482351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e64db-db8c-46c0-b43e-d54d04a74b06 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.503147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18236df7-b981-4fbd-a807-26cce89c860c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.503884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1102/2197] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:31 2022] PUT /identity/v3/projects/9df7d4075cce40afb8851d0d51cd57ea/users/0e8d642f1f244a3d9abac7529fe94a0b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:31.515440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.515818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.516100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.549244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] MFA Rules not processed for user `0e8d642f1f244a3d9abac7529fe94a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:31.550647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.578087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.578604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.578886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.581344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] PATH_INFO: `/v3/users/c48a6852b33241abb96d0d1eef56ab01` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.582640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48a6852b33241abb96d0d1eef56ab01)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.585182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.617373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:31.642709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:31.650424 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1103/2198] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:31.651391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.652857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.662484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48a6852b33241abb96d0d1eef56ab01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.672334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1bf8d-a4fa-4aaf-9ba8-f09d7f872636 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.678638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.679553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.679989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.680337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] PATH_INFO: `/v3/projects/60e7e4ba813748518f453fb87b2f0e17` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.681970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e7e4ba813748518f453fb87b2f0e17)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.683567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48a6852b33241abb96d0d1eef56ab01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.684591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.692851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48a6852b33241abb96d0d1eef56ab01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.703514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48a6852b33241abb96d0d1eef56ab01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.704423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48a6852b33241abb96d0d1eef56ab01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.716190 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1096/2199] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:31 2022] DELETE /identity/v3/users/c48a6852b33241abb96d0d1eef56ab01 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:31.720190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.727062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.744252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e7e4ba813748518f453fb87b2f0e17', 'request_id': 'req-037b1b13-4fb7-4f3e-9239-3041d853930f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.754671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e7e4ba813748518f453fb87b2f0e17', 'request_id': 'req-037b1b13-4fb7-4f3e-9239-3041d853930f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.767370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e7e4ba813748518f453fb87b2f0e17', 'request_id': 'req-037b1b13-4fb7-4f3e-9239-3041d853930f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.784165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] The token cache is being invalidate because project 60e7e4ba813748518f453fb87b2f0e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:31.784477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.785525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.788169 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1104/2200] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:31 2022] DELETE /identity/v3/projects/60e7e4ba813748518f453fb87b2f0e17 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:31.792170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.794414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.816195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709d216-46c4-4d05-ac37-5dc216ffedc3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.869660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b1b13-4fb7-4f3e-9239-3041d853930f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:31.876990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.878302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.879186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.879809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] PATH_INFO: `/v3/users/b77e361aa6cb477083688130e08db6c0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.881534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77e361aa6cb477083688130e08db6c0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.884222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.885479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:31.886232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:31.886486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:31.886952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:31.887653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:31.889627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:31.931283 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1097/2201] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:31 2022] POST /identity/v3/projects => generated 391 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:31.935791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:31.937477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:31.945670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77e361aa6cb477083688130e08db6c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.953552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77e361aa6cb477083688130e08db6c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.954247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77e361aa6cb477083688130e08db6c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.968381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77e361aa6cb477083688130e08db6c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.969477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77e361aa6cb477083688130e08db6c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:31.979902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:31.994318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:32.004768 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1105/2202] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:31 2022] DELETE /identity/v3/users/b77e361aa6cb477083688130e08db6c0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:32.023521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba4b2d-03c9-4c58-9f4c-38abb42fa7f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.077523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.079192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.145492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.146168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.146451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.148029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.148706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.150850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.198099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb6d64-bfa0-42cd-bfef-1ee7e9e18e24 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.206229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.206888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1098/2203] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:31 2022] POST /identity/v3/users => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:32.207012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.207169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.207460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] PATH_INFO: `/v3/projects/17323cb776504d8a95ca8e8e4cc0c027` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.208610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17323cb776504d8a95ca8e8e4cc0c027)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.211407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.213038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.214798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.229588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e8b5a-e17e-41a2-b797-5337870d06e9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.239542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.240335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.241100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.241100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.242010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.243747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.268237 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1099/2204] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:32.269731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.275500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.293687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f66db4-6b90-404d-812d-14c5c9ddf2d8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.300321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.300937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.301224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.301680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] PATH_INFO: `/v3/projects/e9832c54f08946e699d60a980cc621e5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.303464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9832c54f08946e699d60a980cc621e5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.305587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.312636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17323cb776504d8a95ca8e8e4cc0c027', 'request_id': 'req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.319733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17323cb776504d8a95ca8e8e4cc0c027', 'request_id': 'req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.328149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17323cb776504d8a95ca8e8e4cc0c027', 'request_id': 'req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.340061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] The token cache is being invalidate because project 17323cb776504d8a95ca8e8e4cc0c027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:32.340346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.341038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.346547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1106/2205] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:32 2022] DELETE /identity/v3/projects/17323cb776504d8a95ca8e8e4cc0c027 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:32.346547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.346547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.358361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9832c54f08946e699d60a980cc621e5', 'request_id': 'req-dde814cd-02ea-4a5c-932d-15a4d20db2c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.364328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9832c54f08946e699d60a980cc621e5', 'request_id': 'req-dde814cd-02ea-4a5c-932d-15a4d20db2c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.371785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9832c54f08946e699d60a980cc621e5', 'request_id': 'req-dde814cd-02ea-4a5c-932d-15a4d20db2c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.384925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] The token cache is being invalidate because project e9832c54f08946e699d60a980cc621e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:32.385216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.385918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dde814cd-02ea-4a5c-932d-15a4d20db2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:32.387431 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1100/2206] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:32 2022] DELETE /identity/v3/projects/e9832c54f08946e699d60a980cc621e5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:32.392160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07a0e5-0b3d-49ee-9fd1-871a08421c07 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.392447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07a0e5-0b3d-49ee-9fd1-871a08421c07 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.392723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07a0e5-0b3d-49ee-9fd1-871a08421c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.411120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b6a0b-932a-4f3b-94e3-a05c0d844e0b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.413587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9f07a0e5-0b3d-49ee-9fd1-871a08421c07 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:32.414948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9f07a0e5-0b3d-49ee-9fd1-871a08421c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.431778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:32.462137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:32.476186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3efd80-3b12-4bbb-abb5-f7c9cbf3bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.476863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3efd80-3b12-4bbb-abb5-f7c9cbf3bab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.477145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3efd80-3b12-4bbb-abb5-f7c9cbf3bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.477420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3efd80-3b12-4bbb-abb5-f7c9cbf3bab2 admin admin] PATH_INFO: `/v3/projects/539e7c8210ac417b99d003f6489c160a/users/68603c914f9d4748869d95be13c74aac/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.481290 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1101/2207] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:32.482538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.482894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.483242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.496126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3efd80-3b12-4bbb-abb5-f7c9cbf3bab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539e7c8210ac417b99d003f6489c160a, user_id=68603c914f9d4748869d95be13c74aac, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.499024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3efd80-3b12-4bbb-abb5-f7c9cbf3bab2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.508570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:32.509844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.518378 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1107/2208] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:32 2022] PUT /identity/v3/projects/539e7c8210ac417b99d003f6489c160a/users/68603c914f9d4748869d95be13c74aac/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:32.520351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc5242-6d08-4f76-b9a0-b160183a704c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.520848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc5242-6d08-4f76-b9a0-b160183a704c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.521276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc5242-6d08-4f76-b9a0-b160183a704c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.558491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f6cc5242-6d08-4f76-b9a0-b160183a704c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:32.559768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f6cc5242-6d08-4f76-b9a0-b160183a704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.585641 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1102/2209] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:32.586880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.588499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.603489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38556aa-286c-487e-8863-4eacf47b3270 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.614690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c740f-08e7-4932-9d01-f6c83081e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.616329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c740f-08e7-4932-9d01-f6c83081e7ff admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.616718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c740f-08e7-4932-9d01-f6c83081e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.617098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c740f-08e7-4932-9d01-f6c83081e7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.617925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c740f-08e7-4932-9d01-f6c83081e7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.619963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c740f-08e7-4932-9d01-f6c83081e7ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.635689 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1103/2210] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:32.635689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66c1dd-50e6-4f71-ba90-421fa4219653 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.635689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66c1dd-50e6-4f71-ba90-421fa4219653 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.635689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66c1dd-50e6-4f71-ba90-421fa4219653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.640447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1108/2211] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:32.641688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc5242-6d08-4f76-b9a0-b160183a704c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.658438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2e66c1dd-50e6-4f71-ba90-421fa4219653 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:32.659680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2e66c1dd-50e6-4f71-ba90-421fa4219653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.661061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc5242-6d08-4f76-b9a0-b160183a704c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.672677 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1104/2212] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:32.672677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66c1dd-50e6-4f71-ba90-421fa4219653 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.680368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.681129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.681370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66c1dd-50e6-4f71-ba90-421fa4219653 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.681507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.681937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] PATH_INFO: `/v3/projects/539e7c8210ac417b99d003f6489c160a/users/68603c914f9d4748869d95be13c74aac/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.684113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=539e7c8210ac417b99d003f6489c160a, user_id=68603c914f9d4748869d95be13c74aac, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.687884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97777e1d-2b51-466f-ae58-024971cd666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.688170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.688500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97777e1d-2b51-466f-ae58-024971cd666b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.688784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97777e1d-2b51-466f-ae58-024971cd666b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.689065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97777e1d-2b51-466f-ae58-024971cd666b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.689713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97777e1d-2b51-466f-ae58-024971cd666b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:32.690572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97777e1d-2b51-466f-ae58-024971cd666b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.693067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97777e1d-2b51-466f-ae58-024971cd666b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.709962 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1109/2213] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:32 2022] PUT /identity/v3/projects/539e7c8210ac417b99d003f6489c160a/users/68603c914f9d4748869d95be13c74aac/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:32.709962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.731631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1105/2214] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:32.733124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76a996-a7ed-4b94-a031-5b72cb2f5edf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.733408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76a996-a7ed-4b94-a031-5b72cb2f5edf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.733686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76a996-a7ed-4b94-a031-5b72cb2f5edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.768875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0a76a996-a7ed-4b94-a031-5b72cb2f5edf None None] MFA Rules not processed for user `68603c914f9d4748869d95be13c74aac`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:32.771047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a76a996-a7ed-4b94-a031-5b72cb2f5edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:32.810032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53e238-8776-4075-b764-b6e2c8a00dae admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.816259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.817582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.817897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.818171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.819151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:32.819276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.820808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.833909 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1110/2215] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:32.834814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.849166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79432-d282-4c39-be1a-0308b784afba admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.854775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.855388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.855826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.856107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.856776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.858744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.867850 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1106/2216] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:32.868509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76a996-a7ed-4b94-a031-5b72cb2f5edf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.883853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76a996-a7ed-4b94-a031-5b72cb2f5edf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.889178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:32.889784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:32.890102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:32.890376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:32.891027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:32.892946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:32.910434 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1111/2217] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:32.911586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.921298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:32.923223 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1107/2218] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:32.924285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:32.951061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:32.986370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbd66a-31a0-423a-a1b5-5a0f9fc8f4d3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:32.997147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d70684-b7ad-4588-a6bc-a5b9f44e6fb1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.002120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.002834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.003215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.003609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.004260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.006406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.027990 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1108/2219] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:32 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:33.028784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.042037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460aea7-389c-4e13-b055-50c00c5daad0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.046714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.047305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.047587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.047861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.048121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.048482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.048763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.049041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.049324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] PATH_INFO: `/v3/users/0e8d642f1f244a3d9abac7529fe94a0b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.050422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.050648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8d642f1f244a3d9abac7529fe94a0b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.052620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.075613 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1109/2220] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:33 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:33.075613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.087551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a373e7f-013d-4c0b-8faa-3ffdd0a04f93 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.092531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.093041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.093332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.093680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.094432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.096447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.100840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8d642f1f244a3d9abac7529fe94a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:33.107633 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1110/2221] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:33.108267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8d642f1f244a3d9abac7529fe94a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:33.108508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.111082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8d642f1f244a3d9abac7529fe94a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:33.118532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cd11a-3358-47a9-8c64-8a63732c0a9c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.119890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8d642f1f244a3d9abac7529fe94a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:33.120700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8d642f1f244a3d9abac7529fe94a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:33.123713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.124303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.124603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.124855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.125481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.127088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.138305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1111/2222] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:33.139194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.140151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1112/2223] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:32 2022] DELETE /identity/v3/users/0e8d642f1f244a3d9abac7529fe94a0b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:33.140846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.142487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.143997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.154975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be797b-4d3c-4152-a461-b7b940d32a88 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.181891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:33.215297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b9b9f-cc42-47cf-991b-e585a2f1edfa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.215518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03bee1-4d07-4f77-9615-76c4447d6c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.216136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03bee1-4d07-4f77-9615-76c4447d6c19 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.216445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03bee1-4d07-4f77-9615-76c4447d6c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.216677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03bee1-4d07-4f77-9615-76c4447d6c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.217318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03bee1-4d07-4f77-9615-76c4447d6c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.218946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03bee1-4d07-4f77-9615-76c4447d6c19 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.220563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.221166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.221431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.221699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.222549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.224594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.231276 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1112/2224] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:33.232341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d560ee-f12e-4b7a-8c8c-a00bc504ab38 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.232618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d560ee-f12e-4b7a-8c8c-a00bc504ab38 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.232899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d560ee-f12e-4b7a-8c8c-a00bc504ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.242647 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1113/2225] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:49:33.242647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.253268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbcbd9-a4fa-4372-819e-08368708eca7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.254173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-88d560ee-f12e-4b7a-8c8c-a00bc504ab38 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:33.255747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-88d560ee-f12e-4b7a-8c8c-a00bc504ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.258783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.259497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.259720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.260028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] PATH_INFO: `/v3/projects/5bd065954b0e4520a3c1deba0916db85/users/f74a2c8cd22c454fa8fdd7c2e1ef9230/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.264689 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1113/2226] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:33.265587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d560ee-f12e-4b7a-8c8c-a00bc504ab38 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.276216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d560ee-f12e-4b7a-8c8c-a00bc504ab38 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.276416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd065954b0e4520a3c1deba0916db85, user_id=f74a2c8cd22c454fa8fdd7c2e1ef9230, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.279528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.280897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb427a31-eccb-439c-81a0-0dbfb27c57e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.281557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb427a31-eccb-439c-81a0-0dbfb27c57e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.281858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb427a31-eccb-439c-81a0-0dbfb27c57e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.282149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb427a31-eccb-439c-81a0-0dbfb27c57e5 admin admin] PATH_INFO: `/v3/projects/c82fb214ece04574bf521a45ba055bdf/users/7721fd95d802434d905b50d23eaad9b7/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.298063 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1114/2227] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:33 2022] PUT /identity/v3/projects/5bd065954b0e4520a3c1deba0916db85/users/f74a2c8cd22c454fa8fdd7c2e1ef9230/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:33.299051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.299304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb427a31-eccb-439c-81a0-0dbfb27c57e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82fb214ece04574bf521a45ba055bdf, user_id=7721fd95d802434d905b50d23eaad9b7, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.302114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb427a31-eccb-439c-81a0-0dbfb27c57e5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.302718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.325915 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1114/2228] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:33 2022] PUT /identity/v3/projects/c82fb214ece04574bf521a45ba055bdf/users/7721fd95d802434d905b50d23eaad9b7/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:33.328309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477d42-1967-4871-94a7-18219816e826 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.328707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477d42-1967-4871-94a7-18219816e826 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.329083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477d42-1967-4871-94a7-18219816e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.351974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f3477d42-1967-4871-94a7-18219816e826 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:33.353404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f3477d42-1967-4871-94a7-18219816e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.372830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cbd69-9623-4cfa-a163-4bb6d859a096 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.397702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:33.433289 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1115/2229] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:33.434732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac11fc6-e2f9-4d1f-81e0-5bf12b280fc2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.435016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac11fc6-e2f9-4d1f-81e0-5bf12b280fc2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.435828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac11fc6-e2f9-4d1f-81e0-5bf12b280fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.450109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.450931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.451309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.451742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.462215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.463820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.468397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6ac11fc6-e2f9-4d1f-81e0-5bf12b280fc2 None None] MFA Rules not processed for user `7721fd95d802434d905b50d23eaad9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:33.469605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ac11fc6-e2f9-4d1f-81e0-5bf12b280fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.475302 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1115/2230] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:33.476241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.477680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.489447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a80bb2-4f93-414f-bd10-8a4ed239fa3d service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.503874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.504816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.505146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.505885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.517792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.519322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.532349 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1116/2231] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:33.554408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1116/2232] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:33.559243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.572392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaeeed-26c8-490e-a2b9-c3f336c6bd72 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.577356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.578008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.578293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.578568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.579204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.581194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.615137 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1117/2233] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:33 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:33.620171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac11fc6-e2f9-4d1f-81e0-5bf12b280fc2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.635407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac11fc6-e2f9-4d1f-81e0-5bf12b280fc2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.699285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.699905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.700186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.700459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.701091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.703183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.730591 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1117/2234] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:33 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:33.734660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.747471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005cbb2-6718-49e8-af10-05de1cb6e09e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.754349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.757022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.757540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.759360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.759748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.763243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.781729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1118/2235] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:33.784531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.798409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585063e-2460-49df-b22f-91112bddffe3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.805758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.806453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.806771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.807259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] PATH_INFO: `/v3/projects/5abd6c399336495e87515a95e895282e/users/3e95489068c843aaad1a10314d58520a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.825102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abd6c399336495e87515a95e895282e, user_id=3e95489068c843aaad1a10314d58520a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.827863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:33.844227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1118/2236] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:33 2022] PUT /identity/v3/projects/5abd6c399336495e87515a95e895282e/users/3e95489068c843aaad1a10314d58520a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:33.849598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.866739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a490c-d4a0-4d9e-8e1d-a999a3d48156 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:33.886698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:33.958615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:33.962990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:33.981983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:33.982877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:33.983492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:33.984212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:33.985323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:33.987531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:34.017034 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1119/2237] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:34.018172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:34.041582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ff2be-75ed-4f83-b262-a5882ea94ca4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:34.046889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c58734-e5c9-40fa-8c38-31b4e878f7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:34.047393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58734-e5c9-40fa-8c38-31b4e878f7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:34.047684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58734-e5c9-40fa-8c38-31b4e878f7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:34.047962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58734-e5c9-40fa-8c38-31b4e878f7cd admin admin] PATH_INFO: `/v3/projects/5abd6c399336495e87515a95e895282e/users/3e95489068c843aaad1a10314d58520a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:34.050080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c58734-e5c9-40fa-8c38-31b4e878f7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abd6c399336495e87515a95e895282e, user_id=3e95489068c843aaad1a10314d58520a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:34.052776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c58734-e5c9-40fa-8c38-31b4e878f7cd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:34.061750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:34.075886 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1120/2238] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:34 2022] PUT /identity/v3/projects/5abd6c399336495e87515a95e895282e/users/3e95489068c843aaad1a10314d58520a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:34.079265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222bb66-7af7-4312-955a-8d7b8db215d2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:34.081326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222bb66-7af7-4312-955a-8d7b8db215d2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:34.081326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222bb66-7af7-4312-955a-8d7b8db215d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:34.105223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dda3b-a5ac-4387-b0a8-89d9d9d8035a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:34.127605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2222bb66-7af7-4312-955a-8d7b8db215d2 None None] MFA Rules not processed for user `3e95489068c843aaad1a10314d58520a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:34.129347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2222bb66-7af7-4312-955a-8d7b8db215d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:34.235048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:34.236352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:34.236592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:34.236873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] PATH_INFO: `/v3/projects/9df7d4075cce40afb8851d0d51cd57ea` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:34.238063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df7d4075cce40afb8851d0d51cd57ea)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:34.240063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:34.265849 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1121/2239] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:34.270060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:34.272030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:34.272547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:34.309994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df7d4075cce40afb8851d0d51cd57ea', 'request_id': 'req-ab2189e3-faa7-4dd8-ba63-77d63f613c76'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.318223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df7d4075cce40afb8851d0d51cd57ea', 'request_id': 'req-ab2189e3-faa7-4dd8-ba63-77d63f613c76'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.321274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] MFA Rules not processed for user `7721fd95d802434d905b50d23eaad9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:34.322713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:34.327737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df7d4075cce40afb8851d0d51cd57ea', 'request_id': 'req-ab2189e3-faa7-4dd8-ba63-77d63f613c76'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.365076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] The token cache is being invalidate because project 9df7d4075cce40afb8851d0d51cd57ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:34.365365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.366976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.372235 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1119/2240] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:33 2022] DELETE /identity/v3/projects/9df7d4075cce40afb8851d0d51cd57ea => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:34.373605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:34.375029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:34.454269 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1122/2241] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:34.469197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:34.472967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:34.489576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2189e3-faa7-4dd8-ba63-77d63f613c76 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:34.502212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:34.504352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:34.504779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:34.508177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] PATH_INFO: `/v3/users/68603c914f9d4748869d95be13c74aac` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:34.513455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68603c914f9d4748869d95be13c74aac)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:34.516034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:34.550417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:34.556294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:34.613487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68603c914f9d4748869d95be13c74aac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.631435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68603c914f9d4748869d95be13c74aac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.635972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fb962-5d3f-4551-902b-1eb4af15a7ad None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:34.639466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68603c914f9d4748869d95be13c74aac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.650136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68603c914f9d4748869d95be13c74aac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.651878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-501240a3-3375-4491-8cdd-1c533907ef31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68603c914f9d4748869d95be13c74aac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:34.652596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:34.653323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:34.653704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:34.654104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:34.661206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:34.671968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1120/2242] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:34 2022] DELETE /identity/v3/users/68603c914f9d4748869d95be13c74aac => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:34.685034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:34.687572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:34.699801 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1123/2243] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:35.254603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5fd7-f08a-44ff-8404-cdcfe3c19698 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:35.254892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5fd7-f08a-44ff-8404-cdcfe3c19698 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:35.255173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5fd7-f08a-44ff-8404-cdcfe3c19698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:35.293431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-491c5fd7-f08a-44ff-8404-cdcfe3c19698 None None] MFA Rules not processed for user `a020e015186b4c32950c2d4edd26374e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:35.294707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-491c5fd7-f08a-44ff-8404-cdcfe3c19698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:35.432664 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1121/2244] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:36.687484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:36.689049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:36.722269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a53c7e-6311-46f9-9c83-7b4b7de6b5c0 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:36.949092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:36.950365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:36.951072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:36.951654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] PATH_INFO: `/v3/projects/539e7c8210ac417b99d003f6489c160a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:36.954154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=539e7c8210ac417b99d003f6489c160a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:36.957069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:37.025110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539e7c8210ac417b99d003f6489c160a', 'request_id': 'req-0af62af3-d0ea-4477-b224-a835c757c5d7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:37.034422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539e7c8210ac417b99d003f6489c160a', 'request_id': 'req-0af62af3-d0ea-4477-b224-a835c757c5d7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:37.089900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539e7c8210ac417b99d003f6489c160a', 'request_id': 'req-0af62af3-d0ea-4477-b224-a835c757c5d7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:37.116892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] The token cache is being invalidate because project 539e7c8210ac417b99d003f6489c160a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:37.117429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:37.119524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0af62af3-d0ea-4477-b224-a835c757c5d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:37.129399 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1124/2245] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:36 2022] DELETE /identity/v3/projects/539e7c8210ac417b99d003f6489c160a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:37.131624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:37.132125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:37.132411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:37.197180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:37.198722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:37.347264 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1122/2246] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:37.355583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619bc04-11e5-43fe-aae7-6bb4f3fff020 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:37.355864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619bc04-11e5-43fe-aae7-6bb4f3fff020 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:37.356752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619bc04-11e5-43fe-aae7-6bb4f3fff020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:37.383517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:37.385282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:37.399252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a619bc04-11e5-43fe-aae7-6bb4f3fff020 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:37.400664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a619bc04-11e5-43fe-aae7-6bb4f3fff020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:37.417978 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1125/2247] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:37.422697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619bc04-11e5-43fe-aae7-6bb4f3fff020 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:37.466091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619bc04-11e5-43fe-aae7-6bb4f3fff020 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:37.472793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:37.473333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:37.473601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:37.473924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:37.474807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:37.475121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:37.478211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:37.495651 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1126/2248] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:37.504081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:37.519004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:37.534575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c608f-ffd4-4a34-a208-20c2f586ce95 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:37.555286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:37.555905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:37.556186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:37.559400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:37.560073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:37.572069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:37.635372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e8eaa-3f5f-4bb3-b5d1-f4a1c15dda7b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:37.658250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:37.659036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:37.659324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:37.660343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:37.666290 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1127/2249] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:37 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:37.669547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:37.683396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:37.689337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:37.700788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:37.811130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:37.813034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:37.854409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1123/2250] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:37.855609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:37.889653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56988f7-0ea5-4dbb-9930-dded0a884f6e service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:37.905939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:37.906676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:37.906955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:37.907226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:37.907875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:37.911768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d0f88-163b-4dcb-bd2c-318a97f204aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:37.918002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:37.919598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:37.920480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:37.920873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:37.921259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] PATH_INFO: `/v3/users/5aab292e925940568e9e5db8b6910f45` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:37.922783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aab292e925940568e9e5db8b6910f45)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:37.925151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:37.993400 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1124/2251] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:37 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:37.994549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:38.019315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aab292e925940568e9e5db8b6910f45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:38.026333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520702-f02a-421e-8150-b6f67977b602 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:38.033486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aab292e925940568e9e5db8b6910f45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:38.036096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aab292e925940568e9e5db8b6910f45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:38.042306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:38.043467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:38.043467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:38.043467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:38.046452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:38.055479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:38.055651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aab292e925940568e9e5db8b6910f45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:38.055908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d78e14ac-0c87-42bb-8bea-7646f3d24ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aab292e925940568e9e5db8b6910f45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:38.080192 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1125/2252] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:38.106738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:38.111729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:38.116850 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1128/2253] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:37 2022] DELETE /identity/v3/users/5aab292e925940568e9e5db8b6910f45 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:38.123387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66ecd-7659-42dc-b191-15127731032c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:38.123871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66ecd-7659-42dc-b191-15127731032c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:38.124354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66ecd-7659-42dc-b191-15127731032c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:38.147697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfa8d1-8f0a-4b73-8d55-ec55e1cef49b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:38.179093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-67f66ecd-7659-42dc-b191-15127731032c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:38.180313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-67f66ecd-7659-42dc-b191-15127731032c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:38.317022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:38.318098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:38.318930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:38.319399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:38.320186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:38.322037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:38.331915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1129/2254] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:38.342863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66ecd-7659-42dc-b191-15127731032c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:38.344289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-67f66ecd-7659-42dc-b191-15127731032c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:38.347237 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1126/2255] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:38.348483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:38.365050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518c58c-b928-496e-b693-85794d158961 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:38.371923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfbdcf-0b85-4da0-8cfb-8ba3e41b31f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:38.372825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfbdcf-0b85-4da0-8cfb-8ba3e41b31f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:38.373188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfbdcf-0b85-4da0-8cfb-8ba3e41b31f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:38.373683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfbdcf-0b85-4da0-8cfb-8ba3e41b31f5 admin admin] PATH_INFO: `/v3/projects/63eb4f57847e4e799b89697d34253337/users/0817d9268715460fbbd6365dff685a23/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:38.409423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:38.427354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfbdcf-0b85-4da0-8cfb-8ba3e41b31f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63eb4f57847e4e799b89697d34253337, user_id=0817d9268715460fbbd6365dff685a23, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:38.431423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfbdcf-0b85-4da0-8cfb-8ba3e41b31f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:38.462067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1127/2256] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:38 2022] PUT /identity/v3/projects/63eb4f57847e4e799b89697d34253337/users/0817d9268715460fbbd6365dff685a23/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:38.463629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e972ed-1a6a-4f39-88f2-71b94d1ac277 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:38.464789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e972ed-1a6a-4f39-88f2-71b94d1ac277 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:38.468366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e972ed-1a6a-4f39-88f2-71b94d1ac277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:38.478237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66ecd-7659-42dc-b191-15127731032c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:38.510047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a6e972ed-1a6a-4f39-88f2-71b94d1ac277 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:38.513844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a6e972ed-1a6a-4f39-88f2-71b94d1ac277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:38.539347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:38.625433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:38.628007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:38.628459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:38.629463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:38.701247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1128/2257] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:38.722022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e972ed-1a6a-4f39-88f2-71b94d1ac277 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:38.744135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e972ed-1a6a-4f39-88f2-71b94d1ac277 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:38.744663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:38.747412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:38.749606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:38.750276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:38.750558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:38.750833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:38.779306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1130/2258] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:38.780678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:38.782402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:38.823696 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1129/2259] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:39.251979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:39.259160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:39.299213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b336aad-b3ee-4182-9118-7038f72db194 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:39.330574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:39.332563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:39.348868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:39.349176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] PATH_INFO: `/v3/projects/35ef1b5d5dba4e3c93d8bb622ff79c5e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:39.354585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ef1b5d5dba4e3c93d8bb622ff79c5e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:39.356768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:39.489404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ef1b5d5dba4e3c93d8bb622ff79c5e', 'request_id': 'req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.511068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ef1b5d5dba4e3c93d8bb622ff79c5e', 'request_id': 'req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.535279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ef1b5d5dba4e3c93d8bb622ff79c5e', 'request_id': 'req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.585088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] The token cache is being invalidate because project 35ef1b5d5dba4e3c93d8bb622ff79c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:39.586181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.589559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.591592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1131/2260] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:39 2022] DELETE /identity/v3/projects/35ef1b5d5dba4e3c93d8bb622ff79c5e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:39.596052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:39.612777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1de8c-5fcd-48a5-805a-5407a4c480b6 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:39.758882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:39.759656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:39.760007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:39.760353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] PATH_INFO: `/v3/users/35c7c4e102e94a5c92d52b94099d50b5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:39.761616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c7c4e102e94a5c92d52b94099d50b5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:39.763925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:39.873372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c7c4e102e94a5c92d52b94099d50b5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.897757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c7c4e102e94a5c92d52b94099d50b5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.907629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c7c4e102e94a5c92d52b94099d50b5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.917461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c7c4e102e94a5c92d52b94099d50b5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.918270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c7c4e102e94a5c92d52b94099d50b5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:39.925302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1130/2261] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:49:39 2022] DELETE /identity/v3/users/35c7c4e102e94a5c92d52b94099d50b5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:40.907177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:40.909494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:40.954405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ca8cc-7acc-4c20-8960-69ee66f6a3ba admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:40.970889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:41.067193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:41.067895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.068191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.068978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] PATH_INFO: `/v3/projects/1b12288ef70e4b38b128866c7f2efd47` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.079456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b12288ef70e4b38b128866c7f2efd47)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:41.079456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:41.093065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:41.094721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:41.150973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b12288ef70e4b38b128866c7f2efd47', 'request_id': 'req-1d4426be-aa2a-405c-9a01-6043b64093e2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:41.159470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b12288ef70e4b38b128866c7f2efd47', 'request_id': 'req-1d4426be-aa2a-405c-9a01-6043b64093e2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:41.175412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b12288ef70e4b38b128866c7f2efd47', 'request_id': 'req-1d4426be-aa2a-405c-9a01-6043b64093e2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:41.189196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:41.203184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] The token cache is being invalidate because project 1b12288ef70e4b38b128866c7f2efd47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:49:41.203726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:41.207451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d4426be-aa2a-405c-9a01-6043b64093e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:49:41.212235 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1132/2262] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:49:40 2022] DELETE /identity/v3/projects/1b12288ef70e4b38b128866c7f2efd47 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:41.217560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07a2f-9344-43e0-a82c-fdd36a01418e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.218555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07a2f-9344-43e0-a82c-fdd36a01418e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.218921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b07a2f-9344-43e0-a82c-fdd36a01418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.277367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a7b07a2f-9344-43e0-a82c-fdd36a01418e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:41.278670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7b07a2f-9344-43e0-a82c-fdd36a01418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:41.349540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efbf64-efd8-4efc-9567-19952c97c236 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:41.408115 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1133/2263] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:41.418191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307c82-0290-42fe-8849-0318c4eda7b8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.419215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307c82-0290-42fe-8849-0318c4eda7b8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.420250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307c82-0290-42fe-8849-0318c4eda7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.505712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d2307c82-0290-42fe-8849-0318c4eda7b8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:41.509489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d2307c82-0290-42fe-8849-0318c4eda7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:41.527945 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1134/2264] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:41.535774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307c82-0290-42fe-8849-0318c4eda7b8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:41.557315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307c82-0290-42fe-8849-0318c4eda7b8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:41.567618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:41.569537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.569930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.570212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.576600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:49:41.577090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:41.577413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:41.578070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.578891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.579188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.579479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:41.581869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:41.610397 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1135/2265] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:49:41.612163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:41.634749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e10b1-fcf0-4559-a116-97fb9bbd5aa0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:41.647074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:41.650001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.650313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.650595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.651267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:41.653211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:41.720967 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1136/2266] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:41 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:41.725605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:41.738651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:41.761206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:41.763860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:41.777242 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1131/2267] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:41 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:49:41.824253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aac6eb-a43e-4d6a-ab25-bbc02d99afa1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:41.834741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:41.835377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.835813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.836126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.836808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:41.838839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:41.889914 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1137/2268] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:41 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:41.894307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:41.908604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b3cac-2483-4df4-87eb-d6d7fdede0e5 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:41.921200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:41.922010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:41.922398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:41.922958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:41.923753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:41.925302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:41.970942 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1132/2269] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:41.975763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:41.992538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9f96c-d504-4895-8224-ed4a5a198165 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:42.013739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebe3a5-4bc0-447b-b7fc-f2ad6bbdd730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:42.015420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe3a5-4bc0-447b-b7fc-f2ad6bbdd730 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:42.015816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe3a5-4bc0-447b-b7fc-f2ad6bbdd730 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:42.016276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe3a5-4bc0-447b-b7fc-f2ad6bbdd730 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:42.017893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebe3a5-4bc0-447b-b7fc-f2ad6bbdd730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:42.020410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebe3a5-4bc0-447b-b7fc-f2ad6bbdd730 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:42.040013 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1138/2270] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:42.041796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:42.056332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676197a-b17a-40ff-8ab7-0218c2d55fdb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:42.095690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:42.096474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:42.096641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:42.096919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] PATH_INFO: `/v3/projects/5580128bff364c858eafedcc84dad425/users/cbabc9a296d14628b649da7d81bc8766/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:42.146868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5580128bff364c858eafedcc84dad425, user_id=cbabc9a296d14628b649da7d81bc8766, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:42.149889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:42.182586 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1133/2271] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:42 2022] PUT /identity/v3/projects/5580128bff364c858eafedcc84dad425/users/cbabc9a296d14628b649da7d81bc8766/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:42.196754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea25f2-054a-4e8a-b869-eec5578e9c16 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:42.197077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea25f2-054a-4e8a-b869-eec5578e9c16 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:42.197275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea25f2-054a-4e8a-b869-eec5578e9c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:42.263199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a3ea25f2-054a-4e8a-b869-eec5578e9c16 None None] MFA Rules not processed for user `cbabc9a296d14628b649da7d81bc8766`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:42.266186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a3ea25f2-054a-4e8a-b869-eec5578e9c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:42.407196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:42.421499 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1139/2272] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:42.436986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:42.534166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d4ce0-b3cb-443f-81c1-26e115da75b0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:42.548687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:42.549325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:42.549612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:42.549916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:42.550840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:42.555094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:42.634022 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1134/2273] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:42 2022] POST /identity/v3/projects => generated 375 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:49:42.638964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea25f2-054a-4e8a-b869-eec5578e9c16 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:42.665671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea25f2-054a-4e8a-b869-eec5578e9c16 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:42.920590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:42.921981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:42.922588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:42.922868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:42.924016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:42.926081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:42.958221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:42.965954 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1140/2274] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:42 2022] POST /identity/v3/users => generated 294 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:42.970195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:42.997700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09f746-3af1-402e-ad0b-a71b4be63d8d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:43.004932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b5b55-9750-46e0-a1f3-e10d182eb44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:43.005752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5b55-9750-46e0-a1f3-e10d182eb44d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:43.006308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5b55-9750-46e0-a1f3-e10d182eb44d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:43.006881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5b55-9750-46e0-a1f3-e10d182eb44d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:43.007929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5b55-9750-46e0-a1f3-e10d182eb44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:43.012661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5b55-9750-46e0-a1f3-e10d182eb44d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:43.039400 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1135/2275] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:43.043260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:43.062048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07866b88-1741-4d1e-90c3-ef47473e2df3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:43.086012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:43.087215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:43.087505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:43.087781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] PATH_INFO: `/v3/projects/3bc41ed54a5745d8b6f127e3ac482da1/users/61323f1fb63a438fa8d7dbfe6caeb8ea/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:43.159603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc41ed54a5745d8b6f127e3ac482da1, user_id=61323f1fb63a438fa8d7dbfe6caeb8ea, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:43.170687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:43.171510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd710e-0ff4-4a0d-a0be-65e1c96056ec None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:43.171967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd710e-0ff4-4a0d-a0be-65e1c96056ec None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:43.172048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd710e-0ff4-4a0d-a0be-65e1c96056ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:43.230605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6cdd710e-0ff4-4a0d-a0be-65e1c96056ec None None] MFA Rules not processed for user `365336e9f59e4b65ae0a05554776b0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:43.232268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6cdd710e-0ff4-4a0d-a0be-65e1c96056ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:43.255261 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1141/2276] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:43 2022] PUT /identity/v3/projects/3bc41ed54a5745d8b6f127e3ac482da1/users/61323f1fb63a438fa8d7dbfe6caeb8ea/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:43.260099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:43.436860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c3728-6955-44d6-9c6b-7e2a9b06c75f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:43.441055 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1136/2277] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:49:43.464651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724c118-ef8d-417c-81af-035eb9066d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:43.465777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724c118-ef8d-417c-81af-035eb9066d08 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:43.466516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724c118-ef8d-417c-81af-035eb9066d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:43.466995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724c118-ef8d-417c-81af-035eb9066d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:43.467866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724c118-ef8d-417c-81af-035eb9066d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:43.475708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724c118-ef8d-417c-81af-035eb9066d08 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:43.503461 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1142/2278] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:43.508341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd710e-0ff4-4a0d-a0be-65e1c96056ec None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:43.522997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd710e-0ff4-4a0d-a0be-65e1c96056ec None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:43.542530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb04f61-4afc-4781-ba07-311e017182a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:43.543354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04f61-4afc-4781-ba07-311e017182a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:43.543736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04f61-4afc-4781-ba07-311e017182a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:43.544109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04f61-4afc-4781-ba07-311e017182a6 admin admin] PATH_INFO: `/v3/projects/3bc41ed54a5745d8b6f127e3ac482da1/users/61323f1fb63a438fa8d7dbfe6caeb8ea/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:43.548004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb04f61-4afc-4781-ba07-311e017182a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc41ed54a5745d8b6f127e3ac482da1, user_id=61323f1fb63a438fa8d7dbfe6caeb8ea, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:43.555913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb04f61-4afc-4781-ba07-311e017182a6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:43.590249 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1137/2279] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:43 2022] PUT /identity/v3/projects/3bc41ed54a5745d8b6f127e3ac482da1/users/61323f1fb63a438fa8d7dbfe6caeb8ea/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:43.603447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:43.603795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:43.604019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:43.692520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] MFA Rules not processed for user `61323f1fb63a438fa8d7dbfe6caeb8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:43.695844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:43.915051 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1143/2280] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:43.924824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be579eb-6f84-4f6d-8aa0-f08f10962f4f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:43.925300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be579eb-6f84-4f6d-8aa0-f08f10962f4f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:43.925743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be579eb-6f84-4f6d-8aa0-f08f10962f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:44.015081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5be579eb-6f84-4f6d-8aa0-f08f10962f4f None None] MFA Rules not processed for user `cbabc9a296d14628b649da7d81bc8766`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:44.019625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5be579eb-6f84-4f6d-8aa0-f08f10962f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:44.267584 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1138/2281] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:45.424625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:45.426115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:45.612049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:45.646273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb9b55-b6ca-41f2-a87b-e218958b7ff3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:45.654557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebd9ee-446d-4154-9957-03153ee4ffa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:45.655331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd9ee-446d-4154-9957-03153ee4ffa6 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:45.655707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd9ee-446d-4154-9957-03153ee4ffa6 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:45.656076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebd9ee-446d-4154-9957-03153ee4ffa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:45.668853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebd9ee-446d-4154-9957-03153ee4ffa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:45.674239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebd9ee-446d-4154-9957-03153ee4ffa6 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:45.696828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1144/2282] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:47.904354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:47.905229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:47.905712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:47.963760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] MFA Rules not processed for user `cbabc9a296d14628b649da7d81bc8766`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:47.968048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:48.023152 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1139/2283] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:50.931372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d35a2-84b2-4ab1-a253-5b9ef2c1faba None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:50.934857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d35a2-84b2-4ab1-a253-5b9ef2c1faba None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:50.936051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d35a2-84b2-4ab1-a253-5b9ef2c1faba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:51.007267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2a8d35a2-84b2-4ab1-a253-5b9ef2c1faba None None] MFA Rules not processed for user `f74a2c8cd22c454fa8fdd7c2e1ef9230`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:51.015308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a8d35a2-84b2-4ab1-a253-5b9ef2c1faba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:51.199637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:51.202015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:51.210651 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1145/2284] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:51.229596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:51.289718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0014f-7447-43af-91ac-c72eade0f55a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:51.295216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:51.296089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:51.296732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:51.297203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:51.298208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:51.300324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:51.351808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d35a2-84b2-4ab1-a253-5b9ef2c1faba None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:51.362359 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1140/2285] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:49:51 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:51.368952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d35a2-84b2-4ab1-a253-5b9ef2c1faba None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:51.530901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:51.531642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:51.532028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:51.533062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:51.533931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:51.536747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:51.585540 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1146/2286] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:49:51 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:49:51.589603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:51.605627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0d2e1-8a07-4b85-aa40-d6cdb2a439bd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:51.612944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:51.614044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:51.616477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:51.616971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:51.617886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:51.619833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:51.645378 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1141/2287] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:51.645916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:51.661083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860c248-6079-4527-bcd3-6903eac509b6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:51.681042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:51.681737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:51.682036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:51.682318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] PATH_INFO: `/v3/projects/f74799f9b954449490f73fc5bffef0a1/users/41e511ffe08c4d89b852ed8d1f228118/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:51.707107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74799f9b954449490f73fc5bffef0a1, user_id=41e511ffe08c4d89b852ed8d1f228118, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:51.710810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:51.736708 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1147/2288] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:51 2022] PUT /identity/v3/projects/f74799f9b954449490f73fc5bffef0a1/users/41e511ffe08c4d89b852ed8d1f228118/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:49:51.738643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:51.759432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b08b95-2002-45db-98d4-9bcd9aecab00 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:51.901210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e76df-f13d-46fa-b770-eb4804a433ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:51.902217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e76df-f13d-46fa-b770-eb4804a433ed admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:51.902790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e76df-f13d-46fa-b770-eb4804a433ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:51.903222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e76df-f13d-46fa-b770-eb4804a433ed admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:51.907153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e76df-f13d-46fa-b770-eb4804a433ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:51.911641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e76df-f13d-46fa-b770-eb4804a433ed admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:51.946983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1142/2289] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:49:51.950828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:51.964763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8f192-02a3-4441-8a63-879308637524 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:51.982281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:51.983151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:51.983608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:51.984037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] PATH_INFO: `/v3/projects/f74799f9b954449490f73fc5bffef0a1/users/41e511ffe08c4d89b852ed8d1f228118/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:51.988895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74799f9b954449490f73fc5bffef0a1, user_id=41e511ffe08c4d89b852ed8d1f228118, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:51.992904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:52.021543 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1148/2290] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:49:51 2022] PUT /identity/v3/projects/f74799f9b954449490f73fc5bffef0a1/users/41e511ffe08c4d89b852ed8d1f228118/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:49:52.857376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e72a5-55cd-48ab-a734-7eacc8af947b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:52.863562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e72a5-55cd-48ab-a734-7eacc8af947b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:52.864068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e72a5-55cd-48ab-a734-7eacc8af947b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:52.953884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c78e72a5-55cd-48ab-a734-7eacc8af947b None None] MFA Rules not processed for user `0817d9268715460fbbd6365dff685a23`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:52.957305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c78e72a5-55cd-48ab-a734-7eacc8af947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:53.099659 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1143/2291] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:49:53.111660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:49:53.113476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:53.192243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:53.326603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b54a35-fa48-4162-ab26-0aad362aff22 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:49:53.337268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9a0bf-9fbc-43a5-ba3a-b76e183e0274 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:49:53.339135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a0bf-9fbc-43a5-ba3a-b76e183e0274 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:53.339573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a0bf-9fbc-43a5-ba3a-b76e183e0274 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:53.340043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a0bf-9fbc-43a5-ba3a-b76e183e0274 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:53.357601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9a0bf-9fbc-43a5-ba3a-b76e183e0274 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:49:53.361464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9a0bf-9fbc-43a5-ba3a-b76e183e0274 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:49:53.392987 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1149/2292] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:49:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:49:58.599024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be232-895d-4f41-9101-5865414e9c92 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:49:58.600161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be232-895d-4f41-9101-5865414e9c92 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:49:58.600266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be232-895d-4f41-9101-5865414e9c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:49:58.630048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-790be232-895d-4f41-9101-5865414e9c92 None None] MFA Rules not processed for user `dd431fc2d9834bea8df807edb91771ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:49:58.631332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-790be232-895d-4f41-9101-5865414e9c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:49:58.689060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:49:58.779516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1144/2293] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:49:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:03.371583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=23 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:05.860894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39bfa-a3dd-4753-a7ca-186f56f58ef8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:05.861968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39bfa-a3dd-4753-a7ca-186f56f58ef8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:05.862419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39bfa-a3dd-4753-a7ca-186f56f58ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:05.923982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bba39bfa-a3dd-4753-a7ca-186f56f58ef8 None None] MFA Rules not processed for user `41e511ffe08c4d89b852ed8d1f228118`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:05.925761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bba39bfa-a3dd-4753-a7ca-186f56f58ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:06.047569 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1150/2294] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:50:06.058733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86ed9-8d6d-43c1-9c8e-15c593e6d3ba None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:06.060352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86ed9-8d6d-43c1-9c8e-15c593e6d3ba None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:06.060789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86ed9-8d6d-43c1-9c8e-15c593e6d3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:06.084545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c1a86ed9-8d6d-43c1-9c8e-15c593e6d3ba None None] MFA Rules not processed for user `f74a2c8cd22c454fa8fdd7c2e1ef9230`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:06.086155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c1a86ed9-8d6d-43c1-9c8e-15c593e6d3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:06.206275 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1145/2295] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:06.733288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba39bfa-a3dd-4753-a7ca-186f56f58ef8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:06.754087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba39bfa-a3dd-4753-a7ca-186f56f58ef8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:06.771204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:06.772155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:06.772447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:06.772727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:06.794264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:06.795960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:06.820572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1151/2296] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:12.676115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:12.676636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:12.676741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:12.697804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:12.699475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:12.795905 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1146/2297] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:12.812215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:12.826749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec931be-c57a-464d-b8f0-617654ac7a90 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:12.833697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:12.834776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:12.835226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:12.836251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:12.838241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:12.852862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:12.854307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:12.950188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:12.951950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:12.998602 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1152/2298] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:12.999964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbe350-4074-47ea-9bd6-4981c43ea28c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:13.007693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36780c08-ef8e-459d-95a7-44fced4ebf57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:13.008316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36780c08-ef8e-459d-95a7-44fced4ebf57 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:13.008927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36780c08-ef8e-459d-95a7-44fced4ebf57 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:13.009643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36780c08-ef8e-459d-95a7-44fced4ebf57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:13.024925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36780c08-ef8e-459d-95a7-44fced4ebf57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:13.026815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36780c08-ef8e-459d-95a7-44fced4ebf57 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:13.038151 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1147/2299] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:50:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:50:13.963853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:13.966906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:14.165067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135b2c5-bf3e-4997-8e83-df611703d21a service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:14.170600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:14.172109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.172866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.173309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] PATH_INFO: `/v3/users/365336e9f59e4b65ae0a05554776b0ff` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.174722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365336e9f59e4b65ae0a05554776b0ff)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:14.178045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:14.189617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:14.283612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365336e9f59e4b65ae0a05554776b0ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:14.310659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365336e9f59e4b65ae0a05554776b0ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:14.311582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365336e9f59e4b65ae0a05554776b0ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:14.346936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365336e9f59e4b65ae0a05554776b0ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:14.347610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365336e9f59e4b65ae0a05554776b0ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:14.387650 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1153/2300] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:13 2022] DELETE /identity/v3/users/365336e9f59e4b65ae0a05554776b0ff => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:14.388885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250c71d-eecd-4273-9440-8176f40c3a7c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.389489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250c71d-eecd-4273-9440-8176f40c3a7c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.390088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250c71d-eecd-4273-9440-8176f40c3a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.406231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:14.429727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9250c71d-eecd-4273-9440-8176f40c3a7c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:14.434314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9250c71d-eecd-4273-9440-8176f40c3a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:14.637563 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1148/2301] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:14.663438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:14.664931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:14.764728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7da43-e090-439d-93dc-57715dba6c22 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:14.795982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:14.796849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:14.797138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:14.797417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:14.818337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:14.819988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:14.854692 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1154/2302] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:15.602503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8c5c-7525-421c-8db4-176a181dc138 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:15.603085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8c5c-7525-421c-8db4-176a181dc138 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:15.603085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8c5c-7525-421c-8db4-176a181dc138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:15.650621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a01c8c5c-7525-421c-8db4-176a181dc138 None None] MFA Rules not processed for user `cbabc9a296d14628b649da7d81bc8766`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:15.654188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a01c8c5c-7525-421c-8db4-176a181dc138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:15.752328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:15.754838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:15.806180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea6f7a-f1a0-4c70-9cd0-6c1f44943d80 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:15.820639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:15.821257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:15.821549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:15.821849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] PATH_INFO: `/v3/projects/5cc9f18c8fd44fca98cda3832b60b26d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:15.824226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc9f18c8fd44fca98cda3832b60b26d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:15.830142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:15.871972 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1149/2303] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:15.935472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc9f18c8fd44fca98cda3832b60b26d', 'request_id': 'req-c426a473-88ac-47fe-bcf0-9e2753a01b25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:15.952167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc9f18c8fd44fca98cda3832b60b26d', 'request_id': 'req-c426a473-88ac-47fe-bcf0-9e2753a01b25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:15.978312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc9f18c8fd44fca98cda3832b60b26d', 'request_id': 'req-c426a473-88ac-47fe-bcf0-9e2753a01b25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:15.993762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] The token cache is being invalidate because project 5cc9f18c8fd44fca98cda3832b60b26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:15.994096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:15.994886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:15.997671 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1155/2304] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:15 2022] DELETE /identity/v3/projects/5cc9f18c8fd44fca98cda3832b60b26d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:16.001614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8c5c-7525-421c-8db4-176a181dc138 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:16.024336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8c5c-7525-421c-8db4-176a181dc138 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:16.146556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:16.150650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:16.151389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:16.152522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] PATH_INFO: `/v3/users/0f668ef345144fc695d1fc0fffe1cb3e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:16.159870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f668ef345144fc695d1fc0fffe1cb3e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:16.179029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:16.271356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:16.273121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:16.310623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f668ef345144fc695d1fc0fffe1cb3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.315458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426a473-88ac-47fe-bcf0-9e2753a01b25 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:16.320777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:16.333875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:16.339244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:16.339542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:16.339811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] PATH_INFO: `/v3/users/0817d9268715460fbbd6365dff685a23` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:16.342438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f668ef345144fc695d1fc0fffe1cb3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.351670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0817d9268715460fbbd6365dff685a23)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:16.353248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:16.364161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f668ef345144fc695d1fc0fffe1cb3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.370435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:16.382878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f668ef345144fc695d1fc0fffe1cb3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.383659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f668ef345144fc695d1fc0fffe1cb3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.410586 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1150/2305] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:16 2022] DELETE /identity/v3/users/0f668ef345144fc695d1fc0fffe1cb3e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:50:16.524522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0817d9268715460fbbd6365dff685a23'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.537386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0817d9268715460fbbd6365dff685a23'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.541183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0817d9268715460fbbd6365dff685a23'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.554103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0817d9268715460fbbd6365dff685a23'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.554992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0817d9268715460fbbd6365dff685a23'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:16.595750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1156/2306] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:16 2022] DELETE /identity/v3/users/0817d9268715460fbbd6365dff685a23 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:17.045407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:17.047687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:17.200586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:17.202271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:17.204703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4432d-58c2-49c0-bc47-5aa660b75780 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:17.434191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a1b75-2774-4add-861a-6028ee80c6cc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:17.437036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:17.437674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:17.437990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:17.438284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] PATH_INFO: `/v3/projects/2600c77488cb4c7882998dc76799b9fc` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:17.440596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2600c77488cb4c7882998dc76799b9fc)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:17.443149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:17.453585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:17.454364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:17.454469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:17.454681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] PATH_INFO: `/v3/projects/63eb4f57847e4e799b89697d34253337` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:17.460974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63eb4f57847e4e799b89697d34253337)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:17.462969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:17.609963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2600c77488cb4c7882998dc76799b9fc', 'request_id': 'req-5637c692-787e-40ec-bae8-8c31ff2e6a1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.618770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2600c77488cb4c7882998dc76799b9fc', 'request_id': 'req-5637c692-787e-40ec-bae8-8c31ff2e6a1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.627337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2600c77488cb4c7882998dc76799b9fc', 'request_id': 'req-5637c692-787e-40ec-bae8-8c31ff2e6a1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.645238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63eb4f57847e4e799b89697d34253337', 'request_id': 'req-264bca21-0921-4460-a03e-41c39ad3f0e3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.659749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] The token cache is being invalidate because project 2600c77488cb4c7882998dc76799b9fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:17.660087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.661101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5637c692-787e-40ec-bae8-8c31ff2e6a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.663119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63eb4f57847e4e799b89697d34253337', 'request_id': 'req-264bca21-0921-4460-a03e-41c39ad3f0e3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.667211 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1151/2307] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:17 2022] DELETE /identity/v3/projects/2600c77488cb4c7882998dc76799b9fc => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:50:17.673851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c2632-c2cb-4b30-8a05-d13cafc5a274 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:17.674201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c2632-c2cb-4b30-8a05-d13cafc5a274 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:17.674484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c2632-c2cb-4b30-8a05-d13cafc5a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:17.678699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63eb4f57847e4e799b89697d34253337', 'request_id': 'req-264bca21-0921-4460-a03e-41c39ad3f0e3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.707158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:17.709797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] The token cache is being invalidate because project 63eb4f57847e4e799b89697d34253337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:17.710538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.713160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264bca21-0921-4460-a03e-41c39ad3f0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:17.719618 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1157/2308] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:17 2022] DELETE /identity/v3/projects/63eb4f57847e4e799b89697d34253337 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:17.726483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b97dd-9646-437d-8165-0eaedc04d8b9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:17.726775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b97dd-9646-437d-8165-0eaedc04d8b9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:17.727060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b97dd-9646-437d-8165-0eaedc04d8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:17.733772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2c8c2632-c2cb-4b30-8a05-d13cafc5a274 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:17.735824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c8c2632-c2cb-4b30-8a05-d13cafc5a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:17.776180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8a8b97dd-9646-437d-8165-0eaedc04d8b9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:17.777583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8a8b97dd-9646-437d-8165-0eaedc04d8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:17.801473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:17.875967 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1152/2309] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:17.887428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fbf6d-4163-4cae-a469-aef1b0935422 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:17.887780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fbf6d-4163-4cae-a469-aef1b0935422 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:17.887990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fbf6d-4163-4cae-a469-aef1b0935422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:17.909620 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1158/2310] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:17.919114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6759-dc00-4929-a17b-7f040de12f85 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:17.919546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6759-dc00-4929-a17b-7f040de12f85 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:17.919925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6759-dc00-4929-a17b-7f040de12f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:17.930995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-844fbf6d-4163-4cae-a469-aef1b0935422 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:17.932252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-844fbf6d-4163-4cae-a469-aef1b0935422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:17.946399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-dcab6759-dc00-4929-a17b-7f040de12f85 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:17.948429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dcab6759-dc00-4929-a17b-7f040de12f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:17.953770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1153/2311] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:17.954663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fbf6d-4163-4cae-a469-aef1b0935422 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:17.982559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fbf6d-4163-4cae-a469-aef1b0935422 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:17.984485 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1159/2312] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:17.991359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab6759-dc00-4929-a17b-7f040de12f85 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:17.995820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:17.996597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:17.997012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:17.997197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:17.998026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:17.998399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.002204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.011678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab6759-dc00-4929-a17b-7f040de12f85 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.028848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.029476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.029760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.030206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.031048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:18.031521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.033324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.037787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1154/2313] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:18.039202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.060108 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1160/2314] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:18.060339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36234554-92f9-4bda-802b-332c43f16787 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.061334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.073450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.073785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a48af-fb11-405c-8660-5efec1ce6dd9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.074258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.074655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.074935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.075683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.078795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.079409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.079697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.079968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.080663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.081505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.083790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.146369 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1161/2315] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:18 2022] POST /identity/v3/projects => generated 401 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:18.151816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.165265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1155/2316] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:18 2022] POST /identity/v3/projects => generated 399 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:18.166062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.319579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b5a3d-2be9-4e22-bbe9-0c68a3af82d4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.370444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af812d-070a-4c1a-983f-73e32392c891 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.399204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.399835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.400122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.400399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.401037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.403001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.451458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1156/2317] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:18 2022] POST /identity/v3/users => generated 300 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:18.453201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.474983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a108183-dd6a-43a0-bb38-b0fc651f473e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.482070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.482796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.483180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.483630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.484402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.487401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.502805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1157/2318] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:18.509813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.524600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b1d8-bba2-4bbc-8a24-a0b72561b254 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.534777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.535518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.535809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.536127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.536831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.539167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.540317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.541173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.541682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.542913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.558089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.560879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.576554 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1158/2319] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:18.581131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.595028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ca683-bcfd-4b68-872f-c1bc4d1bcf41 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.603722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1162/2320] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:18 2022] POST /identity/v3/users => generated 301 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:18.605068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.608640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d027ad-e588-4cf7-b7f7-f588f76ab532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.613643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d027ad-e588-4cf7-b7f7-f588f76ab532 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.615506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d027ad-e588-4cf7-b7f7-f588f76ab532 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.617605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b6104-1e85-4866-9223-895e1f5772e1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.619543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d027ad-e588-4cf7-b7f7-f588f76ab532 admin admin] PATH_INFO: `/v3/projects/046393a40a784a2caaa40331e861ab78/users/aa9e19a592e34a9ab3f4ab6f4d0b6ea8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.622977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.623744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.624038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.624303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.625534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.627196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.630516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:18.646456 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1163/2321] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:18.647779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.668355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abdd06-68fc-46fa-9909-4be2be4ff933 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.678862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d027ad-e588-4cf7-b7f7-f588f76ab532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046393a40a784a2caaa40331e861ab78, user_id=aa9e19a592e34a9ab3f4ab6f4d0b6ea8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.681351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.682104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d027ad-e588-4cf7-b7f7-f588f76ab532 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.706000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.706000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.706454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.711955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.713938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.719673 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1159/2322] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:18 2022] PUT /identity/v3/projects/046393a40a784a2caaa40331e861ab78/users/aa9e19a592e34a9ab3f4ab6f4d0b6ea8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:18.732811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb121bbf-73b1-4a5a-898a-2a2586ba9ce5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.733105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb121bbf-73b1-4a5a-898a-2a2586ba9ce5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.733546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb121bbf-73b1-4a5a-898a-2a2586ba9ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.766818 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1164/2323] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:18.771111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:18.789944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a20ac1-c72e-4365-a3e4-61cca68c6c36 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:18.801671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:18.827062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bb121bbf-73b1-4a5a-898a-2a2586ba9ce5 None None] MFA Rules not processed for user `aa9e19a592e34a9ab3f4ab6f4d0b6ea8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:18.828223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bb121bbf-73b1-4a5a-898a-2a2586ba9ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:18.916584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7254f8-f539-4f4f-8759-01260665b561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:18.917604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7254f8-f539-4f4f-8759-01260665b561 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:18.917916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7254f8-f539-4f4f-8759-01260665b561 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:18.918378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7254f8-f539-4f4f-8759-01260665b561 admin admin] PATH_INFO: `/v3/projects/fbd06bf7cc9343e39cd68fff20f494a0/users/58e806c703214b11a5bb61cc7a7f95e6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:18.937788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7254f8-f539-4f4f-8759-01260665b561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd06bf7cc9343e39cd68fff20f494a0, user_id=58e806c703214b11a5bb61cc7a7f95e6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:18.941103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7254f8-f539-4f4f-8759-01260665b561 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:18.996643 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1160/2324] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:18.997509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1165/2325] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:18 2022] PUT /identity/v3/projects/fbd06bf7cc9343e39cd68fff20f494a0/users/58e806c703214b11a5bb61cc7a7f95e6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:50:19.000098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb121bbf-73b1-4a5a-898a-2a2586ba9ce5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.011892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.012203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.013028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.033180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb121bbf-73b1-4a5a-898a-2a2586ba9ce5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.085195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] MFA Rules not processed for user `58e806c703214b11a5bb61cc7a7f95e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:19.090346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:19.159853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.160526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.161349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.161349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.162023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.164008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.235999 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1161/2326] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:18 2022] POST /identity/v3/projects => generated 401 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:19.247028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.255522 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1166/2327] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:19.262791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.267518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:19.375952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939266d2-c1c1-4b17-b150-d40634546cf5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.382957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.384754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.385279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.386164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.386963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.389343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.424821 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1162/2328] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:19 2022] POST /identity/v3/users => generated 307 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:19.435683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.440652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:19.460678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45352765-5f43-47a9-bf92-22a2437690e9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.471779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b8e97-30e4-4d41-944a-24f5d2f10f75 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.472181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.472814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.473100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.473377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.474885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.479596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.507796 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1163/2329] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:19.513132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.519235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:19.526200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005eed7-6949-480b-a676-3546e91f6f48 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.537952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.538915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.539296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.539665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] PATH_INFO: `/v3/projects/5d289bfa2059463892f4bf326f23e304/users/8e351d08daeb4a25a7f96d8870b8daee/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.588766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d289bfa2059463892f4bf326f23e304, user_id=8e351d08daeb4a25a7f96d8870b8daee, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.591576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.614462 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1164/2330] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:19 2022] PUT /identity/v3/projects/5d289bfa2059463892f4bf326f23e304/users/8e351d08daeb4a25a7f96d8870b8daee/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:19.618198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.636579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.637867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.638168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.638446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.743546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.746339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.757493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601c5af-f94d-4cc7-962b-c6a5d11259d3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.763827 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1167/2331] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:19.769035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.769770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.770548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.770712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.771437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.773725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.788575 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1165/2332] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:50:19.793221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.807315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2b757-f6b7-4e48-977c-167237a8b6fc service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.815175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e63498-409c-4f46-bb6c-3bcceddd7dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:19.818392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e63498-409c-4f46-bb6c-3bcceddd7dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.818392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e63498-409c-4f46-bb6c-3bcceddd7dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.818392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e63498-409c-4f46-bb6c-3bcceddd7dc1 admin admin] PATH_INFO: `/v3/projects/5d289bfa2059463892f4bf326f23e304/users/8e351d08daeb4a25a7f96d8870b8daee/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.825379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e63498-409c-4f46-bb6c-3bcceddd7dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d289bfa2059463892f4bf326f23e304, user_id=8e351d08daeb4a25a7f96d8870b8daee, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:19.830582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e63498-409c-4f46-bb6c-3bcceddd7dc1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:19.862477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:19.862727 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1168/2333] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:19 2022] PUT /identity/v3/projects/5d289bfa2059463892f4bf326f23e304/users/8e351d08daeb4a25a7f96d8870b8daee/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:19.872203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:19.872481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:19.874180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:19.879662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928bdd4-c27b-4f0b-bea9-bfb372250557 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:19.944658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] MFA Rules not processed for user `8e351d08daeb4a25a7f96d8870b8daee`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:19.945666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:20.002929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.003885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.004168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.004473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] PATH_INFO: `/v3/users/58e806c703214b11a5bb61cc7a7f95e6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.007959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e806c703214b11a5bb61cc7a7f95e6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:20.008804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:20.125059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e806c703214b11a5bb61cc7a7f95e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:20.127173 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1169/2334] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:20.132675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.134315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:20.144280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e806c703214b11a5bb61cc7a7f95e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:20.178756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:20.205274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e806c703214b11a5bb61cc7a7f95e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:20.214212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:20.234783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e806c703214b11a5bb61cc7a7f95e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:20.235446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83fc8e8b-64ed-44be-87ed-26afbee0724c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e806c703214b11a5bb61cc7a7f95e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:20.257357 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1166/2335] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:19 2022] DELETE /identity/v3/users/58e806c703214b11a5bb61cc7a7f95e6 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:20.264281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.264616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.264854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.321584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07cfd4-2e94-4914-bffa-9030ae9cb2c7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:20.339583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:20.346831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:20.474543 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1167/2336] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:20.485121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.487192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:20.496899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.497565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.497848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.498687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.515438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:20.675505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:20.679842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:20.715039 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1170/2337] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:20.871702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:20.873277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:20.894856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e0c23-2438-4428-9dd1-72ccf4a53ae7 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:20.903920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:20.905250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:20.905772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:20.906251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:20.908590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:20.977114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cf28a-d277-4304-a534-2c6655c1670e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:21.033854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:21.035222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:21.035508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:21.035788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:21.045993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:21.058996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:21.061034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:21.099817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:21.100556 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1168/2338] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:21.101573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:21.124262 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1171/2339] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:50:21.744550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:21.747565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:21.774280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd673b5-8947-454e-80e4-857cc6b1fe8b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:21.811026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:21.812616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:21.812987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:21.813911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:21.815354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:21.817070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] PATH_INFO: `/v3/projects/fbd06bf7cc9343e39cd68fff20f494a0` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:21.822641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd06bf7cc9343e39cd68fff20f494a0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:21.824778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:21.853101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd37eef-1861-411b-9cf1-b3ad1509ea22 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:21.870631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:21.871383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:21.871839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:21.872214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] PATH_INFO: `/v3/users/aa9e19a592e34a9ab3f4ab6f4d0b6ea8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:21.873731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9e19a592e34a9ab3f4ab6f4d0b6ea8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:21.877242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:22.029975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9e19a592e34a9ab3f4ab6f4d0b6ea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.037679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd06bf7cc9343e39cd68fff20f494a0', 'request_id': 'req-32a52920-94af-4bd1-b203-d869fb7db3ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.050166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9e19a592e34a9ab3f4ab6f4d0b6ea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.051759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9e19a592e34a9ab3f4ab6f4d0b6ea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.055477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd06bf7cc9343e39cd68fff20f494a0', 'request_id': 'req-32a52920-94af-4bd1-b203-d869fb7db3ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.069197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd06bf7cc9343e39cd68fff20f494a0', 'request_id': 'req-32a52920-94af-4bd1-b203-d869fb7db3ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.076259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9e19a592e34a9ab3f4ab6f4d0b6ea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.077867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b41d9b21-e1a0-41ef-b1b0-c10104898af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9e19a592e34a9ab3f4ab6f4d0b6ea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.108114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] The token cache is being invalidate because project fbd06bf7cc9343e39cd68fff20f494a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:22.108407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.112585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-32a52920-94af-4bd1-b203-d869fb7db3ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:22.122109 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1169/2340] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:21 2022] DELETE /identity/v3/projects/fbd06bf7cc9343e39cd68fff20f494a0 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:22.126649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.127097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.127375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.128777 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1172/2341] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:21 2022] DELETE /identity/v3/users/aa9e19a592e34a9ab3f4ab6f4d0b6ea8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:22.139914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbef545-4062-4ef6-bd5f-4a1a3df8ae86 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.140214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbef545-4062-4ef6-bd5f-4a1a3df8ae86 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.140489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbef545-4062-4ef6-bd5f-4a1a3df8ae86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.198841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:22.200672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:22.216539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8dbef545-4062-4ef6-bd5f-4a1a3df8ae86 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:22.217770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8dbef545-4062-4ef6-bd5f-4a1a3df8ae86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:22.233684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:22.329637 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1173/2342] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:22.359481 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1170/2343] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:22.369182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01bb4-e96f-4eaa-bb7d-7c627532f8ab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.369729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01bb4-e96f-4eaa-bb7d-7c627532f8ab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.370233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01bb4-e96f-4eaa-bb7d-7c627532f8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.381547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:22.382463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:22.427616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a9a01bb4-e96f-4eaa-bb7d-7c627532f8ab None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:22.431252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a9a01bb4-e96f-4eaa-bb7d-7c627532f8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:22.445811 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1174/2344] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:22.449972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a01bb4-e96f-4eaa-bb7d-7c627532f8ab None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:22.474680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a01bb4-e96f-4eaa-bb7d-7c627532f8ab None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:22.488935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:22.489766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.490237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.490514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.491174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:22.491478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:22.493098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:22.527903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9032ae-cb92-4219-ba27-b86e88a89b0d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:22.529786 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1175/2345] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:50:22.534044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:22.542205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:22.542872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.543153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.543536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.554811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9978b6c-03f6-430c-81c7-1e3398d52d9f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:22.558279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:22.559723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:22.559841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:22.560435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.560725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.561670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.562393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:22.566197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:22.572885 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1171/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:50:22.617003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1176/2347] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:22 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:22.617658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:22.635420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fdc66-726c-42dd-a528-4891df2bec9c service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:22.750571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:22.753727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.754290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.754663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.756183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:22.759573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:22.824978 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1172/2348] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:22 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:50:22.827110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:22.839722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505994e9-ee4e-4f6e-9981-7598e6eac873 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:22.846631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:22.847260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.847548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.849820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.850780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:22.852718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:22.854263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:22.882076 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1177/2349] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:22.884766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:22.898415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032e78-1292-480e-ba12-c26cbc85e4ae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:22.909762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bae4d-5690-4f1e-af78-3feaaf14642a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:22.910419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bae4d-5690-4f1e-af78-3feaaf14642a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.910703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bae4d-5690-4f1e-af78-3feaaf14642a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.911505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bae4d-5690-4f1e-af78-3feaaf14642a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:22.916588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bae4d-5690-4f1e-af78-3feaaf14642a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:22.919202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bae4d-5690-4f1e-af78-3feaaf14642a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:22.947507 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1173/2350] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:22.951470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:22.965932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575656a-cfc8-458c-a752-bf52cde05ce0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:22.985735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:22.990780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:22.995252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:22.995823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] PATH_INFO: `/v3/projects/203cdcc6dae140de93209159d4b73fcb/users/a4033d708a6a4872bff480d5ce50abbd/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.034665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203cdcc6dae140de93209159d4b73fcb, user_id=a4033d708a6a4872bff480d5ce50abbd, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:23.041816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:23.071090 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1178/2351] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:22 2022] PUT /identity/v3/projects/203cdcc6dae140de93209159d4b73fcb/users/a4033d708a6a4872bff480d5ce50abbd/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:23.081222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05607d6-7691-4f82-b00c-ec6a60a6b5a5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.081560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05607d6-7691-4f82-b00c-ec6a60a6b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.082248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05607d6-7691-4f82-b00c-ec6a60a6b5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.161912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:23.163686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c05607d6-7691-4f82-b00c-ec6a60a6b5a5 None None] MFA Rules not processed for user `a4033d708a6a4872bff480d5ce50abbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:23.164331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:23.166163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c05607d6-7691-4f82-b00c-ec6a60a6b5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:23.251969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:23.279483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85af57f-4ea2-4b10-94a3-6b58cb9c16a4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:23.285802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:23.287086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.287563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.287933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] PATH_INFO: `/v3/projects/046393a40a784a2caaa40331e861ab78` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.289472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046393a40a784a2caaa40331e861ab78)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:23.291285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:23.319630 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1174/2352] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:23.396244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046393a40a784a2caaa40331e861ab78', 'request_id': 'req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.405555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046393a40a784a2caaa40331e861ab78', 'request_id': 'req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.419603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046393a40a784a2caaa40331e861ab78', 'request_id': 'req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.437590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] The token cache is being invalidate because project 046393a40a784a2caaa40331e861ab78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:23.438025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.438713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.440222 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1179/2353] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:23 2022] DELETE /identity/v3/projects/046393a40a784a2caaa40331e861ab78 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:50:23.445953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05607d6-7691-4f82-b00c-ec6a60a6b5a5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:23.458999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05607d6-7691-4f82-b00c-ec6a60a6b5a5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:23.541164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:23.542849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:23.548168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:23.551160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.551160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.551160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] PATH_INFO: `/v3/users/8e351d08daeb4a25a7f96d8870b8daee` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.553338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e351d08daeb4a25a7f96d8870b8daee)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:23.555594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:23.633541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e351d08daeb4a25a7f96d8870b8daee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.634296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:23.653146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e351d08daeb4a25a7f96d8870b8daee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.656068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc7eb2-7dcd-4b97-a503-1f2a9c1b6155 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:23.663143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e351d08daeb4a25a7f96d8870b8daee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.670972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb4f19-1144-44a2-8243-29896dde149e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:23.671623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb4f19-1144-44a2-8243-29896dde149e service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.671911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb4f19-1144-44a2-8243-29896dde149e service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.672199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb4f19-1144-44a2-8243-29896dde149e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.674154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98bb4f19-1144-44a2-8243-29896dde149e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:23.675568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e351d08daeb4a25a7f96d8870b8daee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.676358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36c10afe-79dc-4e4f-b70e-2ab8ba2b2176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e351d08daeb4a25a7f96d8870b8daee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:23.685050 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1175/2354] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:23 2022] DELETE /identity/v3/users/8e351d08daeb4a25a7f96d8870b8daee => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:23.785463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb4f19-1144-44a2-8243-29896dde149e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:23.789387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb4f19-1144-44a2-8243-29896dde149e service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:23.806961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1180/2355] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:23.916124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d2896-f14d-4ce2-a50d-e6a4420e87eb None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.916600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d2896-f14d-4ce2-a50d-e6a4420e87eb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.916713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d2896-f14d-4ce2-a50d-e6a4420e87eb None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.918140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1176/2356] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:50:23.924786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:23.925333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:23.925333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:23.986694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:23.995554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:24.261858 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1181/2357] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:24.289205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d2896-f14d-4ce2-a50d-e6a4420e87eb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.291882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d80d2896-f14d-4ce2-a50d-e6a4420e87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:24.482121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d2896-f14d-4ce2-a50d-e6a4420e87eb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.514192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5ff9b-9b49-402e-8041-981511000ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.515165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5ff9b-9b49-402e-8041-981511000ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.515488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5ff9b-9b49-402e-8041-981511000ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.515757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5ff9b-9b49-402e-8041-981511000ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.531134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.537047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:24.537917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:24.544693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5ff9b-9b49-402e-8041-981511000ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.551518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5ff9b-9b49-402e-8041-981511000ff8 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.582323 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1177/2358] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:24.657377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9376-b2f1-4ea5-afa3-44c1ce1ccdcd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.657668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9376-b2f1-4ea5-afa3-44c1ce1ccdcd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.657973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b9376-b2f1-4ea5-afa3-44c1ce1ccdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.704986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e95b9376-b2f1-4ea5-afa3-44c1ce1ccdcd None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:24.706325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e95b9376-b2f1-4ea5-afa3-44c1ce1ccdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:24.718463 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1178/2359] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:24.751844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9376-b2f1-4ea5-afa3-44c1ce1ccdcd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:24.758106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ad757-6213-4e81-8239-54b3c0627080 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.769931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.771384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.772000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.775243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b9376-b2f1-4ea5-afa3-44c1ce1ccdcd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:24.779310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] PATH_INFO: `/v3/projects/5d289bfa2059463892f4bf326f23e304` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.779925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5b595-7b2c-45b4-8fc9-1155070e5e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:24.780661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b595-7b2c-45b4-8fc9-1155070e5e7a service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.781133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b595-7b2c-45b4-8fc9-1155070e5e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.781505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b595-7b2c-45b4-8fc9-1155070e5e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.782866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d289bfa2059463892f4bf326f23e304)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.786659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.792842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5b595-7b2c-45b4-8fc9-1155070e5e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:24.794652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5b595-7b2c-45b4-8fc9-1155070e5e7a service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:24.805789 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1179/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:24.853813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:24.873777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d289bfa2059463892f4bf326f23e304', 'request_id': 'req-07d05191-978d-4fca-bc81-85178cb5b29f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:24.892176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d289bfa2059463892f4bf326f23e304', 'request_id': 'req-07d05191-978d-4fca-bc81-85178cb5b29f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:24.912530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d289bfa2059463892f4bf326f23e304', 'request_id': 'req-07d05191-978d-4fca-bc81-85178cb5b29f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:24.934737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] The token cache is being invalidate because project 5d289bfa2059463892f4bf326f23e304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:24.935659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:24.937159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07d05191-978d-4fca-bc81-85178cb5b29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:24.946399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:24.946943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:24.947184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:24.950713 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1182/2361] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:24 2022] DELETE /identity/v3/projects/5d289bfa2059463892f4bf326f23e304 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:24.997413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:24.998858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:25.165761 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1180/2362] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:50:25.168228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2b06-d552-4407-8001-34dae65ba665 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.168398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.168498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2b06-d552-4407-8001-34dae65ba665 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.168772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2b06-d552-4407-8001-34dae65ba665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.171196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:25.200852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e80e2b06-d552-4407-8001-34dae65ba665 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:25.202266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e80e2b06-d552-4407-8001-34dae65ba665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:25.211486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a36f7e-7e62-42f8-ac07-ceb8c72acc30 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.216870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1183/2363] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:25.222357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e2b06-d552-4407-8001-34dae65ba665 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.225842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.226577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.226934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.227374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] PATH_INFO: `/v3/users/a4033d708a6a4872bff480d5ce50abbd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.230053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4033d708a6a4872bff480d5ce50abbd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.236947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.240663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e2b06-d552-4407-8001-34dae65ba665 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.253681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.254594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.256730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.257076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.257874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:25.258246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.260775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.282977 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1184/2364] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:25.288324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.326931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4033d708a6a4872bff480d5ce50abbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:25.328862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e262f-09c3-4c50-8b53-19512999c549 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.343424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.344769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.345132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.346473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.347127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.352311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.369963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4033d708a6a4872bff480d5ce50abbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:25.382787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4033d708a6a4872bff480d5ce50abbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:25.392996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4033d708a6a4872bff480d5ce50abbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:25.393912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-027e4ed5-0748-4a67-97be-0829b53a94aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4033d708a6a4872bff480d5ce50abbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:25.408556 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1181/2365] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:25 2022] DELETE /identity/v3/users/a4033d708a6a4872bff480d5ce50abbd => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:25.410433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682a1df-2e52-4646-a883-e5a826d3c61b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.414528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682a1df-2e52-4646-a883-e5a826d3c61b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.415960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682a1df-2e52-4646-a883-e5a826d3c61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.417423 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1185/2366] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:25 2022] POST /identity/v3/projects => generated 393 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:25.418370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.420110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:25.502387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6682a1df-2e52-4646-a883-e5a826d3c61b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:25.504415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6682a1df-2e52-4646-a883-e5a826d3c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:25.587039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa3b98-e4f3-4e9f-bccf-328b8f9dfff5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.598899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.601458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.603254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.603646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.604521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.610069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.619119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:25.652676 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1182/2367] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:50:25.670287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682a1df-2e52-4646-a883-e5a826d3c61b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.678862 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1186/2368] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:25 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:25.682793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.682997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682a1df-2e52-4646-a883-e5a826d3c61b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.688304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.688919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.689293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:25.689701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.690024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.690705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.692896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.707193 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1183/2369] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:25.711142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.736868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d452e9-0ec2-4927-a6a0-4a971afab558 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.750807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.752213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.753388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.753927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.756100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.757869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.788050 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1184/2370] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:25.792447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:25.802967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b5cae-9e76-4685-9a4b-3c32350cc12c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.809647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86d40b-7f40-4a31-b7ab-006146ef5e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.810324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d40b-7f40-4a31-b7ab-006146ef5e0c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.810606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d40b-7f40-4a31-b7ab-006146ef5e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.810879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d40b-7f40-4a31-b7ab-006146ef5e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.815182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77aa618-b7dc-46dc-bf08-f47531e66c9a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:25.825812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:25.827347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.827750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.829004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] PATH_INFO: `/v3/projects/15a54c9ead204616be1ffd9ac9c7d7b5/users/568e205f6f9946e9a73ac605bcafec0b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.834794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86d40b-7f40-4a31-b7ab-006146ef5e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.836291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86d40b-7f40-4a31-b7ab-006146ef5e0c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.877344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1187/2371] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:25.878482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a54c9ead204616be1ffd9ac9c7d7b5, user_id=568e205f6f9946e9a73ac605bcafec0b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:25.882700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:25.918453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ae3c-8a9e-4aeb-a434-8c2236b9af0c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:25.918815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ae3c-8a9e-4aeb-a434-8c2236b9af0c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:25.918923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1185/2372] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:25 2022] PUT /identity/v3/projects/15a54c9ead204616be1ffd9ac9c7d7b5/users/568e205f6f9946e9a73ac605bcafec0b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:25.919007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ae3c-8a9e-4aeb-a434-8c2236b9af0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:25.944807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:25.956949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-af81ae3c-8a9e-4aeb-a434-8c2236b9af0c None None] MFA Rules not processed for user `568e205f6f9946e9a73ac605bcafec0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:25.959310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af81ae3c-8a9e-4aeb-a434-8c2236b9af0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:26.155003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1188/2373] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:26.156656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:26.276843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d86b13-0d12-43c5-aeb9-23a3c7960e3e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:26.292599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:26.293279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:26.293570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:26.294021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:26.294660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:26.297100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:26.354608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81ae3c-8a9e-4aeb-a434-8c2236b9af0c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:26.365277 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1186/2374] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:26 2022] POST /identity/v3/projects => generated 391 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:26.369595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81ae3c-8a9e-4aeb-a434-8c2236b9af0c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:26.481713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:26.482534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:26.482821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:26.483103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:26.484284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:26.488317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:26.544483 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1189/2375] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:26 2022] POST /identity/v3/users => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:50:26.546426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:26.567769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcbec6-df98-4173-a5c4-61b86ae70a28 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:26.590442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:26.592432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:26.592724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:26.593004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:26.597139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:26.601440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:26.633175 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1187/2376] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:26.637868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:26.657520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce3689-4e78-4657-a9e3-aadd2a0f4199 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:26.704727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:26.705326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:26.705612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:26.705988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] PATH_INFO: `/v3/projects/abc01f072bc74a5cbe8272c2c2247b1b/users/95c6ee7d25cb47a985fb4a19ecc0906c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:26.709498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:26.711357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:26.731857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d5f24-68f3-4731-b063-7e4a444b18d7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:26.744264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:26.744415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:26.744763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:26.745101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] PATH_INFO: `/v3/projects/203cdcc6dae140de93209159d4b73fcb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:26.747529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203cdcc6dae140de93209159d4b73fcb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:26.752362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:26.759798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc01f072bc74a5cbe8272c2c2247b1b, user_id=95c6ee7d25cb47a985fb4a19ecc0906c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:26.762556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:26.796248 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1190/2377] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:26 2022] PUT /identity/v3/projects/abc01f072bc74a5cbe8272c2c2247b1b/users/95c6ee7d25cb47a985fb4a19ecc0906c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:26.800678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:26.903188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203cdcc6dae140de93209159d4b73fcb', 'request_id': 'req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:26.906837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b1e49-a790-4364-a061-f3b28ec860c6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:26.919232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203cdcc6dae140de93209159d4b73fcb', 'request_id': 'req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:26.922675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:26.933981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203cdcc6dae140de93209159d4b73fcb', 'request_id': 'req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:26.953444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:26.960428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] The token cache is being invalidate because project 203cdcc6dae140de93209159d4b73fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:26.963397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:26.970268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a5a072d-4c32-476f-ab22-17fadcf7b1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:26.979816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1188/2378] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:26 2022] DELETE /identity/v3/projects/203cdcc6dae140de93209159d4b73fcb => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:26.983147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc00b1-66da-49c6-8137-5d86b7d30ee3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:26.983439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc00b1-66da-49c6-8137-5d86b7d30ee3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:26.983714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc00b1-66da-49c6-8137-5d86b7d30ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.015743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-89fc00b1-66da-49c6-8137-5d86b7d30ee3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:27.016941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-89fc00b1-66da-49c6-8137-5d86b7d30ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:27.023236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.024408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.024757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.025104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.025755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.032791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1189/2379] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:27.032986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.037774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6d75-f29b-458f-af36-c6838de1ec79 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.038158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6d75-f29b-458f-af36-c6838de1ec79 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.038509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6d75-f29b-458f-af36-c6838de1ec79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.042962 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1191/2380] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:27.047307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.049249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:27.066293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c71ff-51bb-4524-97d4-3cf9117b5eeb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.075841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaeaaa9-0e89-4a79-a765-1c023d12b727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.077118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeaaa9-0e89-4a79-a765-1c023d12b727 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.077408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeaaa9-0e89-4a79-a765-1c023d12b727 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.077688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeaaa9-0e89-4a79-a765-1c023d12b727 admin admin] PATH_INFO: `/v3/projects/abc01f072bc74a5cbe8272c2c2247b1b/users/95c6ee7d25cb47a985fb4a19ecc0906c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.078325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-80cb6d75-f29b-458f-af36-c6838de1ec79 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:27.079614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-80cb6d75-f29b-458f-af36-c6838de1ec79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:27.081042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaeaaa9-0e89-4a79-a765-1c023d12b727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc01f072bc74a5cbe8272c2c2247b1b, user_id=95c6ee7d25cb47a985fb4a19ecc0906c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.085722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaeaaa9-0e89-4a79-a765-1c023d12b727 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.089738 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1190/2381] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:27.093243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb6d75-f29b-458f-af36-c6838de1ec79 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.107706 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1192/2382] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:27 2022] PUT /identity/v3/projects/abc01f072bc74a5cbe8272c2c2247b1b/users/95c6ee7d25cb47a985fb4a19ecc0906c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:50:27.109973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb6d75-f29b-458f-af36-c6838de1ec79 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.123880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.124534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.125063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.183420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] MFA Rules not processed for user `95c6ee7d25cb47a985fb4a19ecc0906c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:27.184711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:27.213781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.214449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.214795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.215080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.215827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:27.216141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.218139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.236092 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1191/2383] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:27.238594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.249716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041a8c2-a9a7-4f79-9d56-2dd84c66f3be admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.277712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.278367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.278643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.279167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.279821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.281811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.350869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1193/2384] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:27.366186 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1192/2385] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:27 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:27.366359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.366453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.366537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:27.496889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d98b1d-8fb0-4949-a7cb-d1f3d1a052b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.515416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.517016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.517690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.520047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.521511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.526151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.571382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1193/2386] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:27 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:50:27.575716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.589586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7670a-16c2-42f4-8b86-e9446e46d552 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.598618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40728e-0cc8-4845-9739-647160884053 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.610354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.611290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.611578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.612105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.613604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.616117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.624418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:27.630688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1194/2387] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:27.634533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.654812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3066ce-5118-4030-94d6-b4f5d05074e9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.671934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.672652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.673014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.673354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.674211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.677591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.699866 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1195/2388] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:27.707232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:27.714292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.714917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.715300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.715503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.737609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0aff-8f75-4b3d-947d-e461642f4715 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:27.742948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c283b2-9f00-4ed6-b4c8-2d11f3614c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:27.743758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c283b2-9f00-4ed6-b4c8-2d11f3614c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.744043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c283b2-9f00-4ed6-b4c8-2d11f3614c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.744319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c283b2-9f00-4ed6-b4c8-2d11f3614c51 admin admin] PATH_INFO: `/v3/projects/a7933149ce16427e84de63058fff1142/users/3f0ba978c04048a29bf4132a1e7379b9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.766713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c283b2-9f00-4ed6-b4c8-2d11f3614c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7933149ce16427e84de63058fff1142, user_id=3f0ba978c04048a29bf4132a1e7379b9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.771735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c283b2-9f00-4ed6-b4c8-2d11f3614c51 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.786646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:27.795701 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1196/2389] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:27 2022] PUT /identity/v3/projects/a7933149ce16427e84de63058fff1142/users/3f0ba978c04048a29bf4132a1e7379b9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:27.804531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100a06c-6edb-44f2-acf7-d2a541f6ad29 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:27.804531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100a06c-6edb-44f2-acf7-d2a541f6ad29 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:27.804531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100a06c-6edb-44f2-acf7-d2a541f6ad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:27.828049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:27.830743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:27.847986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8100a06c-6edb-44f2-acf7-d2a541f6ad29 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:27.849218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1194/2390] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:27 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:27.863225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8100a06c-6edb-44f2-acf7-d2a541f6ad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:28.007290 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1197/2391] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:28.030451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.037902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:28.217932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecc67c-1cd4-46f9-9f9f-3a241f5ccdb0 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.350301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efcc49-363f-45f3-9fc9-8b1baea52dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.350301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efcc49-363f-45f3-9fc9-8b1baea52dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.350301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efcc49-363f-45f3-9fc9-8b1baea52dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.350799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efcc49-363f-45f3-9fc9-8b1baea52dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.361979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efcc49-363f-45f3-9fc9-8b1baea52dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.363436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efcc49-363f-45f3-9fc9-8b1baea52dc7 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.375645 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1195/2392] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:28.415513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100a06c-6edb-44f2-acf7-d2a541f6ad29 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.430004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100a06c-6edb-44f2-acf7-d2a541f6ad29 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.435564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.436212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.436493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.436767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] PATH_INFO: `/v3/users/568e205f6f9946e9a73ac605bcafec0b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.437977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=568e205f6f9946e9a73ac605bcafec0b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.440371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.522010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568e205f6f9946e9a73ac605bcafec0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:28.541372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568e205f6f9946e9a73ac605bcafec0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:28.554071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568e205f6f9946e9a73ac605bcafec0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:28.570125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568e205f6f9946e9a73ac605bcafec0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:28.570842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568e205f6f9946e9a73ac605bcafec0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:28.582717 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1198/2393] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:28 2022] DELETE /identity/v3/users/568e205f6f9946e9a73ac605bcafec0b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:28.584234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.584520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.584874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.609577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:28.610832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:28.619983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:28.747003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1196/2394] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:50:28.758356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:28.760738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:28.912822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103307a-c84d-4d7f-a05c-dcc928218a1b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:28.917177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:28.918126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:28.918452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:28.918831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:28.933225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:28.935354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:28.966826 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1199/2395] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:30.148621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:30.150203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:30.171577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6323b-4e6c-487e-bf9c-c0661fd3e01a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:30.174512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:30.176858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:30.193226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:30.193871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:30.196819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:30.197115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] PATH_INFO: `/v3/projects/15a54c9ead204616be1ffd9ac9c7d7b5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:30.198417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a54c9ead204616be1ffd9ac9c7d7b5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:30.207065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:30.222296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:30.341529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a54c9ead204616be1ffd9ac9c7d7b5', 'request_id': 'req-bf1def94-e5e3-47f8-abca-d37c7b13a312'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.363978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a54c9ead204616be1ffd9ac9c7d7b5', 'request_id': 'req-bf1def94-e5e3-47f8-abca-d37c7b13a312'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.378473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a54c9ead204616be1ffd9ac9c7d7b5', 'request_id': 'req-bf1def94-e5e3-47f8-abca-d37c7b13a312'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.403242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabe2f2-02ca-46ab-b420-3b8d480884d3 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:30.407215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] The token cache is being invalidate because project 15a54c9ead204616be1ffd9ac9c7d7b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:30.407518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.408493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.418028 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1197/2396] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:30 2022] DELETE /identity/v3/projects/15a54c9ead204616be1ffd9ac9c7d7b5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:30.419200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:30.420891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:30.586756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1def94-e5e3-47f8-abca-d37c7b13a312 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:30.592794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:30.593054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:30.593340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:30.593616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] PATH_INFO: `/v3/users/95c6ee7d25cb47a985fb4a19ecc0906c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:30.594898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c6ee7d25cb47a985fb4a19ecc0906c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:30.597618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:30.601736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8348a2-dd79-43fe-aab9-5439d4c0302d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:30.602419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8348a2-dd79-43fe-aab9-5439d4c0302d service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:30.602702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8348a2-dd79-43fe-aab9-5439d4c0302d service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:30.602984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8348a2-dd79-43fe-aab9-5439d4c0302d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:30.610998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0b8348a2-dd79-43fe-aab9-5439d4c0302d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:30.667107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:30.687762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c6ee7d25cb47a985fb4a19ecc0906c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.705530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c6ee7d25cb47a985fb4a19ecc0906c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.707554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c6ee7d25cb47a985fb4a19ecc0906c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.725855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c6ee7d25cb47a985fb4a19ecc0906c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.726731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c6ee7d25cb47a985fb4a19ecc0906c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:30.748376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8348a2-dd79-43fe-aab9-5439d4c0302d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:30.751366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8348a2-dd79-43fe-aab9-5439d4c0302d service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:30.767383 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1198/2397] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:30 2022] DELETE /identity/v3/users/95c6ee7d25cb47a985fb4a19ecc0906c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:30.773228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1200/2398] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:31.931595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:31.934405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:32.041657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6638d3-78ed-4283-8841-783e099b8504 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:32.047304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:32.048183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.048542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.049028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] PATH_INFO: `/v3/projects/abc01f072bc74a5cbe8272c2c2247b1b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.050343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abc01f072bc74a5cbe8272c2c2247b1b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:32.053447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:32.148081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abc01f072bc74a5cbe8272c2c2247b1b', 'request_id': 'req-5640ac52-9a20-4747-9496-23e9c2a3f309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:32.161917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abc01f072bc74a5cbe8272c2c2247b1b', 'request_id': 'req-5640ac52-9a20-4747-9496-23e9c2a3f309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:32.177566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abc01f072bc74a5cbe8272c2c2247b1b', 'request_id': 'req-5640ac52-9a20-4747-9496-23e9c2a3f309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:32.196711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] The token cache is being invalidate because project abc01f072bc74a5cbe8272c2c2247b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:32.204574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:32.205732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5640ac52-9a20-4747-9496-23e9c2a3f309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:32.208694 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1199/2399] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:31 2022] DELETE /identity/v3/projects/abc01f072bc74a5cbe8272c2c2247b1b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:32.214874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff1f63-e358-4325-934b-3c5da1095a48 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.214874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff1f63-e358-4325-934b-3c5da1095a48 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.215478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff1f63-e358-4325-934b-3c5da1095a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.249427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-57ff1f63-e358-4325-934b-3c5da1095a48 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:32.251489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-57ff1f63-e358-4325-934b-3c5da1095a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:32.324571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:32.369703 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1201/2400] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:32.378765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf7f3c-d357-40f7-8cdf-d8e290626ace None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.383296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf7f3c-d357-40f7-8cdf-d8e290626ace None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.383710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf7f3c-d357-40f7-8cdf-d8e290626ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.613935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bbaf7f3c-d357-40f7-8cdf-d8e290626ace None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:32.615340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bbaf7f3c-d357-40f7-8cdf-d8e290626ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:32.662768 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1200/2401] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:32.666383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff1f63-e358-4325-934b-3c5da1095a48 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:32.685521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff1f63-e358-4325-934b-3c5da1095a48 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:32.690967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:32.693717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.694239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.694786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.696095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:32.697079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:32.700069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:32.736156 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1202/2402] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:32.741573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf7f3c-d357-40f7-8cdf-d8e290626ace None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:32.754433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf7f3c-d357-40f7-8cdf-d8e290626ace None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:32.772989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:32.773626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.774069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.774313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.774825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:32.776771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:32.843879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:32.852101 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1201/2403] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:32 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:32.859140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fa6fe-b1d5-410d-9b46-3ac663d1dbfb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:32.949774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:32.950757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:32.951342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:32.951824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:32.952792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:32.955113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:33.010519 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1203/2404] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:32 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:50:33.020265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:33.036169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6dc3f-a176-4020-8883-b693365414d3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:33.060980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:33.063880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:33.064252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:33.064450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:33.082097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:33.082097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:33.082306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:33.082306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:33.132507 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1202/2405] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:33.133687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:33.154366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:33.169247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42ea3-261f-4568-9b94-2326c6d47410 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:33.187642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:33.188562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:33.189019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:33.189465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:33.191897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:33.195063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:33.216726 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1203/2406] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:33.219723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:33.243044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11707bc-e305-46eb-81c9-da73b77597ed admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:33.248542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5dc33-51f2-4642-8ed8-6cfee73d8713 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:33.258967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10f0bc-a0cd-4f4f-a487-c0cbf66cd34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:33.259955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10f0bc-a0cd-4f4f-a487-c0cbf66cd34b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:33.260403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10f0bc-a0cd-4f4f-a487-c0cbf66cd34b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:33.260892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:33.261280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10f0bc-a0cd-4f4f-a487-c0cbf66cd34b admin admin] PATH_INFO: `/v3/projects/f8cb888faa2543d0886638aad6c892d9/users/9f7db29759504b0fbdbc11b23225c052/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:33.261532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:33.261816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:33.263167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:33.267249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:33.318116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10f0bc-a0cd-4f4f-a487-c0cbf66cd34b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cb888faa2543d0886638aad6c892d9, user_id=9f7db29759504b0fbdbc11b23225c052, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:33.326018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10f0bc-a0cd-4f4f-a487-c0cbf66cd34b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:33.352201 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1204/2407] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:33 2022] PUT /identity/v3/projects/f8cb888faa2543d0886638aad6c892d9/users/9f7db29759504b0fbdbc11b23225c052/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:33.364876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00e739-b35f-4109-9ad7-a9c83f0027e4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:33.365164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00e739-b35f-4109-9ad7-a9c83f0027e4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:33.365439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00e739-b35f-4109-9ad7-a9c83f0027e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:33.430493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fe00e739-b35f-4109-9ad7-a9c83f0027e4 None None] MFA Rules not processed for user `9f7db29759504b0fbdbc11b23225c052`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:33.432952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fe00e739-b35f-4109-9ad7-a9c83f0027e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:33.450393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:33.457653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:33.466035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:33.509167 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1204/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:50:33.644327 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1205/2409] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:33.647212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:33.662783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642437f3-8841-4e40-9a58-4288b18b25c7 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:33.805744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:33.807062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:33.807414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:33.807681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:33.808333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:33.816712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:33.881674 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1205/2410] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:33 2022] POST /identity/v3/projects => generated 393 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:33.885515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00e739-b35f-4109-9ad7-a9c83f0027e4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:33.897481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00e739-b35f-4109-9ad7-a9c83f0027e4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:34.056852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:34.058355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.058797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.059271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:34.061107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:34.063792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:34.095802 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1206/2411] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:33 2022] POST /identity/v3/users => generated 303 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:34.100185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:34.116895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8182f-1216-4d0f-b851-1e11fbcca0bb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:34.124037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:34.125719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.127169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.131163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:34.133078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:34.135662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:34.150198 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1206/2412] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:50:34.153738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:34.175835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83f8b-ea14-4988-9abb-d8585979abb3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:34.191062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:34.194249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.194908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.196137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] PATH_INFO: `/v3/projects/16e6a4bda4a9432cb209efb92f717c80/users/c2519eace9c147d1a648e4137cce8218/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:34.222297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e6a4bda4a9432cb209efb92f717c80, user_id=c2519eace9c147d1a648e4137cce8218, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:34.226679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:34.295249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:34.300255 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1207/2413] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:34 2022] PUT /identity/v3/projects/16e6a4bda4a9432cb209efb92f717c80/users/c2519eace9c147d1a648e4137cce8218/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:34.307705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ec7a-4ffd-4018-a5c5-4ff5ed9b0b9b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:34.446885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbbd1e-84e5-49f6-88f0-bc790feabf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:34.447513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbbd1e-84e5-49f6-88f0-bc790feabf65 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.447798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbbd1e-84e5-49f6-88f0-bc790feabf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.448077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbbd1e-84e5-49f6-88f0-bc790feabf65 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:34.449271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbbd1e-84e5-49f6-88f0-bc790feabf65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:34.451648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbbd1e-84e5-49f6-88f0-bc790feabf65 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:34.493946 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1207/2414] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:34.498485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:34.517906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d6dd2-941a-4ace-ab75-d589820ce042 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:34.538597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:34.539496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.539959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.540401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] PATH_INFO: `/v3/projects/16e6a4bda4a9432cb209efb92f717c80/users/c2519eace9c147d1a648e4137cce8218/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:34.546611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e6a4bda4a9432cb209efb92f717c80, user_id=c2519eace9c147d1a648e4137cce8218, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:34.549503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:34.584551 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1208/2415] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:34 2022] PUT /identity/v3/projects/16e6a4bda4a9432cb209efb92f717c80/users/c2519eace9c147d1a648e4137cce8218/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:34.596842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874c8a-d092-4b78-a460-fe4d8c16b43d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:34.597178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874c8a-d092-4b78-a460-fe4d8c16b43d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:34.597478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63874c8a-d092-4b78-a460-fe4d8c16b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:34.680324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-63874c8a-d092-4b78-a460-fe4d8c16b43d None None] MFA Rules not processed for user `c2519eace9c147d1a648e4137cce8218`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:34.684264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-63874c8a-d092-4b78-a460-fe4d8c16b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:34.837752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:34.912134 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1208/2416] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:35.514301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:35.516887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:35.543865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:35.709853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc98ac-88fc-408f-bf9b-bbd3320401df admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:35.718129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919f2c0-100a-4c0e-8fb4-7d3e2c78dae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:35.719137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919f2c0-100a-4c0e-8fb4-7d3e2c78dae3 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:35.719533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919f2c0-100a-4c0e-8fb4-7d3e2c78dae3 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:35.720208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919f2c0-100a-4c0e-8fb4-7d3e2c78dae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:35.747815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919f2c0-100a-4c0e-8fb4-7d3e2c78dae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:35.752895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919f2c0-100a-4c0e-8fb4-7d3e2c78dae3 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:35.786694 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1209/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:35 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:36.043926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63874c8a-d092-4b78-a460-fe4d8c16b43d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:36.135472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63874c8a-d092-4b78-a460-fe4d8c16b43d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:36.142281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:36.143264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:36.143780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:36.144352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] PATH_INFO: `/v3/users/9f7db29759504b0fbdbc11b23225c052` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:36.146874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7db29759504b0fbdbc11b23225c052)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:36.149191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:36.221676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7db29759504b0fbdbc11b23225c052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:36.241533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7db29759504b0fbdbc11b23225c052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:36.252378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7db29759504b0fbdbc11b23225c052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:36.264956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7db29759504b0fbdbc11b23225c052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:36.266243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7db29759504b0fbdbc11b23225c052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:36.279859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1209/2418] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:36 2022] DELETE /identity/v3/users/9f7db29759504b0fbdbc11b23225c052 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:36.284206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:36.284692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:36.285145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:36.314706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:36.315971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:36.453485 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1210/2419] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:36.480383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:36.482531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:36.592769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644b997-4328-4d10-9b98-a53969db7246 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:36.597236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:36.597859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:36.598324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:36.598506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:36.626839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:36.628676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:36.649316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1210/2420] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:37.392123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:37.399531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:37.436073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4781cd-e021-4d3a-aa58-40b7ece6fa99 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:37.440886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:37.441535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:37.441885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:37.442163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] PATH_INFO: `/v3/projects/f8cb888faa2543d0886638aad6c892d9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:37.443364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8cb888faa2543d0886638aad6c892d9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:37.445535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:37.547728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8cb888faa2543d0886638aad6c892d9', 'request_id': 'req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.561729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8cb888faa2543d0886638aad6c892d9', 'request_id': 'req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.586905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8cb888faa2543d0886638aad6c892d9', 'request_id': 'req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.613234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] The token cache is being invalidate because project f8cb888faa2543d0886638aad6c892d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:37.613556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.614410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a7b994dc-0bd6-4f59-a63e-c6d410928ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.622542 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1211/2421] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:37 2022] DELETE /identity/v3/projects/f8cb888faa2543d0886638aad6c892d9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:37.623818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:37.634635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376d38-9e95-4663-80f3-fadc129b5a53 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:37.657214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:37.755572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:37.758767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:37.759067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:37.759351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] PATH_INFO: `/v3/users/c2519eace9c147d1a648e4137cce8218` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:37.763779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2519eace9c147d1a648e4137cce8218)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:37.765766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:37.874780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2519eace9c147d1a648e4137cce8218'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.894264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2519eace9c147d1a648e4137cce8218'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.914042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2519eace9c147d1a648e4137cce8218'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.945301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2519eace9c147d1a648e4137cce8218'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.947353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2519eace9c147d1a648e4137cce8218'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:37.957863 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1211/2422] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:37 2022] DELETE /identity/v3/users/c2519eace9c147d1a648e4137cce8218 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:39.033986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:39.034443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:39.034718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:39.082761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:39.084362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:39.095028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] MFA Rules not processed for user `3f0ba978c04048a29bf4132a1e7379b9`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:39.096825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:39.107246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:39.219160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1212/2423] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:50:39.234506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:39.239220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:39.277045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427497d-7acb-4bdf-924a-5716f1d3cfae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:39.283454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:39.284364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:39.284855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:39.285284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] PATH_INFO: `/v3/projects/16e6a4bda4a9432cb209efb92f717c80` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:39.288466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e6a4bda4a9432cb209efb92f717c80)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:39.291933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:39.390037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7832f-f0ae-4d9a-bd6a-1175cf58047e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:39.399814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c577c-a00a-4c3e-a30e-07373c0a4017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:39.400853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c577c-a00a-4c3e-a30e-07373c0a4017 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:39.401451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c577c-a00a-4c3e-a30e-07373c0a4017 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:39.402208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c577c-a00a-4c3e-a30e-07373c0a4017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:39.414458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c577c-a00a-4c3e-a30e-07373c0a4017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:39.416297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c577c-a00a-4c3e-a30e-07373c0a4017 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:39.422994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:39.438143 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1213/2424] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:39.477117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e6a4bda4a9432cb209efb92f717c80', 'request_id': 'req-75c22eb0-5615-47df-bd58-79fe5fcae3c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:39.499625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e6a4bda4a9432cb209efb92f717c80', 'request_id': 'req-75c22eb0-5615-47df-bd58-79fe5fcae3c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:39.511831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e6a4bda4a9432cb209efb92f717c80', 'request_id': 'req-75c22eb0-5615-47df-bd58-79fe5fcae3c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:39.537398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] The token cache is being invalidate because project 16e6a4bda4a9432cb209efb92f717c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:39.538027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:39.538865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75c22eb0-5615-47df-bd58-79fe5fcae3c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:39.540846 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1212/2425] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:39 2022] DELETE /identity/v3/projects/16e6a4bda4a9432cb209efb92f717c80 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:39.545487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e8a44-d16f-4f04-8a72-5f166604b765 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:39.545788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e8a44-d16f-4f04-8a72-5f166604b765 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:39.546173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e8a44-d16f-4f04-8a72-5f166604b765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:39.603439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ed0e8a44-d16f-4f04-8a72-5f166604b765 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:39.608909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed0e8a44-d16f-4f04-8a72-5f166604b765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:39.834147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1214/2426] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:39.842256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12ec4-a467-48e9-8de1-c524c32627db None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:39.842717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12ec4-a467-48e9-8de1-c524c32627db None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:39.843135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12ec4-a467-48e9-8de1-c524c32627db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:39.920000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dfe12ec4-a467-48e9-8de1-c524c32627db None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:39.921607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dfe12ec4-a467-48e9-8de1-c524c32627db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:39.947549 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1213/2427] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:39.952430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e8a44-d16f-4f04-8a72-5f166604b765 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:39.972772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e8a44-d16f-4f04-8a72-5f166604b765 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:39.995534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:39.996125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:39.996410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.002010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.002824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:40.003149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:40.005394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:40.054750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1215/2428] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:40.059442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe12ec4-a467-48e9-8de1-c524c32627db None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:40.078208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe12ec4-a467-48e9-8de1-c524c32627db None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:40.090150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:40.096881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.097181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.097464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.098709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:40.104822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:40.249193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:40.249469 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1214/2429] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:40 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:40.268252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691914-a80f-4699-a026-6242909c8678 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:40.402286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:40.453134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:40.453884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.454393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.454747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.455737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:40.462102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:40.498305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:40.500443 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1216/2430] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:40 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:40.512348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0fa05-89da-4ad6-a481-9e24c957b329 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:40.518771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:40.519830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.521029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.521683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.522707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:40.525142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:40.539661 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1215/2431] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:40.543351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:40.562776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bb33e-68c4-4bbf-86cc-58cb80f200d5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:40.570560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab5936-8050-4807-ae61-ff36ee12ab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:40.571269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab5936-8050-4807-ae61-ff36ee12ab34 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.571618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab5936-8050-4807-ae61-ff36ee12ab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.572226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab5936-8050-4807-ae61-ff36ee12ab34 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.572730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab5936-8050-4807-ae61-ff36ee12ab34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:40.574435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab5936-8050-4807-ae61-ff36ee12ab34 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:40.588142 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1217/2432] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:50:40.592279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:40.606272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafb99b-3377-41cc-b3e4-a125b1aa9388 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:40.613544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:40.614894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.616532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.617008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] PATH_INFO: `/v3/projects/d969d135196e409584d6647d5186c9c9/users/870c4c66a15b4546bd981cfb5932b214/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.647360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d969d135196e409584d6647d5186c9c9, user_id=870c4c66a15b4546bd981cfb5932b214, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:40.650624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:40.685975 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1216/2433] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:40 2022] PUT /identity/v3/projects/d969d135196e409584d6647d5186c9c9/users/870c4c66a15b4546bd981cfb5932b214/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:40.687496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f905ee-4568-465c-988a-04167a79c800 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:40.687864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f905ee-4568-465c-988a-04167a79c800 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:40.688211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f905ee-4568-465c-988a-04167a79c800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:40.759890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-16f905ee-4568-465c-988a-04167a79c800 None None] MFA Rules not processed for user `870c4c66a15b4546bd981cfb5932b214`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:40.761548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-16f905ee-4568-465c-988a-04167a79c800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:40.863370 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1218/2434] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:40.868631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:40.994060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef97d8-df77-40d3-8d82-de2a39082a84 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.005977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.006220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.007927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.007927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.015584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.017729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.089369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f905ee-4568-465c-988a-04167a79c800 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.089897 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1217/2435] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:40 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:50:41.104119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f905ee-4568-465c-988a-04167a79c800 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.243027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.243666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.243950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.244223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.244874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.246851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.291181 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1219/2436] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:50:41 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:50:41.294320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.315088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918a897-8961-4864-8b1d-f7d569621e68 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.322500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.323566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.324056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.326578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.327423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.329426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.347438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1218/2437] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:50:41.349001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.364737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93de11c-d72e-4866-ba2b-7519ace31972 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.376726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.387497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.389132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.389430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] PATH_INFO: `/v3/projects/13d6a80cfe5e49eda7f76cde966f9799/users/ff6039a197324406a774cd55dc06b9be/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.446174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d6a80cfe5e49eda7f76cde966f9799, user_id=ff6039a197324406a774cd55dc06b9be, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.450472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.479209 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1220/2438] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:41 2022] PUT /identity/v3/projects/13d6a80cfe5e49eda7f76cde966f9799/users/ff6039a197324406a774cd55dc06b9be/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:50:41.479454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.500625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288047-263f-4d49-80b7-053dbe680f16 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.573273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:41.657672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed21f-2421-4703-b1cd-4249228cfe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.658988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed21f-2421-4703-b1cd-4249228cfe01 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.659458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed21f-2421-4703-b1cd-4249228cfe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.659915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed21f-2421-4703-b1cd-4249228cfe01 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.661627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed21f-2421-4703-b1cd-4249228cfe01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.664777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed21f-2421-4703-b1cd-4249228cfe01 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.734436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:41.737573 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1219/2439] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:50:41.754249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388feb-8db9-492d-8ec8-9426f234e742 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:41.773348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b3ed4-8352-4489-bf59-a563c27dbbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:41.774064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b3ed4-8352-4489-bf59-a563c27dbbaa admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.774519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b3ed4-8352-4489-bf59-a563c27dbbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.774667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b3ed4-8352-4489-bf59-a563c27dbbaa admin admin] PATH_INFO: `/v3/projects/13d6a80cfe5e49eda7f76cde966f9799/users/ff6039a197324406a774cd55dc06b9be/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.776746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b3ed4-8352-4489-bf59-a563c27dbbaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d6a80cfe5e49eda7f76cde966f9799, user_id=ff6039a197324406a774cd55dc06b9be, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:41.779454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b3ed4-8352-4489-bf59-a563c27dbbaa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:41.800836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:41.818669 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1221/2440] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:50:41 2022] PUT /identity/v3/projects/13d6a80cfe5e49eda7f76cde966f9799/users/ff6039a197324406a774cd55dc06b9be/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:41.843845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ef62-d05c-468a-be6c-fad4e193537e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.845105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ef62-d05c-468a-be6c-fad4e193537e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.845105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ef62-d05c-468a-be6c-fad4e193537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.915641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9595ef62-d05c-468a-be6c-fad4e193537e None None] MFA Rules not processed for user `ff6039a197324406a774cd55dc06b9be`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:41.916875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9595ef62-d05c-468a-be6c-fad4e193537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:41.934589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:41.934877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:41.935161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:41.958179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:41.959755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:42.105550 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1220/2441] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:42.116417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:42.116693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:42.117094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:42.123491 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1222/2442] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:50:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:42.141870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:42.143642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:42.183957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] MFA Rules not processed for user `870c4c66a15b4546bd981cfb5932b214`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:42.187184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:42.361382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1221/2443] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:42.362526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0da47-886f-4cc8-a38d-e3a307a579e4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:42.374467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:42.377734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:42.389371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20913dba-1090-47f9-b01b-13e7e3be04d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:42.391510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20913dba-1090-47f9-b01b-13e7e3be04d1 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:42.391948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20913dba-1090-47f9-b01b-13e7e3be04d1 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:42.392721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20913dba-1090-47f9-b01b-13e7e3be04d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:42.419775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20913dba-1090-47f9-b01b-13e7e3be04d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:42.423640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20913dba-1090-47f9-b01b-13e7e3be04d1 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:42.444737 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1223/2444] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:50:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:50:42.508595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:42.540886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e2419-7764-4fb8-9b7c-d6c4f3d25775 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:42.549386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:42.551002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:42.551339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:42.551906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:42.567880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:42.569811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:42.592433 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1222/2445] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:44.471689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132eef2-d112-45fe-8882-82f293a2e982 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:44.472202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132eef2-d112-45fe-8882-82f293a2e982 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:44.472487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132eef2-d112-45fe-8882-82f293a2e982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:44.537559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9132eef2-d112-45fe-8882-82f293a2e982 None None] MFA Rules not processed for user `3f0ba978c04048a29bf4132a1e7379b9`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:44.541381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9132eef2-d112-45fe-8882-82f293a2e982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:44.742156 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1224/2446] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:47.857032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:47.858565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:48.224979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca173a1e-5de1-4a81-9576-694302fbddc4 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:48.244677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:48.245433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:48.245846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:48.246987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:48.250320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:48.282621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:48.284398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:48.310884 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1223/2447] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:50:52.268793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ad521-05c2-450e-9636-080678b3c2ab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:52.270308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ad521-05c2-450e-9636-080678b3c2ab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:52.270819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ad521-05c2-450e-9636-080678b3c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:52.331540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ec9ad521-05c2-450e-9636-080678b3c2ab None None] MFA Rules not processed for user `dd431fc2d9834bea8df807edb91771ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:52.333494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ec9ad521-05c2-450e-9636-080678b3c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:52.459456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:52.616289 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1225/2448] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:57.236654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:57.237796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:57.326440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09dd76-4c2e-4fc4-8580-3dc29ce20899 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:57.344384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:57.345051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:57.345337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:57.345950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] PATH_INFO: `/v3/users/9232c6fb834f4f68a260f5e10212f891` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:57.347400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9232c6fb834f4f68a260f5e10212f891)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:57.349575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:57.434668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9232c6fb834f4f68a260f5e10212f891'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:57.451474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9232c6fb834f4f68a260f5e10212f891'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:57.461987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9232c6fb834f4f68a260f5e10212f891'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:57.465452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:57.471757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9232c6fb834f4f68a260f5e10212f891'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:57.472687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9232c6fb834f4f68a260f5e10212f891'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:57.480349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1224/2449] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:57 2022] DELETE /identity/v3/users/9232c6fb834f4f68a260f5e10212f891 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:57.484857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:57.485384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:57.485384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:57.506214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:57.507431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:57.575904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1226/2450] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:57.597126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:57.598780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:57.720104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2b021-a47e-4846-9967-e2f970fd1398 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:57.726841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:57.727583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:57.727964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:57.728578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:57.764412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:57.764985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:57.776867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1225/2451] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:50:58.219029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:58.221107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:58.237171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d5d50-2835-4947-8912-a32d93d367f7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:58.243357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:58.244095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:58.244480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:58.244827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] PATH_INFO: `/v3/projects/cda0d7766e3e4ddfa7471f9c9196745a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:58.247606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda0d7766e3e4ddfa7471f9c9196745a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:58.249661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:58.304455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda0d7766e3e4ddfa7471f9c9196745a', 'request_id': 'req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.311418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda0d7766e3e4ddfa7471f9c9196745a', 'request_id': 'req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.320155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda0d7766e3e4ddfa7471f9c9196745a', 'request_id': 'req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.330780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] The token cache is being invalidate because project cda0d7766e3e4ddfa7471f9c9196745a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:58.331076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.331712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.333267 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1227/2452] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:58 2022] DELETE /identity/v3/projects/cda0d7766e3e4ddfa7471f9c9196745a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:58.337246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:58.348304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd7df-ca19-4a97-89d9-ea4acc6ab204 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:58.406728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:58.407359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:58.407635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:58.407907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] PATH_INFO: `/v3/users/ed52a55d09dc4e2b89faf896307312ff` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:58.409040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed52a55d09dc4e2b89faf896307312ff)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:58.411810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:58.459958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed52a55d09dc4e2b89faf896307312ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.473188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed52a55d09dc4e2b89faf896307312ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.480572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed52a55d09dc4e2b89faf896307312ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.488072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed52a55d09dc4e2b89faf896307312ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.488789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9d18bf3-c655-4f5e-b739-b6b34776e8c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed52a55d09dc4e2b89faf896307312ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:58.494846 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1226/2453] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:50:58 2022] DELETE /identity/v3/users/ed52a55d09dc4e2b89faf896307312ff => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:59.329947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:59.332361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:59.351557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7aa4-dc62-49a3-8919-ec0a1669e633 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:59.485592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:59.486497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:59.487007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:59.522629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:59.523439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:59.523818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:59.524183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] PATH_INFO: `/v3/projects/b2fd49585fde4c078fff9ded78b036a1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:59.527464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2fd49585fde4c078fff9ded78b036a1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:59.529522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:59.532026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:59.533405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:59.545566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:59.557754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:50:59.646186 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1227/2454] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:50:59.663396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:59.665218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:59.669214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2fd49585fde4c078fff9ded78b036a1', 'request_id': 'req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:59.678617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2fd49585fde4c078fff9ded78b036a1', 'request_id': 'req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:59.695479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2fd49585fde4c078fff9ded78b036a1', 'request_id': 'req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:59.723483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] The token cache is being invalidate because project b2fd49585fde4c078fff9ded78b036a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:50:59.723926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:59.724905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1a46f2e-af43-4e0e-b0bf-f450f6d20b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:50:59.731025 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1228/2455] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:50:59 2022] DELETE /identity/v3/projects/b2fd49585fde4c078fff9ded78b036a1 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:50:59.737596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f7354-6af9-4c6b-a0e5-644e61a2dc02 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:59.739358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f7354-6af9-4c6b-a0e5-644e61a2dc02 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:59.740527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f7354-6af9-4c6b-a0e5-644e61a2dc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:59.776600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-330f7354-6af9-4c6b-a0e5-644e61a2dc02 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:59.778921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-330f7354-6af9-4c6b-a0e5-644e61a2dc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:59.829563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd238efa-f376-40fd-b6fe-9b177fdce613 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:59.899134 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1229/2456] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:50:59.907117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b432f09-7567-4ce4-b1b2-fe6a93a621e8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:59.907598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b432f09-7567-4ce4-b1b2-fe6a93a621e8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:59.908343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b432f09-7567-4ce4-b1b2-fe6a93a621e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:59.933204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7b432f09-7567-4ce4-b1b2-fe6a93a621e8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:50:59.934444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:59.934910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7b432f09-7567-4ce4-b1b2-fe6a93a621e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:59.935202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:59.935547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:59.935990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:59.938484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:50:59.944601 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1230/2457] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:50:59.948978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b432f09-7567-4ce4-b1b2-fe6a93a621e8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:50:59.960099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b432f09-7567-4ce4-b1b2-fe6a93a621e8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:50:59.969785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:50:59.970638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:50:59.971362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:50:59.971768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:50:59.972965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:50:59.973784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:50:59.976013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:50:59.995239 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1231/2458] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:50:59.998686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:00.015856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab458a0-b1b8-4689-bfd5-2dd38bc5c8bc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:00.022129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:00.023027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:00.023280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:00.023641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:00.024276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:00.027623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:00.058582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:00.060291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:00.067939 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1232/2459] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:50:59 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:51:00.070175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:00.073622 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1228/2460] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:50:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:51:00.178002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ba883-4bb5-46bd-bf71-716fc85839bf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:00.180111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:00.192561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:00.193453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:00.193453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:00.193696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:00.194705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:00.197750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:00.254474 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1233/2461] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:51:00 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:51:00.255970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:00.272704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:00.274100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6116f5-f155-4cbe-8579-53249697bcd8 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:00.281758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:00.282415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:00.282698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:00.282975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:00.283621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:00.287419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:00.310233 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1229/2462] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:51:00.313158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:00.336723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637a9bb-44e7-4d56-8ba2-1bebeb7a3e03 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:00.349174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb28b1-150d-478b-886f-492c8dc85265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:00.349794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb28b1-150d-478b-886f-492c8dc85265 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:00.353031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb28b1-150d-478b-886f-492c8dc85265 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:00.353494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb28b1-150d-478b-886f-492c8dc85265 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:00.354398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb28b1-150d-478b-886f-492c8dc85265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:00.362518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb28b1-150d-478b-886f-492c8dc85265 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:00.407327 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1234/2463] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:51:00.411299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:00.451140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40ec47-7f12-452f-a7fb-8cdf72012459 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:00.467064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:00.469155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:00.469819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:00.470273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] PATH_INFO: `/v3/projects/4f60e45cf4604d9497f253f7bb1e3e71/users/ef5c9474c1864436aa4a35057087ea26/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:00.516033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f60e45cf4604d9497f253f7bb1e3e71, user_id=ef5c9474c1864436aa4a35057087ea26, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:00.524068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:00.558325 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1230/2464] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:51:00 2022] PUT /identity/v3/projects/4f60e45cf4604d9497f253f7bb1e3e71/users/ef5c9474c1864436aa4a35057087ea26/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:51:00.570346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530da6ab-70f4-40eb-87f0-7edce28f3a33 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:00.570670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530da6ab-70f4-40eb-87f0-7edce28f3a33 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:00.571780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530da6ab-70f4-40eb-87f0-7edce28f3a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:00.639009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-530da6ab-70f4-40eb-87f0-7edce28f3a33 None None] MFA Rules not processed for user `ef5c9474c1864436aa4a35057087ea26`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:00.654311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-530da6ab-70f4-40eb-87f0-7edce28f3a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:00.863081 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1235/2465] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:00.866633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:01.016557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde8d79-440b-4cd6-ae4f-c728ad453921 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:01.028198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:01.028831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.029165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.029449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.030124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:01.032280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:01.070543 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1231/2466] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:51:00 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:51:01.074552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530da6ab-70f4-40eb-87f0-7edce28f3a33 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:01.087420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530da6ab-70f4-40eb-87f0-7edce28f3a33 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:01.182083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:01.182903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.183280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.183714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.184816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:01.187205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:01.219178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1236/2467] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:51:01 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:51:01.220559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:01.248322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cc47f-542f-4f9a-81a7-2ab1cfd77183 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:01.257467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:01.258416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.258625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.259630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.260376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:01.262388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:01.292018 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1232/2468] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:51:01.295837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:01.307348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2958a-3b2a-4e89-bf31-cd536ee60371 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:01.312402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:01.313073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.313356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.313663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] PATH_INFO: `/v3/projects/51bd31e4ee2644669935f06b30898771/users/25ac8c28aa0444d4981773615b312fdd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.335549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bd31e4ee2644669935f06b30898771, user_id=25ac8c28aa0444d4981773615b312fdd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:01.338342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:01.361877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1237/2469] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:51:01 2022] PUT /identity/v3/projects/51bd31e4ee2644669935f06b30898771/users/25ac8c28aa0444d4981773615b312fdd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:51:01.362973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:01.380158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e774a-176d-4aac-a4ef-9a3a1f5790aa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:01.474104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b4e20-7342-4f7b-94e3-849620aec2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:01.474104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b4e20-7342-4f7b-94e3-849620aec2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.474104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b4e20-7342-4f7b-94e3-849620aec2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.474104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b4e20-7342-4f7b-94e3-849620aec2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.474534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b4e20-7342-4f7b-94e3-849620aec2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:01.474534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b4e20-7342-4f7b-94e3-849620aec2b8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:01.509527 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1233/2470] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:51:01.513230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:01.525386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63089f-e586-40fd-92b9-b7ffdf303d30 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:01.533455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca7635-0373-4a3b-a860-0bf86d646329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:01.536379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7635-0373-4a3b-a860-0bf86d646329 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.536871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7635-0373-4a3b-a860-0bf86d646329 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.537287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7635-0373-4a3b-a860-0bf86d646329 admin admin] PATH_INFO: `/v3/projects/51bd31e4ee2644669935f06b30898771/users/25ac8c28aa0444d4981773615b312fdd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.539628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca7635-0373-4a3b-a860-0bf86d646329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bd31e4ee2644669935f06b30898771, user_id=25ac8c28aa0444d4981773615b312fdd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:01.545582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca7635-0373-4a3b-a860-0bf86d646329 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:01.565261 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1238/2471] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:51:01 2022] PUT /identity/v3/projects/51bd31e4ee2644669935f06b30898771/users/25ac8c28aa0444d4981773615b312fdd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:51:01.575961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.576831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.577110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.618638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] MFA Rules not processed for user `25ac8c28aa0444d4981773615b312fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:01.620333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:01.701990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:01.810040 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1234/2472] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:01.823414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd3e1-1d2e-4ea0-9781-e847f9d0c34c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:01.825106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd3e1-1d2e-4ea0-9781-e847f9d0c34c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:01.825382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd3e1-1d2e-4ea0-9781-e847f9d0c34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:01.873400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-081bd3e1-1d2e-4ea0-9781-e847f9d0c34c None None] MFA Rules not processed for user `ef5c9474c1864436aa4a35057087ea26`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:01.874664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-081bd3e1-1d2e-4ea0-9781-e847f9d0c34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:01.906800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:02.063951 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1239/2473] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:02.384169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:02.385859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:02.502015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e202aa-b64f-4164-a7ff-01973049385e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:02.512916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:02.513857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:02.514298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:02.515068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:02.527192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:02.529398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:02.561249 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1235/2474] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:03.068669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df9e1e-a8c6-4fbe-b133-8f94f998888c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:03.069195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df9e1e-a8c6-4fbe-b133-8f94f998888c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:03.069532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df9e1e-a8c6-4fbe-b133-8f94f998888c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:03.091650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-55df9e1e-a8c6-4fbe-b133-8f94f998888c None None] MFA Rules not processed for user `f74a2c8cd22c454fa8fdd7c2e1ef9230`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:03.092965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55df9e1e-a8c6-4fbe-b133-8f94f998888c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:03.168082 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1240/2475] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:06.655698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:06.657922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:06.752115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99d98b-96db-4a5d-8a2d-a7ceb46d97d9 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:06.758529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a15b2c-b588-4ccf-a249-5b6a736932d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:06.759214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a15b2c-b588-4ccf-a249-5b6a736932d8 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:06.759566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a15b2c-b588-4ccf-a249-5b6a736932d8 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:06.760048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a15b2c-b588-4ccf-a249-5b6a736932d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:06.762073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00a15b2c-b588-4ccf-a249-5b6a736932d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:06.779052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a15b2c-b588-4ccf-a249-5b6a736932d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:06.781061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a15b2c-b588-4ccf-a249-5b6a736932d8 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:06.795969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1236/2476] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:08.038355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df9e1e-a8c6-4fbe-b133-8f94f998888c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:08.072734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df9e1e-a8c6-4fbe-b133-8f94f998888c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:08.089485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd76d97-05ce-43a3-b615-009e8c5bc200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:08.090138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd76d97-05ce-43a3-b615-009e8c5bc200 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:08.090415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd76d97-05ce-43a3-b615-009e8c5bc200 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:08.090872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd76d97-05ce-43a3-b615-009e8c5bc200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:08.093653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bfd76d97-05ce-43a3-b615-009e8c5bc200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:08.215191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd76d97-05ce-43a3-b615-009e8c5bc200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:08.217517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd76d97-05ce-43a3-b615-009e8c5bc200 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:08.245884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1241/2477] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:10.699668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc412ef-bc77-4eac-9b01-620ba4af7f7c None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:10.700492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc412ef-bc77-4eac-9b01-620ba4af7f7c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:10.700996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc412ef-bc77-4eac-9b01-620ba4af7f7c None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:10.703149 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1237/2478] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:51:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:10.709730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:10.710270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:10.710476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:10.753332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:10.758440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:10.786450 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1242/2479] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:51:10.820377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc412ef-bc77-4eac-9b01-620ba4af7f7c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:10.822335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7bc412ef-bc77-4eac-9b01-620ba4af7f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:10.894361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:10.931055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc412ef-bc77-4eac-9b01-620ba4af7f7c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:10.935266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:10.935954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:10.937340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:10.937985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:10.951300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:10.953232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:10.983788 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1238/2480] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:19.260685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:19.262125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:19.276480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43550624-c548-4a5f-bbab-f5539cd7f196 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:19.282092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aced8ce-6202-4188-89d5-66f6ae31b0de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:19.282715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aced8ce-6202-4188-89d5-66f6ae31b0de service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:19.282997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aced8ce-6202-4188-89d5-66f6ae31b0de service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:19.283268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aced8ce-6202-4188-89d5-66f6ae31b0de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:19.292410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aced8ce-6202-4188-89d5-66f6ae31b0de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:19.294045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aced8ce-6202-4188-89d5-66f6ae31b0de service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:19.304708 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1243/2481] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:20.970495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:25.511770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:25.578926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961457-4b0c-41a3-8250-7da405442d0d service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:25.584387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:25.585016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:25.585374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:25.585569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] PATH_INFO: `/v3/users/ef5c9474c1864436aa4a35057087ea26` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:25.586713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5c9474c1864436aa4a35057087ea26)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:25.588626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:25.641346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5c9474c1864436aa4a35057087ea26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:25.656589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5c9474c1864436aa4a35057087ea26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:25.664535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5c9474c1864436aa4a35057087ea26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:25.672017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5c9474c1864436aa4a35057087ea26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:25.672617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5c9474c1864436aa4a35057087ea26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:25.678637 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1239/2482] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:51:25 2022] DELETE /identity/v3/users/ef5c9474c1864436aa4a35057087ea26 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:51:25.683106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:25.683638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:25.683763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:25.703994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:25.705182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:25.746958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:25.776425 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1244/2483] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:51:25.783221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:25.785045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:25.844240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b22ad0-0e40-47d8-88b7-290807b1169b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:25.848450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:25.849056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:25.849335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:25.849607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:25.859266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:25.860725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:25.873817 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1240/2484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:51:26.375261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:26.377416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:26.391848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0f9bc-ea14-4627-b0e7-e754f9602e77 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:26.398147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:26.399013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:26.399458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:26.399884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] PATH_INFO: `/v3/projects/4f60e45cf4604d9497f253f7bb1e3e71` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:26.401717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f60e45cf4604d9497f253f7bb1e3e71)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:26.404331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:26.478028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f60e45cf4604d9497f253f7bb1e3e71', 'request_id': 'req-8e5107a2-deca-41ac-a667-5613e62f53bb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.484983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f60e45cf4604d9497f253f7bb1e3e71', 'request_id': 'req-8e5107a2-deca-41ac-a667-5613e62f53bb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.495918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f60e45cf4604d9497f253f7bb1e3e71', 'request_id': 'req-8e5107a2-deca-41ac-a667-5613e62f53bb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.512551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] The token cache is being invalidate because project 4f60e45cf4604d9497f253f7bb1e3e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:51:26.512934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.513677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.515846 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1245/2485] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:51:26 2022] DELETE /identity/v3/projects/4f60e45cf4604d9497f253f7bb1e3e71 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:51:26.520657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:26.552061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439732d-7505-4cea-abc8-bcd3cfe536a1 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:26.584510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:26.645368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:26.646991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:26.648671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:26.648671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] PATH_INFO: `/v3/users/25ac8c28aa0444d4981773615b312fdd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:26.650231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ac8c28aa0444d4981773615b312fdd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:26.652736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:26.738644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ac8c28aa0444d4981773615b312fdd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.760162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ac8c28aa0444d4981773615b312fdd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.771941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ac8c28aa0444d4981773615b312fdd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.799538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ac8c28aa0444d4981773615b312fdd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.800286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab66fcbe-cb70-4a78-88b4-e483e2491b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ac8c28aa0444d4981773615b312fdd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:26.807481 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1241/2486] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:51:26 2022] DELETE /identity/v3/users/25ac8c28aa0444d4981773615b312fdd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:51:27.610180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:27.612143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:27.639998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5107a2-deca-41ac-a667-5613e62f53bb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:27.759443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:27.760415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:27.761184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:27.761792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] PATH_INFO: `/v3/projects/51bd31e4ee2644669935f06b30898771` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:27.763286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51bd31e4ee2644669935f06b30898771)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:27.766897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:27.864909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51bd31e4ee2644669935f06b30898771', 'request_id': 'req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:27.873130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51bd31e4ee2644669935f06b30898771', 'request_id': 'req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:27.885116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51bd31e4ee2644669935f06b30898771', 'request_id': 'req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:27.894935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:27.910715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] The token cache is being invalidate because project 51bd31e4ee2644669935f06b30898771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:51:27.911398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:27.913224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b539b8d-4eb2-4a8d-9abb-9c0ecbf5cfc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:27.916063 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1246/2487] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:51:27 2022] DELETE /identity/v3/projects/51bd31e4ee2644669935f06b30898771 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:51:27.921792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577fe2-8100-415d-87a2-3976380733f8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:27.922304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577fe2-8100-415d-87a2-3976380733f8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:27.922386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577fe2-8100-415d-87a2-3976380733f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:27.959217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-22577fe2-8100-415d-87a2-3976380733f8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:27.960710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22577fe2-8100-415d-87a2-3976380733f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:28.038433 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1242/2488] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:28.047077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffd4b1-c5fc-4d15-b09d-304f1438747f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.047366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffd4b1-c5fc-4d15-b09d-304f1438747f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.047641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffd4b1-c5fc-4d15-b09d-304f1438747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.068739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a6ffd4b1-c5fc-4d15-b09d-304f1438747f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:28.070265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a6ffd4b1-c5fc-4d15-b09d-304f1438747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:28.083412 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1247/2489] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:28.084953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22577fe2-8100-415d-87a2-3976380733f8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:28.097002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22577fe2-8100-415d-87a2-3976380733f8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:28.101945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:28.103221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.103745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.104341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.105157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:51:28.105627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:28.107271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:28.122262 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1243/2490] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:51:28.127151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffd4b1-c5fc-4d15-b09d-304f1438747f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:28.139354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffd4b1-c5fc-4d15-b09d-304f1438747f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:28.147711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:28.148588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.149067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.149489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.151249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:28.154016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:28.207793 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1248/2491] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:51:28 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:51:28.212708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:28.226166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40324fa-f8e2-4e44-bf30-b6e550c6099a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:28.271065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:28.370798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:28.372471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.372754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.373026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.373685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:28.380054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:28.425215 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1244/2492] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:51:28 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:51:28.429146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:28.446566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af35176-a910-4589-8b15-b9914586777f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:28.463306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:28.464117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.464392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.464668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.465421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:28.467075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:28.484883 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1249/2493] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:51:28.489729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:28.500702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bc6b6-4e80-4f4f-b7e5-1f04d8e0506c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:28.509456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e018a5e-1a4f-428d-b4ab-6c3d70fd3064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:28.510204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e018a5e-1a4f-428d-b4ab-6c3d70fd3064 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.510556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e018a5e-1a4f-428d-b4ab-6c3d70fd3064 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.511336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e018a5e-1a4f-428d-b4ab-6c3d70fd3064 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.511598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e018a5e-1a4f-428d-b4ab-6c3d70fd3064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:28.513264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e018a5e-1a4f-428d-b4ab-6c3d70fd3064 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:28.526541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1245/2494] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:51:28.529923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:28.542513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf44d0-4e44-42c3-8376-4fe0692366e6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:28.548722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:28.549677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.550157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.550505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] PATH_INFO: `/v3/projects/8693bafb9e424052ab91e56e9e4da6e7/users/43b9b4d83834456388c99abd3ecae3c7/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.571565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8693bafb9e424052ab91e56e9e4da6e7, user_id=43b9b4d83834456388c99abd3ecae3c7, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:28.574498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:28.614758 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1250/2495] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:51:28 2022] PUT /identity/v3/projects/8693bafb9e424052ab91e56e9e4da6e7/users/43b9b4d83834456388c99abd3ecae3c7/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:51:28.618697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b363a-b73e-4b18-9e8e-c76188202e9a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.618904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b363a-b73e-4b18-9e8e-c76188202e9a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.619180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b363a-b73e-4b18-9e8e-c76188202e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.685175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-210b363a-b73e-4b18-9e8e-c76188202e9a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:28.689974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-210b363a-b73e-4b18-9e8e-c76188202e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:28.825876 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1246/2496] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:28.835287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:28.837495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:28.923576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac92a-0fae-4415-9442-7de197ce333b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:28.928889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac16ca2-f92b-4578-9754-383cb22eb3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:28.929595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac16ca2-f92b-4578-9754-383cb22eb3fb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:28.930267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac16ca2-f92b-4578-9754-383cb22eb3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:28.930634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac16ca2-f92b-4578-9754-383cb22eb3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:28.943899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac16ca2-f92b-4578-9754-383cb22eb3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:28.945596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac16ca2-f92b-4578-9754-383cb22eb3fb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:28.957103 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1251/2497] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:30.858874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa415-bcd6-4548-b7ff-c3da125cc10d None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:30.858874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa415-bcd6-4548-b7ff-c3da125cc10d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:30.858874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa415-bcd6-4548-b7ff-c3da125cc10d None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:30.859723 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1247/2498] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:51:30.905357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5e796-8a03-4f62-a128-c90a34bcee00 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:30.905819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5e796-8a03-4f62-a128-c90a34bcee00 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:30.906116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5e796-8a03-4f62-a128-c90a34bcee00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:30.938945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-34d5e796-8a03-4f62-a128-c90a34bcee00 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:30.940410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-34d5e796-8a03-4f62-a128-c90a34bcee00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:31.084315 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1252/2499] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:31.104743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa415-bcd6-4548-b7ff-c3da125cc10d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:31.106732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f92fa415-bcd6-4548-b7ff-c3da125cc10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:31.140749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa415-bcd6-4548-b7ff-c3da125cc10d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:31.150563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3315a-e3e8-4ac7-bd9f-9770fce16f34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:31.151825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3315a-e3e8-4ac7-bd9f-9770fce16f34 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:31.153648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3315a-e3e8-4ac7-bd9f-9770fce16f34 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:31.154091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3315a-e3e8-4ac7-bd9f-9770fce16f34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:31.169406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3315a-e3e8-4ac7-bd9f-9770fce16f34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:31.172446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3315a-e3e8-4ac7-bd9f-9770fce16f34 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:31.193600 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1248/2500] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:34.268256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:34.268779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:34.269146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:34.292824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] MFA Rules not processed for user `3f0ba978c04048a29bf4132a1e7379b9`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:34.294727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:34.329487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:34.398819 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1253/2501] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:35.260837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd8ce7-3a91-4c04-8d9f-45c1495ed73d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:35.261765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd8ce7-3a91-4c04-8d9f-45c1495ed73d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:35.262368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd8ce7-3a91-4c04-8d9f-45c1495ed73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:35.296229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-afcd8ce7-3a91-4c04-8d9f-45c1495ed73d None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:35.298168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-afcd8ce7-3a91-4c04-8d9f-45c1495ed73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:35.383531 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1249/2502] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:51:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:51:35.410879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:35.413100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:35.497540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdab778-a8a0-4e70-86c6-33e2a5a47865 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:35.503818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad73a-d129-414c-884b-97f5a230374d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:35.504566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad73a-d129-414c-884b-97f5a230374d service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:35.504991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad73a-d129-414c-884b-97f5a230374d service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:35.505458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad73a-d129-414c-884b-97f5a230374d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:35.515193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad73a-d129-414c-884b-97f5a230374d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:35.517185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad73a-d129-414c-884b-97f5a230374d service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:35.528083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1254/2503] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:51:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:51:45.368258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:45.523584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=43 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:46.326890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:46.327186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:46.327460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:46.421642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] MFA Rules not processed for user `43b9b4d83834456388c99abd3ecae3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:46.423691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:46.641967 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1250/2504] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:46.738874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db1eb6-21fd-41b7-8562-948c178e92c0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:46.739851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db1eb6-21fd-41b7-8562-948c178e92c0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:46.740709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db1eb6-21fd-41b7-8562-948c178e92c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:46.780259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-36db1eb6-21fd-41b7-8562-948c178e92c0 None None] MFA Rules not processed for user `43b9b4d83834456388c99abd3ecae3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:46.782497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36db1eb6-21fd-41b7-8562-948c178e92c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:46.798038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1255/2505] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:47.395713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:47.397172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:47.620820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0c9f2-fb63-4b7e-9d3c-7c139ae08264 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:47.646548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:47.649515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:47.650011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:47.650453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:47.674688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:47.680497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:47.707536 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1251/2506] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:51:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:51:54.166211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614a07-0788-4fd2-aadd-92a0b72e16dd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:54.166733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614a07-0788-4fd2-aadd-92a0b72e16dd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:54.166833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76614a07-0788-4fd2-aadd-92a0b72e16dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:54.212446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-76614a07-0788-4fd2-aadd-92a0b72e16dd None None] MFA Rules not processed for user `f74a2c8cd22c454fa8fdd7c2e1ef9230`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:54.214371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-76614a07-0788-4fd2-aadd-92a0b72e16dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:54.361253 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1256/2507] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:57.695642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=66 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:58.427257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:58.428957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:58.445399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e34ac-8e7f-4f7e-8a37-7a27b0643f5e service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:58.454528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:58.456333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:58.456882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:58.457389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] PATH_INFO: `/v3/users/a020e015186b4c32950c2d4edd26374e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:58.459073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a020e015186b4c32950c2d4edd26374e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:58.462069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:58.611838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a020e015186b4c32950c2d4edd26374e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:58.649681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a020e015186b4c32950c2d4edd26374e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:58.679380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a020e015186b4c32950c2d4edd26374e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:58.699727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a020e015186b4c32950c2d4edd26374e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:58.700463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a020e015186b4c32950c2d4edd26374e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:58.722462 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1252/2508] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:51:58 2022] DELETE /identity/v3/users/a020e015186b4c32950c2d4edd26374e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:51:59.351468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ab60-469d-4064-92de-78081946c50a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:59.352672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ab60-469d-4064-92de-78081946c50a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:59.353572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ab60-469d-4064-92de-78081946c50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:59.384585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-eda1ab60-469d-4064-92de-78081946c50a None None] MFA Rules not processed for user `870c4c66a15b4546bd981cfb5932b214`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:51:59.386670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eda1ab60-469d-4064-92de-78081946c50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:59.423127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:59.425035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:51:59.501265 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1257/2509] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:51:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:51:59.545095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3f496-be2c-4bcd-888d-36a453f2939a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:59.552624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:59.553613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:59.554080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:59.554606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] PATH_INFO: `/v3/projects/d44361bde7c94fb6b01d992a8df306bb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:59.556222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44361bde7c94fb6b01d992a8df306bb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:59.558523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:59.677860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44361bde7c94fb6b01d992a8df306bb', 'request_id': 'req-3e89385d-72ae-40f6-98ab-de6f61ac1755'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.685158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44361bde7c94fb6b01d992a8df306bb', 'request_id': 'req-3e89385d-72ae-40f6-98ab-de6f61ac1755'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.692459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44361bde7c94fb6b01d992a8df306bb', 'request_id': 'req-3e89385d-72ae-40f6-98ab-de6f61ac1755'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.707389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] The token cache is being invalidate because project d44361bde7c94fb6b01d992a8df306bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:51:59.708080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.708306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.710032 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1253/2510] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:51:59 2022] DELETE /identity/v3/projects/d44361bde7c94fb6b01d992a8df306bb => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:51:59.714073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1ab60-469d-4064-92de-78081946c50a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:51:59.726420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1ab60-469d-4064-92de-78081946c50a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:51:59.817520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:51:59.818344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:51:59.818944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:51:59.819413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] PATH_INFO: `/v3/users/7ded5b10c6294acba430ea04b44fb72f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:51:59.820756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ded5b10c6294acba430ea04b44fb72f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:51:59.823097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:51:59.893787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:51:59.895847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ded5b10c6294acba430ea04b44fb72f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.904860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ded5b10c6294acba430ea04b44fb72f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.905447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ded5b10c6294acba430ea04b44fb72f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.917705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ded5b10c6294acba430ea04b44fb72f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.918855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e5d0f4a-38c2-45f6-be2d-f212f4ba73ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ded5b10c6294acba430ea04b44fb72f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:51:59.945875 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1258/2511] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:51:59 2022] DELETE /identity/v3/users/7ded5b10c6294acba430ea04b44fb72f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:00.552616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:00.554254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:00.570527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:00.581626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89385d-72ae-40f6-98ab-de6f61ac1755 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:00.735570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:00.737623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:00.738329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:00.738904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] PATH_INFO: `/v3/projects/ca6b53504e15462ea8552410a6481fca` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:00.741884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6b53504e15462ea8552410a6481fca)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:00.745662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:00.826462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6b53504e15462ea8552410a6481fca', 'request_id': 'req-98df67bd-61b7-4bed-ae01-f5452d443e58'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:00.836911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6b53504e15462ea8552410a6481fca', 'request_id': 'req-98df67bd-61b7-4bed-ae01-f5452d443e58'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:00.844511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6b53504e15462ea8552410a6481fca', 'request_id': 'req-98df67bd-61b7-4bed-ae01-f5452d443e58'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:00.869602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] The token cache is being invalidate because project ca6b53504e15462ea8552410a6481fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:00.870355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:00.871393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-98df67bd-61b7-4bed-ae01-f5452d443e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:00.886534 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1254/2512] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:00 2022] DELETE /identity/v3/projects/ca6b53504e15462ea8552410a6481fca => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:00.892453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f06-b5cc-451f-9934-558a31086b3a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:00.893558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f06-b5cc-451f-9934-558a31086b3a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:00.893672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f06-b5cc-451f-9934-558a31086b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:00.939047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a1b73f06-b5cc-451f-9934-558a31086b3a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:00.947609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a1b73f06-b5cc-451f-9934-558a31086b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:01.063209 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1259/2513] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:01.071559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98844-4501-4636-b939-7ef5529ed025 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.071853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98844-4501-4636-b939-7ef5529ed025 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.072140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98844-4501-4636-b939-7ef5529ed025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.094915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1cf98844-4501-4636-b939-7ef5529ed025 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:01.096604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1cf98844-4501-4636-b939-7ef5529ed025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:01.104987 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1255/2514] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:01.109263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73f06-b5cc-451f-9934-558a31086b3a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.119752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73f06-b5cc-451f-9934-558a31086b3a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.124696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.125305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.125624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.125955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.126611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:01.126918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.129552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.142609 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1260/2515] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:52:01.147497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf98844-4501-4636-b939-7ef5529ed025 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.157040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf98844-4501-4636-b939-7ef5529ed025 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.162392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.163071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.163535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.163881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.164604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.166978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.196313 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1256/2516] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:01 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:01.200891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.211997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc08d09-f36e-4955-a84f-83431e4e6c7f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.275659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.276273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.276559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.276837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.277465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.279417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.300154 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1261/2517] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:01 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:01.304541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.315443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f02e7-4b85-47f7-a66d-f9d96fe42edd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.320364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.320953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.321232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.321503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.322238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.323805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.334274 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1257/2518] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:01.338376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.348806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8022f-b2e3-4937-8304-e3e0b7db140b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.353508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d62deb-9e68-4ddd-b2d0-57a087f8822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.354225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62deb-9e68-4ddd-b2d0-57a087f8822b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.354492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62deb-9e68-4ddd-b2d0-57a087f8822b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.354769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62deb-9e68-4ddd-b2d0-57a087f8822b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.355409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d62deb-9e68-4ddd-b2d0-57a087f8822b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.357005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d62deb-9e68-4ddd-b2d0-57a087f8822b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.368259 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1262/2519] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:01.372460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.383257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718cb2f-971f-4a17-a000-baebde5ce960 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.388055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.388702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.388980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.389256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] PATH_INFO: `/v3/projects/359f9191e22641c9be16709d2fdf0b25/users/7874508e24964410b3e1f85cf0792be3/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.404429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=359f9191e22641c9be16709d2fdf0b25, user_id=7874508e24964410b3e1f85cf0792be3, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.407152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.420557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1258/2520] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:01 2022] PUT /identity/v3/projects/359f9191e22641c9be16709d2fdf0b25/users/7874508e24964410b3e1f85cf0792be3/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:01.431076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822835d-dc02-41ad-8f60-82f0115e75ff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.431367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822835d-dc02-41ad-8f60-82f0115e75ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.431641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822835d-dc02-41ad-8f60-82f0115e75ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.465725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2822835d-dc02-41ad-8f60-82f0115e75ff None None] MFA Rules not processed for user `7874508e24964410b3e1f85cf0792be3`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:01.467711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2822835d-dc02-41ad-8f60-82f0115e75ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:01.491221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:01.557406 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1263/2521] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:01.562148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.601090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:01.663875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a176-812a-4441-b29e-e5dc31303c0f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.670378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.671079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.671899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.672212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.672852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.675921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.712134 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1259/2522] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:01 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:01.716293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822835d-dc02-41ad-8f60-82f0115e75ff None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.728116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822835d-dc02-41ad-8f60-82f0115e75ff None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.801864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.802546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.802830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.803108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.804035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.806905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.836520 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1264/2523] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:01 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:01.840752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.872017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd075616-e905-4202-9566-b8919306d4ef admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.885606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.886410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.886689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.886959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.887601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.891084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:01.922913 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1260/2524] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:01.927688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:01.943467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd645a8-8999-4d3f-a5e9-e73c36027b28 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:01.953438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:01.954276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:01.954567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:01.954916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] PATH_INFO: `/v3/projects/273af7a59c414f60b2254a23a1966dc0/users/3a03d3b28f2f4769a09388de6223b42f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:01.987242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=273af7a59c414f60b2254a23a1966dc0, user_id=3a03d3b28f2f4769a09388de6223b42f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:01.990750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:02.026570 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1265/2525] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:01 2022] PUT /identity/v3/projects/273af7a59c414f60b2254a23a1966dc0/users/3a03d3b28f2f4769a09388de6223b42f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:02.027822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:02.039415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252b989-69d4-4023-a3b0-e43791e4294a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:02.113321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c5b23-752a-4463-a8a2-b2944540b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:02.114053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c5b23-752a-4463-a8a2-b2944540b4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:02.114454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c5b23-752a-4463-a8a2-b2944540b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:02.114823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c5b23-752a-4463-a8a2-b2944540b4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:02.115606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c5b23-752a-4463-a8a2-b2944540b4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:02.117977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c5b23-752a-4463-a8a2-b2944540b4b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:02.138380 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1261/2526] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:02.144067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:02.156616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9d9c0-9df2-4653-a5b3-4dedfe01cfba admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:02.161638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:02.162628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:02.162908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:02.163180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] PATH_INFO: `/v3/projects/273af7a59c414f60b2254a23a1966dc0/users/3a03d3b28f2f4769a09388de6223b42f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:02.165409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=273af7a59c414f60b2254a23a1966dc0, user_id=3a03d3b28f2f4769a09388de6223b42f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:02.168178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:02.199895 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1266/2527] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:02 2022] PUT /identity/v3/projects/273af7a59c414f60b2254a23a1966dc0/users/3a03d3b28f2f4769a09388de6223b42f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:02.230892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:02.231097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:02.231425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:02.263404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] MFA Rules not processed for user `3a03d3b28f2f4769a09388de6223b42f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:02.265154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:02.346927 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1262/2528] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:02.357864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:02.359870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:02.459096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e53e61-4040-4a88-94a9-4944221f6c0e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:02.465031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:02.465641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:02.465959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:02.466237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:02.474684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:02.477278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:02.479074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:02.508240 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1267/2529] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:03.745895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:03.747306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:03.764378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18d4a4-3e3d-4eab-a881-c478c1c7b2a4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:03.770647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc703501-4321-48b2-92dc-615f9fedd309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:03.771252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703501-4321-48b2-92dc-615f9fedd309 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:03.771536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703501-4321-48b2-92dc-615f9fedd309 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:03.771807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703501-4321-48b2-92dc-615f9fedd309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:03.796796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:03.873560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc703501-4321-48b2-92dc-615f9fedd309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:03.875450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc703501-4321-48b2-92dc-615f9fedd309 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:03.885404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1263/2530] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:07.379983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:07.483888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02d01f-67d4-4bcb-9308-a580d6021dff service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:07.489372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:07.490157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:07.490500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:07.490841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] PATH_INFO: `/v3/users/7874508e24964410b3e1f85cf0792be3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:07.492336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7874508e24964410b3e1f85cf0792be3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:07.494439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:07.589372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7874508e24964410b3e1f85cf0792be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:07.615674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7874508e24964410b3e1f85cf0792be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:07.618379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7874508e24964410b3e1f85cf0792be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:07.640705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7874508e24964410b3e1f85cf0792be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:07.641402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7874508e24964410b3e1f85cf0792be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:07.673887 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1268/2531] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:07 2022] DELETE /identity/v3/users/7874508e24964410b3e1f85cf0792be3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:07.678772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:07.679288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:07.679586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:07.699500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:07.701384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:07.800581 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1264/2532] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:07.813736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:07.816319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:07.885817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb19e81-a3cb-449c-899d-b35fb35a47aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:07.890641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:07.891372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:07.891753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:07.892135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:07.902842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:07.904576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:07.917105 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1269/2533] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:08.438847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:08.440814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:08.466544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf41ca2-955e-40a8-815e-220c6fa7a0a5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:08.478109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:08.478872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:08.479976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:08.480710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] PATH_INFO: `/v3/projects/359f9191e22641c9be16709d2fdf0b25` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:08.482217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=359f9191e22641c9be16709d2fdf0b25)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:08.485397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:08.613441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359f9191e22641c9be16709d2fdf0b25', 'request_id': 'req-ef5d4cb3-2acd-4a58-8259-5cfa530db546'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.620761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359f9191e22641c9be16709d2fdf0b25', 'request_id': 'req-ef5d4cb3-2acd-4a58-8259-5cfa530db546'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.629808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359f9191e22641c9be16709d2fdf0b25', 'request_id': 'req-ef5d4cb3-2acd-4a58-8259-5cfa530db546'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.668183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] The token cache is being invalidate because project 359f9191e22641c9be16709d2fdf0b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:08.668400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.669105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.673973 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1265/2534] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:08 2022] DELETE /identity/v3/projects/359f9191e22641c9be16709d2fdf0b25 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:08.674931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:08.692314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9ef2f-ceeb-4f58-af0c-9e77493a6c5a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:08.707549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:08.768440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:08.769572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:08.770102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:08.770586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] PATH_INFO: `/v3/users/3a03d3b28f2f4769a09388de6223b42f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:08.771802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a03d3b28f2f4769a09388de6223b42f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:08.773733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:08.826360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a03d3b28f2f4769a09388de6223b42f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.841351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a03d3b28f2f4769a09388de6223b42f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.842861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a03d3b28f2f4769a09388de6223b42f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.858377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a03d3b28f2f4769a09388de6223b42f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.859121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73d1d648-eb04-468d-bad0-073c361f5515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a03d3b28f2f4769a09388de6223b42f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:08.882153 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1270/2535] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:08 2022] DELETE /identity/v3/users/3a03d3b28f2f4769a09388de6223b42f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:09.627492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:09.630197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:09.658369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d4cb3-2acd-4a58-8259-5cfa530db546 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:09.710001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:09.726606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:09.727341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:09.729076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:09.730497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] PATH_INFO: `/v3/projects/273af7a59c414f60b2254a23a1966dc0` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:09.733574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] RBAC: Authorizing `identity:delete_project(project_id=273af7a59c414f60b2254a23a1966dc0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:09.738135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:09.840627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273af7a59c414f60b2254a23a1966dc0', 'request_id': 'req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:09.848338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273af7a59c414f60b2254a23a1966dc0', 'request_id': 'req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:09.856817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273af7a59c414f60b2254a23a1966dc0', 'request_id': 'req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:09.876076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] The token cache is being invalidate because project 273af7a59c414f60b2254a23a1966dc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:09.877620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:09.878757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7ec2fde-8e5f-485f-b8a0-3a641ff6557f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:09.881326 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1266/2536] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:09 2022] DELETE /identity/v3/projects/273af7a59c414f60b2254a23a1966dc0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:09.886663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2020f3-f3c0-43e5-9fea-9db934e6c81f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:09.887498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2020f3-f3c0-43e5-9fea-9db934e6c81f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:09.887775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2020f3-f3c0-43e5-9fea-9db934e6c81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:09.955031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3a2020f3-f3c0-43e5-9fea-9db934e6c81f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:09.956661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3a2020f3-f3c0-43e5-9fea-9db934e6c81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:10.076813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1271/2537] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:10.083630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efb28d-12c6-4af8-8f09-d3f4680421e8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.084270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efb28d-12c6-4af8-8f09-d3f4680421e8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.084828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efb28d-12c6-4af8-8f09-d3f4680421e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.121455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-91efb28d-12c6-4af8-8f09-d3f4680421e8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:10.122730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91efb28d-12c6-4af8-8f09-d3f4680421e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:10.133066 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1267/2538] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:10.137209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2020f3-f3c0-43e5-9fea-9db934e6c81f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.148813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2020f3-f3c0-43e5-9fea-9db934e6c81f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:10.154583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:10.155479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.155763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.156259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.157314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:10.157314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:10.159911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:10.176121 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1272/2539] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:10.180445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efb28d-12c6-4af8-8f09-d3f4680421e8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.194125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efb28d-12c6-4af8-8f09-d3f4680421e8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:10.224781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:10.225422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.225698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.226000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.227204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:10.230517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:10.319061 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1268/2540] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:10 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:10.323203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.342389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03dcdd-5fbe-4811-ab11-7a871c6d622f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:10.373290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:10.429195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:10.429813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.430120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.430393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.431302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:10.433493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:10.459847 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1273/2541] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:10 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:10.464254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.475246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43178259-9ace-419e-9ae0-19ab90f6b370 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:10.480877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:10.481547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.481865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.482138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.482951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:10.484547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:10.497119 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1269/2542] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:10.500747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.512578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262cf312-b62f-4b52-b906-c55bf25fb7e8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:10.525046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11af9b-0c5b-4cdd-a173-848b3d42dc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:10.526014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11af9b-0c5b-4cdd-a173-848b3d42dc22 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.526467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11af9b-0c5b-4cdd-a173-848b3d42dc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.526895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11af9b-0c5b-4cdd-a173-848b3d42dc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.528689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11af9b-0c5b-4cdd-a173-848b3d42dc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:10.530553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11af9b-0c5b-4cdd-a173-848b3d42dc22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:10.549754 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1274/2543] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:10.554618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.565873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6479d-07bd-4d58-b377-1dd9a2403be4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:10.570829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:10.571486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.571762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.572035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] PATH_INFO: `/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc/users/7f5fc6ed15af434998a9a3aed3d0e1ca/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.590586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef419dcd60a4af7ba7d02cb93631efc, user_id=7f5fc6ed15af434998a9a3aed3d0e1ca, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:10.592788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:10.618711 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1270/2544] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:10 2022] PUT /identity/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc/users/7f5fc6ed15af434998a9a3aed3d0e1ca/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:10.630880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603f76d-d009-4fc6-9c95-bdd63b6d2328 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.631172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603f76d-d009-4fc6-9c95-bdd63b6d2328 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.631450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603f76d-d009-4fc6-9c95-bdd63b6d2328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.668985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8603f76d-d009-4fc6-9c95-bdd63b6d2328 None None] MFA Rules not processed for user `7f5fc6ed15af434998a9a3aed3d0e1ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:10.670578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8603f76d-d009-4fc6-9c95-bdd63b6d2328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:10.801035 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1275/2545] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:10.801400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:10.894795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:10.894795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:10.894795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:10.936029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:10.938709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:10.993630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e28c9c-f632-479e-9e8a-2367ead70747 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:11.011051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:11.012045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.012389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.013324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.014133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:11.019275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:11.063722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:11.093733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1271/2546] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:10 2022] POST /identity/v3/projects => generated 379 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:11.095985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:11.099114 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1276/2547] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:11.127661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:11.135458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:11.237358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:11.293920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62e155-4b7c-4e43-a9d2-16608dc014bb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:11.312980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506d7c6-6d10-4d9b-bbf4-5b6572506a28 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:11.321269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:11.322223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.322652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.323071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.323844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:11.326117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:11.372596 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1272/2548] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:11 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:11.375960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:11.398341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fe940-9443-4784-8951-65623b1af470 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:11.404898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22140280-af35-41de-9010-aee3865710ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:11.405348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140280-af35-41de-9010-aee3865710ef admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.405666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140280-af35-41de-9010-aee3865710ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.405981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140280-af35-41de-9010-aee3865710ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.406628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22140280-af35-41de-9010-aee3865710ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:11.408865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22140280-af35-41de-9010-aee3865710ef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:11.420951 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1273/2549] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:11.425330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22140280-af35-41de-9010-aee3865710ef admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:11.436202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:11.436548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22140280-af35-41de-9010-aee3865710ef admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:11.439451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.440700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.441222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.446746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:11.447429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.447715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.448349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] PATH_INFO: `/v3/projects/906db44e7e4e4ebaa96aa9452bc20577/users/bc297f8e5cce4defbe3301e5035e62a1/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.468387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] RBAC: Authorizing `identity:create_grant(project_id=906db44e7e4e4ebaa96aa9452bc20577, user_id=bc297f8e5cce4defbe3301e5035e62a1, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:11.472379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:11.495438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1274/2550] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:11 2022] PUT /identity/v3/projects/906db44e7e4e4ebaa96aa9452bc20577/users/bc297f8e5cce4defbe3301e5035e62a1/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:11.499746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:11.538821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:11.544206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:11.560160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1277/2551] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:52:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:52:11.627193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707cf8b-e5bc-4303-9607-76ea8fff241c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:11.645053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8695ae-cf70-49dd-8203-38ce7b983056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:11.645674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8695ae-cf70-49dd-8203-38ce7b983056 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.645983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8695ae-cf70-49dd-8203-38ce7b983056 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.646603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8695ae-cf70-49dd-8203-38ce7b983056 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.646906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8695ae-cf70-49dd-8203-38ce7b983056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:11.648564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8695ae-cf70-49dd-8203-38ce7b983056 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:11.680317 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1275/2552] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:52:11.681081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:11.717390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcfe3c-ceb0-4960-92cf-6bd6b131e75f service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:11.755943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:11.761610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.761610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.763556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] PATH_INFO: `/v3/projects/906db44e7e4e4ebaa96aa9452bc20577/users/bc297f8e5cce4defbe3301e5035e62a1/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.776812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=906db44e7e4e4ebaa96aa9452bc20577, user_id=bc297f8e5cce4defbe3301e5035e62a1, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:11.794324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:11.823705 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1278/2553] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:11 2022] PUT /identity/v3/projects/906db44e7e4e4ebaa96aa9452bc20577/users/bc297f8e5cce4defbe3301e5035e62a1/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:11.835204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901ac98-1408-479b-9b74-16e8c80fe47f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:11.835831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901ac98-1408-479b-9b74-16e8c80fe47f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:11.835831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901ac98-1408-479b-9b74-16e8c80fe47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:11.878843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7901ac98-1408-479b-9b74-16e8c80fe47f None None] MFA Rules not processed for user `bc297f8e5cce4defbe3301e5035e62a1`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:11.881050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7901ac98-1408-479b-9b74-16e8c80fe47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:11.954519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:12.042618 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1276/2554] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:12.441386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:12.446489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:12.591725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2bda1-d2b0-4874-a344-757e42ba6e49 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:12.600043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:12.600929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:12.601344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:12.601700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:12.607727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:12.616020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:12.620897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:12.636098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1279/2555] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:13.041253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f951d-4a85-4008-8180-190e780fcd6b None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:13.041847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f951d-4a85-4008-8180-190e780fcd6b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:13.041847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f951d-4a85-4008-8180-190e780fcd6b None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:13.043578 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1277/2556] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:13.048785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:52:13.061408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:13.072969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3812bc-1599-448f-a863-7f24243c55c3 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:13.077949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de387e22-67fc-4ac4-8be2-8154ede93d7a tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc297f8e5cce4defbe3301e5035e62a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906db44e7e4e4ebaa96aa9452bc20577', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:13.078328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de387e22-67fc-4ac4-8be2-8154ede93d7a tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:13.078608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de387e22-67fc-4ac4-8be2-8154ede93d7a tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:13.078882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de387e22-67fc-4ac4-8be2-8154ede93d7a tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] PATH_INFO: `/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:13.080845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de387e22-67fc-4ac4-8be2-8154ede93d7a tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] RBAC: Authorizing `identity:get_project(project_id=2ef419dcd60a4af7ba7d02cb93631efc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:13.083358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de387e22-67fc-4ac4-8be2-8154ede93d7a tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:13.085038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1280/2557] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:52:13 2022] GET /identity/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc => generated 377 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:13.447677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f951d-4a85-4008-8180-190e780fcd6b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:13.468374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f951d-4a85-4008-8180-190e780fcd6b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:13.476974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:13.477738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:13.478070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:13.478406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:13.552675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:13.554474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:13.565252 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1278/2558] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:14.242569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82604c5-0368-4e57-acc0-6c84ff9645be None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:14.243117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82604c5-0368-4e57-acc0-6c84ff9645be None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:14.243117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82604c5-0368-4e57-acc0-6c84ff9645be None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:14.244696 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1281/2559] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:52:14.249598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:52:14.261549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:14.272817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e501778-ab14-47bd-b9bd-6a8a69925926 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:14.277932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d75f3-fb20-43e2-bd69-739a495efb68 tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc297f8e5cce4defbe3301e5035e62a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '906db44e7e4e4ebaa96aa9452bc20577', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:14.278596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d75f3-fb20-43e2-bd69-739a495efb68 tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:14.278883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d75f3-fb20-43e2-bd69-739a495efb68 tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:14.279161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d75f3-fb20-43e2-bd69-739a495efb68 tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] PATH_INFO: `/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:14.280293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d75f3-fb20-43e2-bd69-739a495efb68 tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] RBAC: Authorizing `identity:get_project(project_id=2ef419dcd60a4af7ba7d02cb93631efc)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:14.282404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d75f3-fb20-43e2-bd69-739a495efb68 tempest-FlavorsAccessTestJSON-1861483752 tempest-FlavorsAccessTestJSON-1861483752-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:14.284682 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1279/2560] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:52:14 2022] GET /identity/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc => generated 377 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:15.341211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82604c5-0368-4e57-acc0-6c84ff9645be None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:15.441016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82604c5-0368-4e57-acc0-6c84ff9645be None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:15.451896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:15.453274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:15.453750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:15.454245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] PATH_INFO: `/v3/users/7f5fc6ed15af434998a9a3aed3d0e1ca` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:15.467107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5fc6ed15af434998a9a3aed3d0e1ca)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:15.470795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:15.566212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5fc6ed15af434998a9a3aed3d0e1ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:15.585430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5fc6ed15af434998a9a3aed3d0e1ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:15.590290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5fc6ed15af434998a9a3aed3d0e1ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:15.599656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5fc6ed15af434998a9a3aed3d0e1ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:15.600457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5fc6ed15af434998a9a3aed3d0e1ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:15.627972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:15.628539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:15.628539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:15.633802 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1282/2561] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:15 2022] DELETE /identity/v3/users/7f5fc6ed15af434998a9a3aed3d0e1ca => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:15.655512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:15.656762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:15.804976 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1280/2562] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:15.817729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:15.819901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:15.969434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824c6d0-fc5c-47a3-b5ca-0706f23c7d77 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:15.990555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:15.991509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:15.991957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:15.992392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:16.020756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:16.025165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:16.074807 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1283/2563] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:16.657332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:16.659266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:16.678864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b7949-a3fc-4095-a242-5138bef93952 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:16.685123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:16.686217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:16.686688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:16.687039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] PATH_INFO: `/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:16.688552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef419dcd60a4af7ba7d02cb93631efc)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:16.692163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:16.784305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef419dcd60a4af7ba7d02cb93631efc', 'request_id': 'req-c509758b-a4c9-4154-a2f4-bdafe905bf1b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:16.790652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef419dcd60a4af7ba7d02cb93631efc', 'request_id': 'req-c509758b-a4c9-4154-a2f4-bdafe905bf1b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:16.791876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:16.799388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef419dcd60a4af7ba7d02cb93631efc', 'request_id': 'req-c509758b-a4c9-4154-a2f4-bdafe905bf1b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:16.814008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] The token cache is being invalidate because project 2ef419dcd60a4af7ba7d02cb93631efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:16.821153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:16.822053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:16.823889 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1281/2564] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:16 2022] DELETE /identity/v3/projects/2ef419dcd60a4af7ba7d02cb93631efc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:16.828121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:16.838759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070b90a-3190-4f9b-b062-a84fc17c564e service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:16.842712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:16.934835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:16.935955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:16.936240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:16.936565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] PATH_INFO: `/v3/users/bc297f8e5cce4defbe3301e5035e62a1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:16.937739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc297f8e5cce4defbe3301e5035e62a1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:16.940685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:17.038474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc297f8e5cce4defbe3301e5035e62a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.054361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc297f8e5cce4defbe3301e5035e62a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.055109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc297f8e5cce4defbe3301e5035e62a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.071899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc297f8e5cce4defbe3301e5035e62a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.072682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-982f1b67-4327-4bd0-b16f-ea7eeb9f9b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc297f8e5cce4defbe3301e5035e62a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.124759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1284/2565] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:16 2022] DELETE /identity/v3/users/bc297f8e5cce4defbe3301e5035e62a1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:17.622389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:17.628147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:17.663752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509758b-a4c9-4154-a2f4-bdafe905bf1b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:17.765128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:17.765817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:17.766155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:17.766487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] PATH_INFO: `/v3/projects/906db44e7e4e4ebaa96aa9452bc20577` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:17.767733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=906db44e7e4e4ebaa96aa9452bc20577)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:17.770263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:17.874919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '906db44e7e4e4ebaa96aa9452bc20577', 'request_id': 'req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.881266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '906db44e7e4e4ebaa96aa9452bc20577', 'request_id': 'req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.888967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '906db44e7e4e4ebaa96aa9452bc20577', 'request_id': 'req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.923274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] The token cache is being invalidate because project 906db44e7e4e4ebaa96aa9452bc20577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:17.924216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.925965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3da69ae5-e9f3-46f5-b2a3-8c2d589cab6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:17.947232 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1282/2566] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:17 2022] DELETE /identity/v3/projects/906db44e7e4e4ebaa96aa9452bc20577 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:17.953450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da415-cf1b-4ac5-870a-aa3bbd8fc7e5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:17.954074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da415-cf1b-4ac5-870a-aa3bbd8fc7e5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:17.954266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645da415-cf1b-4ac5-870a-aa3bbd8fc7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.010618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-645da415-cf1b-4ac5-870a-aa3bbd8fc7e5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:18.011963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-645da415-cf1b-4ac5-870a-aa3bbd8fc7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:18.125140 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1285/2567] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:18.133903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972e7b6-766a-4017-90d7-34151f9ec8ed None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.134527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972e7b6-766a-4017-90d7-34151f9ec8ed None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.135059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972e7b6-766a-4017-90d7-34151f9ec8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.182685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0972e7b6-766a-4017-90d7-34151f9ec8ed None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:18.184253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0972e7b6-766a-4017-90d7-34151f9ec8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:18.194889 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1283/2568] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:18.199008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645da415-cf1b-4ac5-870a-aa3bbd8fc7e5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:18.226800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645da415-cf1b-4ac5-870a-aa3bbd8fc7e5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:18.234756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:18.236636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.237257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.237922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.238997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:18.239526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:18.241410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:18.256586 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1286/2569] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:18.260716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972e7b6-766a-4017-90d7-34151f9ec8ed None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:18.275895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972e7b6-766a-4017-90d7-34151f9ec8ed None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:18.294405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:18.296122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.296412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.296713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.297361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:18.301746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:18.364315 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1284/2570] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:18 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:18.366307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:18.384012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b1b2-c252-4c47-a59a-4c9379bc770e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:18.404229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:18.534822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:18.535852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.536356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.536848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.537717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:18.543274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:18.591187 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1287/2571] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:18 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:18.596308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:18.619298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d75e3b-c5cb-460b-ba55-e0cd0b56f7b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:18.634076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:18.635429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.635709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.636072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.638723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:18.640567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:18.654531 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1285/2572] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:18.668730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:18.681792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20d826-9758-433e-a587-fdb8a63f4d16 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:18.700018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a173-5c3b-4769-883a-c790d86aef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:18.700914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a173-5c3b-4769-883a-c790d86aef80 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.701440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a173-5c3b-4769-883a-c790d86aef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.702351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a173-5c3b-4769-883a-c790d86aef80 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.702607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a173-5c3b-4769-883a-c790d86aef80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:18.704468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a173-5c3b-4769-883a-c790d86aef80 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:18.728196 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1288/2573] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:18.732140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:18.745812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aec10-7368-4984-bab3-658152c916d6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:18.762574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:18.763341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.763626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.763905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] PATH_INFO: `/v3/projects/845366ec964b4b5a9d253f34b3a276d7/users/aec94c541da74742aee050f13cf37049/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.789879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=845366ec964b4b5a9d253f34b3a276d7, user_id=aec94c541da74742aee050f13cf37049, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:18.792558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:18.816150 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1286/2574] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:18 2022] PUT /identity/v3/projects/845366ec964b4b5a9d253f34b3a276d7/users/aec94c541da74742aee050f13cf37049/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:18.844216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103a6d6-d1bd-479c-9748-352146fc5de3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:18.844770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103a6d6-d1bd-479c-9748-352146fc5de3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:18.845272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103a6d6-d1bd-479c-9748-352146fc5de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:18.942769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8103a6d6-d1bd-479c-9748-352146fc5de3 None None] MFA Rules not processed for user `aec94c541da74742aee050f13cf37049`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:18.944192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8103a6d6-d1bd-479c-9748-352146fc5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:19.046221 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1289/2575] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:19.051228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:19.094399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:19.220777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646681f-e967-4445-834a-ea5bd092aec1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:19.232346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:19.233075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:19.233469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:19.233818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:19.243495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:19.245593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:19.313331 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1287/2576] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:19 2022] POST /identity/v3/projects => generated 385 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:19.314637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103a6d6-d1bd-479c-9748-352146fc5de3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:19.328885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103a6d6-d1bd-479c-9748-352146fc5de3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:19.431965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:19.432740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:19.433032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:19.433462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:19.434198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:19.443844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:19.509924 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1290/2577] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:19 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:19.510618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:19.523929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeef80-d81e-4c25-927b-845ecd36ced9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:19.532476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:19.533715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:19.534324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:19.534858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:19.535777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:19.538371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:19.556739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:19.557585 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1288/2578] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:52:19.570925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9036b-209b-4f45-8b2d-0497b5337d70 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:19.580275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:19.581237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:19.581675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:19.582082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] PATH_INFO: `/v3/projects/93c96de03309487c9d4ebcb4fb29837e/users/91017af5cadc4ce5950c40157bb20db4/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:19.608519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:19.629776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c96de03309487c9d4ebcb4fb29837e, user_id=91017af5cadc4ce5950c40157bb20db4, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:19.632674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:19.659674 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1291/2579] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:19 2022] PUT /identity/v3/projects/93c96de03309487c9d4ebcb4fb29837e/users/91017af5cadc4ce5950c40157bb20db4/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:19.660961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:19.675664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67750735-6961-42c3-9300-a406b1abbc55 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:19.783331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405502e7-da02-4cc1-9266-e31a24125cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:19.784870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405502e7-da02-4cc1-9266-e31a24125cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:19.785644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405502e7-da02-4cc1-9266-e31a24125cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:19.786408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405502e7-da02-4cc1-9266-e31a24125cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:19.787734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405502e7-da02-4cc1-9266-e31a24125cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:19.790345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405502e7-da02-4cc1-9266-e31a24125cf6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:19.805939 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1289/2580] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:52:19.807524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:19.821857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c5b74-0ca6-43f0-89ba-edb9ea82b871 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:19.826734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:19.827408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:19.827689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:19.827975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] PATH_INFO: `/v3/projects/93c96de03309487c9d4ebcb4fb29837e/users/91017af5cadc4ce5950c40157bb20db4/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:19.830226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c96de03309487c9d4ebcb4fb29837e, user_id=91017af5cadc4ce5950c40157bb20db4, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:19.834090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:19.847614 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1292/2581] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:19 2022] PUT /identity/v3/projects/93c96de03309487c9d4ebcb4fb29837e/users/91017af5cadc4ce5950c40157bb20db4/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:19.858177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5afc7b-918c-4b4c-9324-d69d0487a257 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:19.858465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5afc7b-918c-4b4c-9324-d69d0487a257 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:19.858742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5afc7b-918c-4b4c-9324-d69d0487a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:19.915214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ae5afc7b-918c-4b4c-9324-d69d0487a257 None None] MFA Rules not processed for user `91017af5cadc4ce5950c40157bb20db4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:19.916681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ae5afc7b-918c-4b4c-9324-d69d0487a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:20.065227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1290/2582] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:20.784012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:20.786060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:20.952098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baac9d9-33db-4be5-a35c-ddc693a9e963 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:20.969039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:20.971731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:20.972293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:20.972671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:20.990143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:20.993603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:21.016280 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1293/2583] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:21.888416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5afc7b-918c-4b4c-9324-d69d0487a257 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:21.905353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:21.960227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5afc7b-918c-4b4c-9324-d69d0487a257 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:21.976438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbbb03-3e0e-480d-8edc-f0cb32c7bd80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:21.976438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbbb03-3e0e-480d-8edc-f0cb32c7bd80 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:21.976438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbbb03-3e0e-480d-8edc-f0cb32c7bd80 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:21.976438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbbb03-3e0e-480d-8edc-f0cb32c7bd80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:22.077020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbbb03-3e0e-480d-8edc-f0cb32c7bd80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:22.079315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbbb03-3e0e-480d-8edc-f0cb32c7bd80 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:22.092310 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1291/2584] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:22.429905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:22.431291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:22.563862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:22.568693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b58c84-8001-40ce-89b8-4ddcff511982 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:22.581687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:22.582974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:22.583245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:22.583592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] PATH_INFO: `/v3/users/cbabc9a296d14628b649da7d81bc8766` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:22.585047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbabc9a296d14628b649da7d81bc8766)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:22.587326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:22.699481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbabc9a296d14628b649da7d81bc8766'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:22.717165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbabc9a296d14628b649da7d81bc8766'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:22.718237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbabc9a296d14628b649da7d81bc8766'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:22.727955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbabc9a296d14628b649da7d81bc8766'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:22.728677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbabc9a296d14628b649da7d81bc8766'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:22.756864 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1294/2585] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:22 2022] DELETE /identity/v3/users/cbabc9a296d14628b649da7d81bc8766 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:22.761813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:22.762550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:22.764473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:22.836208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:22.837559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:22.904525 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1292/2586] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:22.912303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:22.914509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:23.047119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f02dc3-58a4-4ab4-94a6-014a55124c7b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:23.051627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:23.052363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:23.052846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:23.053126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:23.064990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:23.066902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:23.111287 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1295/2587] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:24.113725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:24.116304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:24.132024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715110dd-fc95-476b-b54f-49067b0f58a1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:24.165247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:24.165919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:24.166298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:24.166467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] PATH_INFO: `/v3/projects/5580128bff364c858eafedcc84dad425` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:24.169294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=5580128bff364c858eafedcc84dad425)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:24.171730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:24.280623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5580128bff364c858eafedcc84dad425', 'request_id': 'req-197fad38-e430-46da-ac37-486237ce7dab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.296132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5580128bff364c858eafedcc84dad425', 'request_id': 'req-197fad38-e430-46da-ac37-486237ce7dab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.309820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5580128bff364c858eafedcc84dad425', 'request_id': 'req-197fad38-e430-46da-ac37-486237ce7dab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.317938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:24.339176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] The token cache is being invalidate because project 5580128bff364c858eafedcc84dad425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:24.339488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.340141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.351551 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1293/2588] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:24 2022] DELETE /identity/v3/projects/5580128bff364c858eafedcc84dad425 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:24.359704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:24.394307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed672a-31fd-451f-bd7d-e40986eff694 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:24.490505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:24.508393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:24.509879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:24.510025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:24.510262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] PATH_INFO: `/v3/users/61323f1fb63a438fa8d7dbfe6caeb8ea` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:24.511660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61323f1fb63a438fa8d7dbfe6caeb8ea)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:24.515412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:24.554329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:24.556113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:24.572122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fad38-e430-46da-ac37-486237ce7dab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:24.578550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:24.580405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:24.581038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:24.581523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] PATH_INFO: `/v3/users/43b9b4d83834456388c99abd3ecae3c7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:24.583158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b9b4d83834456388c99abd3ecae3c7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:24.585486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:24.618368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61323f1fb63a438fa8d7dbfe6caeb8ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.629243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61323f1fb63a438fa8d7dbfe6caeb8ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.630193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61323f1fb63a438fa8d7dbfe6caeb8ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.639912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61323f1fb63a438fa8d7dbfe6caeb8ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.640600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61323f1fb63a438fa8d7dbfe6caeb8ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.668364 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1296/2589] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:24 2022] DELETE /identity/v3/users/61323f1fb63a438fa8d7dbfe6caeb8ea => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:24.697190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b9b4d83834456388c99abd3ecae3c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.731494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b9b4d83834456388c99abd3ecae3c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.747944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b9b4d83834456388c99abd3ecae3c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.766484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b9b4d83834456388c99abd3ecae3c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.767256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b9b4d83834456388c99abd3ecae3c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:24.777358 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1294/2590] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:24 2022] DELETE /identity/v3/users/43b9b4d83834456388c99abd3ecae3c7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:25.101252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.103223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.209319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.211662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.223546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8192cb2-bb34-4ef2-b067-778f4d56d150 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:25.311852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37394255-c761-442c-85b3-a2391c38a7ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:25.318144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:25.318785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.319066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.319514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] PATH_INFO: `/v3/projects/8693bafb9e424052ab91e56e9e4da6e7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.320400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:25.321108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.321243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8693bafb9e424052ab91e56e9e4da6e7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:25.321392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.321790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] PATH_INFO: `/v3/projects/3bc41ed54a5745d8b6f127e3ac482da1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.323156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:25.323290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc41ed54a5745d8b6f127e3ac482da1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:25.325379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:25.352053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:25.374979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8693bafb9e424052ab91e56e9e4da6e7', 'request_id': 'req-02d6109c-dcfb-4d29-9770-fe2914d80df6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.381537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8693bafb9e424052ab91e56e9e4da6e7', 'request_id': 'req-02d6109c-dcfb-4d29-9770-fe2914d80df6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.387188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc41ed54a5745d8b6f127e3ac482da1', 'request_id': 'req-87deb2b1-4d14-432e-8193-7e66c9769751'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.388164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8693bafb9e424052ab91e56e9e4da6e7', 'request_id': 'req-02d6109c-dcfb-4d29-9770-fe2914d80df6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.394717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:25.395622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc41ed54a5745d8b6f127e3ac482da1', 'request_id': 'req-87deb2b1-4d14-432e-8193-7e66c9769751'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.406346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc41ed54a5745d8b6f127e3ac482da1', 'request_id': 'req-87deb2b1-4d14-432e-8193-7e66c9769751'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.409781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] The token cache is being invalidate because project 8693bafb9e424052ab91e56e9e4da6e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:25.410078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.410870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.416516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1295/2591] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:25 2022] DELETE /identity/v3/projects/8693bafb9e424052ab91e56e9e4da6e7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:25.416516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.416516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.424906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] The token cache is being invalidate because project 3bc41ed54a5745d8b6f127e3ac482da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:25.425200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.426059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87deb2b1-4d14-432e-8193-7e66c9769751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.427885 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1297/2592] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:25 2022] DELETE /identity/v3/projects/3bc41ed54a5745d8b6f127e3ac482da1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:25.429369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7a73-53f4-40a0-a538-0340b7614363 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.429637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7a73-53f4-40a0-a538-0340b7614363 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.429981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7a73-53f4-40a0-a538-0340b7614363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.468870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-913c7a73-53f4-40a0-a538-0340b7614363 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:25.470758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-913c7a73-53f4-40a0-a538-0340b7614363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.507949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6109c-dcfb-4d29-9770-fe2914d80df6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:25.547949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1298/2593] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:25.547949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15b6d-2904-4cc0-b8ab-d01a38b07dd1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.547949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15b6d-2904-4cc0-b8ab-d01a38b07dd1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.547949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c15b6d-2904-4cc0-b8ab-d01a38b07dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.567654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b0c15b6d-2904-4cc0-b8ab-d01a38b07dd1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:25.568872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b0c15b6d-2904-4cc0-b8ab-d01a38b07dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.574688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:25.575345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.575641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.576044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] PATH_INFO: `/v3/users/aec94c541da74742aee050f13cf37049` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.577069 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1299/2594] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:25.577185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec94c541da74742aee050f13cf37049)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:25.578297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529af9cf-fcb0-4298-9cc9-9e5caf9dac78 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.578696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529af9cf-fcb0-4298-9cc9-9e5caf9dac78 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.578977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529af9cf-fcb0-4298-9cc9-9e5caf9dac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.580729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:25.601673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-529af9cf-fcb0-4298-9cc9-9e5caf9dac78 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:25.602996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-529af9cf-fcb0-4298-9cc9-9e5caf9dac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.612748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1300/2595] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:25.614349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e512b5-7986-4aaf-b4a7-dd01fc5593ff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.614786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e512b5-7986-4aaf-b4a7-dd01fc5593ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.615099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e512b5-7986-4aaf-b4a7-dd01fc5593ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.637087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-04e512b5-7986-4aaf-b4a7-dd01fc5593ff None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:25.637500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec94c541da74742aee050f13cf37049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.638521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-04e512b5-7986-4aaf-b4a7-dd01fc5593ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.650031 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1301/2596] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:25.650996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e512b5-7986-4aaf-b4a7-dd01fc5593ff None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.656209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec94c541da74742aee050f13cf37049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.662589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e512b5-7986-4aaf-b4a7-dd01fc5593ff None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:25.665908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec94c541da74742aee050f13cf37049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.667435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d71f2a-cc54-4733-917c-10a2e291331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:25.668137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d71f2a-cc54-4733-917c-10a2e291331b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.668515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d71f2a-cc54-4733-917c-10a2e291331b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.668886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d71f2a-cc54-4733-917c-10a2e291331b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.669630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d71f2a-cc54-4733-917c-10a2e291331b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:25.670063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d71f2a-cc54-4733-917c-10a2e291331b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:25.671767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d71f2a-cc54-4733-917c-10a2e291331b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:25.675165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec94c541da74742aee050f13cf37049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.675632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec94c541da74742aee050f13cf37049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:25.683620 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1296/2597] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:25 2022] DELETE /identity/v3/users/aec94c541da74742aee050f13cf37049 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:25.684979 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1302/2598] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:25.687389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.689903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.690255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.690650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.691440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.711705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:25.713071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.731612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:25.794917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b70c97-2179-403b-bf0d-867b8144f5e5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:25.804256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:25.804919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.805207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.806176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.808148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:25.808538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:25.810279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:25.811609 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1303/2599] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:25.815648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.823324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.825407 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1297/2600] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:25.834405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.836738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:25.838414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fe81d-e6fc-4ebc-85a7-aec3bbda031e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:25.843983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:25.844591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:25.844879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:25.845207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:25.846057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:25.848240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:25.853546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:25.913414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1304/2601] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:25 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:25.914521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:25.922159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5563f7-95cf-4dbd-8afc-3b0cb7d28c2e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.014159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.015007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.016184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.016646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.049404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a922508-73cd-4169-8e80-71e08e2ea267 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.055127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.055734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.056076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.056338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.056985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.059014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.113283 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1305/2602] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:25 2022] POST /identity/v3/projects => generated 391 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:26.114319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.134700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.136413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.148836 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1298/2603] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:26.150036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.162676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f99df-b3f4-482d-8be2-0f79adc4d6c0 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.190805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:26.222537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b743d-c02a-4e75-a61a-8734da319346 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.231716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.232429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.232709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.233411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.234393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.238874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.260671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.262164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.262836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.263695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.264663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.268849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.294798 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1306/2604] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:26 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:26.296953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.302867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1299/2605] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:26 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:26.308400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.319445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b495bb6-400b-4da2-9d82-562581c43f1e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.325948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345592de-bcab-4916-87ef-1e7986f3d24e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.326254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.327444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.327939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.328403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.329109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.331119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.332500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.333297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.333443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.333661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.334337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.335943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.347174 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1307/2606] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:26.348088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.350842 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1300/2607] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:26.351746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.365037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539b1e-77a8-46fb-98a9-3bd38ce99bd5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.366058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab4ade-de97-472d-bf8b-90062553a62d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.369916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.370524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.370810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.371090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.371736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.372710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.373453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.373590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.374320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.374677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.375493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.379330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.385858 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1301/2608] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:26.390128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.395253 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1308/2609] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:26.396364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.404282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0cd83-0c6e-45f5-98ea-da76aa8e006c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.406381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c41f3-8328-4957-a9d9-74d03370afe4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.412216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b13ee-63aa-41c1-ab43-5cc7775eb6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.413034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b13ee-63aa-41c1-ab43-5cc7775eb6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.413328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed9e71-3ae4-4729-9d82-ae1906fc8ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.413504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b13ee-63aa-41c1-ab43-5cc7775eb6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.414059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b13ee-63aa-41c1-ab43-5cc7775eb6d5 admin admin] PATH_INFO: `/v3/projects/0b0053a439224236892d617e65718f09/users/2c92dafad95d466fbbc6f38608fa0c1d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.414542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9e71-3ae4-4729-9d82-ae1906fc8ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.414921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9e71-3ae4-4729-9d82-ae1906fc8ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.415289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9e71-3ae4-4729-9d82-ae1906fc8ca2 admin admin] PATH_INFO: `/v3/projects/b0e594afa9b14d5180bfc2b46afada1b/users/45a9194f6cd7487db29d3d928633fb79/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.431058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b13ee-63aa-41c1-ab43-5cc7775eb6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0053a439224236892d617e65718f09, user_id=2c92dafad95d466fbbc6f38608fa0c1d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.434504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b13ee-63aa-41c1-ab43-5cc7775eb6d5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.436563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed9e71-3ae4-4729-9d82-ae1906fc8ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e594afa9b14d5180bfc2b46afada1b, user_id=45a9194f6cd7487db29d3d928633fb79, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.440717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed9e71-3ae4-4729-9d82-ae1906fc8ca2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.451266 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1302/2610] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:26 2022] PUT /identity/v3/projects/0b0053a439224236892d617e65718f09/users/2c92dafad95d466fbbc6f38608fa0c1d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:26.457298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1309/2611] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:26 2022] PUT /identity/v3/projects/b0e594afa9b14d5180bfc2b46afada1b/users/45a9194f6cd7487db29d3d928633fb79/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:26.463601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.463886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.464586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.468488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8469f2f-f74b-49c2-8b3d-e73fd5078f0d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.468879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8469f2f-f74b-49c2-8b3d-e73fd5078f0d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.469257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8469f2f-f74b-49c2-8b3d-e73fd5078f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.502025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] MFA Rules not processed for user `2c92dafad95d466fbbc6f38608fa0c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:26.504461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:26.506604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:26.511535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d8469f2f-f74b-49c2-8b3d-e73fd5078f0d None None] MFA Rules not processed for user `45a9194f6cd7487db29d3d928633fb79`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:26.514138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d8469f2f-f74b-49c2-8b3d-e73fd5078f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:26.598594 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1310/2612] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:26.602263 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1303/2613] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:26.606523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8469f2f-f74b-49c2-8b3d-e73fd5078f0d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.612697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.614953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:26.678630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8469f2f-f74b-49c2-8b3d-e73fd5078f0d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.683282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.683888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.684291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.684571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.684878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:26.685207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.687584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.707471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3db80-19ad-4de6-8c4d-8c895be7f517 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.713857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.714545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.714833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.715112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.716035 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1311/2614] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:26 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:26.717012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.719110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:26.727445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.729097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.738989 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1304/2615] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:26.739767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.750069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78576b3-ebf2-4096-8060-f3daf3c8c958 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.801974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45bce-f57f-433e-8035-3e77bcfc8433 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.806721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.807352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.807637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.807910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] PATH_INFO: `/v3/projects/845366ec964b4b5a9d253f34b3a276d7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.809107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=845366ec964b4b5a9d253f34b3a276d7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.811028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.822520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.823260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.823527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.823805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.824429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.826386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.850580 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1305/2616] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:26 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:26.854960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.866281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:26.873269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9772f8d-ad67-4f14-86eb-6c350e4c86d5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.883500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '845366ec964b4b5a9d253f34b3a276d7', 'request_id': 'req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:26.891034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '845366ec964b4b5a9d253f34b3a276d7', 'request_id': 'req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:26.899845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '845366ec964b4b5a9d253f34b3a276d7', 'request_id': 'req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:26.913019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] The token cache is being invalidate because project 845366ec964b4b5a9d253f34b3a276d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:26.913322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:26.914371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:26.917870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1312/2617] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:26 2022] DELETE /identity/v3/projects/845366ec964b4b5a9d253f34b3a276d7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:26.920243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.922699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:26.948031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.948957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.949270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.949554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:26.950379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:26.952535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:26.967075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1306/2618] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:26.972656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:26.974604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:26.988909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d67f8-5b3f-4443-b941-5dbbdb380fbb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:26.995725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:26.996404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:26.996684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:26.997237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] PATH_INFO: `/v3/projects/0a8671221cb54eb3a2a4247f6318d2f2/users/d39dd5a23b89458c966ba9c93a3869a3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.014699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8671221cb54eb3a2a4247f6318d2f2, user_id=d39dd5a23b89458c966ba9c93a3869a3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:27.014951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e0dee-ace2-4326-a6e3-07f85351fdb0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:27.018358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:27.021036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:27.022177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.022654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.023218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] PATH_INFO: `/v3/users/91017af5cadc4ce5950c40157bb20db4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.024560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91017af5cadc4ce5950c40157bb20db4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:27.027930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:27.041555 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1307/2619] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:26 2022] PUT /identity/v3/projects/0a8671221cb54eb3a2a4247f6318d2f2/users/d39dd5a23b89458c966ba9c93a3869a3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:27.050970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:27.098683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91017af5cadc4ce5950c40157bb20db4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.109103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91017af5cadc4ce5950c40157bb20db4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.109870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91017af5cadc4ce5950c40157bb20db4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.121512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91017af5cadc4ce5950c40157bb20db4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.122218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91017af5cadc4ce5950c40157bb20db4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.131503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:27.146937 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1313/2620] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:26 2022] DELETE /identity/v3/users/91017af5cadc4ce5950c40157bb20db4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:27.155363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540c108-4902-4ea8-b36d-e2637b93c17e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:27.227059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fe619-a471-4b55-86a3-602e171c462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:27.227671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fe619-a471-4b55-86a3-602e171c462e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.228023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fe619-a471-4b55-86a3-602e171c462e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.228215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fe619-a471-4b55-86a3-602e171c462e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.229287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fe619-a471-4b55-86a3-602e171c462e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:27.231643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fe619-a471-4b55-86a3-602e171c462e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:27.246338 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1308/2621] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:27.250288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:27.252559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.275602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc75e5-b760-4c38-a565-e227fde85c7e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:27.291759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:27.297341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.298672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.299642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] PATH_INFO: `/v3/projects/0a8671221cb54eb3a2a4247f6318d2f2/users/d39dd5a23b89458c966ba9c93a3869a3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.305246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8671221cb54eb3a2a4247f6318d2f2, user_id=d39dd5a23b89458c966ba9c93a3869a3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:27.308535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:27.333011 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1314/2622] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:27 2022] PUT /identity/v3/projects/0a8671221cb54eb3a2a4247f6318d2f2/users/d39dd5a23b89458c966ba9c93a3869a3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:27.344148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.344705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.344802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.395705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] MFA Rules not processed for user `d39dd5a23b89458c966ba9c93a3869a3`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:27.403259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.553988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1309/2623] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:27.571103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:27.572865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.605866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:27.607443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.631677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:27.694247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc9d25-5055-4886-8061-0b3d5d9a8ee2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:27.699799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:27.700433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.700851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.701192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] PATH_INFO: `/v3/projects/93c96de03309487c9d4ebcb4fb29837e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.702431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c96de03309487c9d4ebcb4fb29837e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:27.705292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:27.707139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77403c-42f4-48f5-8c5c-53d67c543ade None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:27.722164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:27.723296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.723591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.723863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.730815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:27.733632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:27.736715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:27.748157 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1310/2624] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:27.775681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c96de03309487c9d4ebcb4fb29837e', 'request_id': 'req-f6506397-e3e4-4475-a7e7-a10b67f2c52e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.790312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c96de03309487c9d4ebcb4fb29837e', 'request_id': 'req-f6506397-e3e4-4475-a7e7-a10b67f2c52e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.820292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c96de03309487c9d4ebcb4fb29837e', 'request_id': 'req-f6506397-e3e4-4475-a7e7-a10b67f2c52e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.825139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:27.827040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.837217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] The token cache is being invalidate because project 93c96de03309487c9d4ebcb4fb29837e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:27.837594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.838416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f6506397-e3e4-4475-a7e7-a10b67f2c52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:27.842401 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1315/2625] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:27 2022] DELETE /identity/v3/projects/93c96de03309487c9d4ebcb4fb29837e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:27.845333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.845771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.846087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.848802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838b85e-6b02-439e-bf98-3a36778040c9 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:27.871386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:27.873019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.924105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee8b3a-b482-4b2b-a398-f2aee1b00621 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:27.924641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee8b3a-b482-4b2b-a398-f2aee1b00621 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.924925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee8b3a-b482-4b2b-a398-f2aee1b00621 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.925226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee8b3a-b482-4b2b-a398-f2aee1b00621 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.927065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8ee8b3a-b482-4b2b-a398-f2aee1b00621 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.943757 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1316/2626] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:27.944682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:27.945981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:27.958095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747339e0-89c2-4804-9e7a-5374f3dc0862 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:27.968476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:27.969115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:27.969397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:27.969674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] PATH_INFO: `/v3/users/45a9194f6cd7487db29d3d928633fb79` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:27.973901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a9194f6cd7487db29d3d928633fb79)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:27.973901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.000119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee8b3a-b482-4b2b-a398-f2aee1b00621 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.002766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee8b3a-b482-4b2b-a398-f2aee1b00621 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.015049 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1311/2627] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:28.016525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.016869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.017196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.036477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a9194f6cd7487db29d3d928633fb79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.036477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:28.042939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:28.044224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.045586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a9194f6cd7487db29d3d928633fb79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.046368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a9194f6cd7487db29d3d928633fb79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.053294 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1312/2628] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:28.054128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.054788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a9194f6cd7487db29d3d928633fb79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.055348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.055522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a9194f6cd7487db29d3d928633fb79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.067588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3d02-835e-4ff0-a18f-2fc3bd3b0576 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.072320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.072935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.073279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.073484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] PATH_INFO: `/v3/users/2c92dafad95d466fbbc6f38608fa0c1d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.074486 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1317/2629] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:27 2022] DELETE /identity/v3/users/45a9194f6cd7487db29d3d928633fb79 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:28.074932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c92dafad95d466fbbc6f38608fa0c1d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.075284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.076573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.077501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.129019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c92dafad95d466fbbc6f38608fa0c1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.141428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c92dafad95d466fbbc6f38608fa0c1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.144975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf1ae0-90b8-4f8d-8d66-faacc39f526f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.148501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c92dafad95d466fbbc6f38608fa0c1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.149847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edef497-689c-4689-a702-cc04bf7ad10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.150460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef497-689c-4689-a702-cc04bf7ad10d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.150856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef497-689c-4689-a702-cc04bf7ad10d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.151133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef497-689c-4689-a702-cc04bf7ad10d admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.151771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edef497-689c-4689-a702-cc04bf7ad10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:28.152069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edef497-689c-4689-a702-cc04bf7ad10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.153698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edef497-689c-4689-a702-cc04bf7ad10d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.157049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c92dafad95d466fbbc6f38608fa0c1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.158370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e71d08a6-2084-4316-9bf4-8b916eb390ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c92dafad95d466fbbc6f38608fa0c1d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:28.159876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:28.169219 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1313/2630] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:28 2022] DELETE /identity/v3/users/2c92dafad95d466fbbc6f38608fa0c1d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:28.169416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1318/2631] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:28.169521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.169521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.169521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.171963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.172310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.172655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.189405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:28.190658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.193544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:28.194745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.254290 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1314/2632] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:28.255123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.258323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.272729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1319/2633] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:28.272729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.274300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.280452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749038f1-916f-4227-8c22-87b06b9af97f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.286429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.287045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.287320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.287588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.288216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.290122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.327401 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1315/2634] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:28 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:28.327401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.328378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.378840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab9c66-284c-4370-98fb-b56f6b5ae3e9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.411835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:28.429190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3bea8-b2e0-4d68-8099-ba227e842d18 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.435088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.436214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.436338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.436580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.465600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.466341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.466686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.467075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.524292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.527248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.552060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1316/2635] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:28.558826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.570529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343911ca-1818-430d-89c1-acb4b1713c9f service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.574508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.575999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.578324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.579130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.579977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.580255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.581682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.584904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.588664 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1320/2636] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:28.618688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1317/2637] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:28 2022] POST /identity/v3/users => generated 286 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:28.621433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.649090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aacf9b-c23d-405e-a88c-3c3c777edeeb service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.655732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.656651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.657203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.658368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.659276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.661522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.673961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1321/2638] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:28.677947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.688324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3c68b-c61d-45bc-ba8b-7e9ca7c3ca05 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.693452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eb2ce-8fff-4acc-ada3-776053147cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.694173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eb2ce-8fff-4acc-ada3-776053147cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.694457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eb2ce-8fff-4acc-ada3-776053147cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.694836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eb2ce-8fff-4acc-ada3-776053147cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.695545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eb2ce-8fff-4acc-ada3-776053147cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.697208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eb2ce-8fff-4acc-ada3-776053147cd6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.700298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:28.710557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1318/2639] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:28.714655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:28.727276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd53c3-d20c-4a3e-b7e3-4a5076a9e54a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:28.732511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:28.733179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.733468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.733746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] PATH_INFO: `/v3/projects/c76057169d15499ca9b00409f8657818/users/c582bea406ea46498d7f0c175a6d19e6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.750542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76057169d15499ca9b00409f8657818, user_id=c582bea406ea46498d7f0c175a6d19e6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:28.753934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:28.768572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1322/2640] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:28 2022] PUT /identity/v3/projects/c76057169d15499ca9b00409f8657818/users/c582bea406ea46498d7f0c175a6d19e6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:28.780116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0844-cfc5-4d65-8493-77b2f6576a5f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:28.780404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0844-cfc5-4d65-8493-77b2f6576a5f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:28.780675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0844-cfc5-4d65-8493-77b2f6576a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:28.810066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8f9c0844-cfc5-4d65-8493-77b2f6576a5f None None] MFA Rules not processed for user `c582bea406ea46498d7f0c175a6d19e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:28.811281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8f9c0844-cfc5-4d65-8493-77b2f6576a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:28.910532 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1319/2641] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:28.915712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.023330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ca024-04db-4280-ad23-6ff6cf84d6f9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.031649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.032999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.033640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.034114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.034936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.037130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:29.072495 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1323/2642] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:28 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:29.076981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c0844-cfc5-4d65-8493-77b2f6576a5f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.090444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c0844-cfc5-4d65-8493-77b2f6576a5f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.168420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.168776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.169374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.169785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.170609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.173769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:29.205860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1320/2643] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:29 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:29.208339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.221188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09693d0-9d83-4e04-8662-7cbe4ef0f3d3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.228561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.230240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.230524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:29.230882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.231136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.231409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.232051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.233689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:29.256444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8789a7-c7ea-4ba1-9004-cbea24698361 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.260773 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1324/2644] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:29.264721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.274806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.275404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.275760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.275942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] PATH_INFO: `/v3/projects/0b0053a439224236892d617e65718f09` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.278380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b0053a439224236892d617e65718f09)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.279339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cb87e-13c0-4c71-a981-8d0cfadeab5b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.280946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:29.293475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30964a78-5b1c-455d-a240-e986c82e215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.297770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30964a78-5b1c-455d-a240-e986c82e215b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.298744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30964a78-5b1c-455d-a240-e986c82e215b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.299160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30964a78-5b1c-455d-a240-e986c82e215b admin admin] PATH_INFO: `/v3/projects/d1894536ca864c60911b1465e8d6a8a4/users/a6653e696f084be886124bc2ae88eea8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.302586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:29.342202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30964a78-5b1c-455d-a240-e986c82e215b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1894536ca864c60911b1465e8d6a8a4, user_id=a6653e696f084be886124bc2ae88eea8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.354464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30964a78-5b1c-455d-a240-e986c82e215b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:29.367196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b0053a439224236892d617e65718f09', 'request_id': 'req-4215f8cb-70d8-4b2f-957b-d1179d202e50'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.374802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b0053a439224236892d617e65718f09', 'request_id': 'req-4215f8cb-70d8-4b2f-957b-d1179d202e50'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.384556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b0053a439224236892d617e65718f09', 'request_id': 'req-4215f8cb-70d8-4b2f-957b-d1179d202e50'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.397615 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1325/2645] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:29 2022] PUT /identity/v3/projects/d1894536ca864c60911b1465e8d6a8a4/users/a6653e696f084be886124bc2ae88eea8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:29.400813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.400813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.400813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.403223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:29.409407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] The token cache is being invalidate because project 0b0053a439224236892d617e65718f09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:29.409996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.412256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.414959 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1321/2646] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:29 2022] DELETE /identity/v3/projects/0b0053a439224236892d617e65718f09 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:29.416189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.418526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:29.453295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:29.454631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:29.513293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215f8cb-70d8-4b2f-957b-d1179d202e50 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.587641 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1326/2647] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:29.588412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.603562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:29.641879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.642432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.642717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.642994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] PATH_INFO: `/v3/projects/b0e594afa9b14d5180bfc2b46afada1b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.646839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e594afa9b14d5180bfc2b46afada1b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.648047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:29.733931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc79d2-75a2-4bf8-9852-b13f9f50b4f3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.741797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e594afa9b14d5180bfc2b46afada1b', 'request_id': 'req-f6985d24-2bde-43e6-9d56-d3516b93afbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.748983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e594afa9b14d5180bfc2b46afada1b', 'request_id': 'req-f6985d24-2bde-43e6-9d56-d3516b93afbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.757219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e594afa9b14d5180bfc2b46afada1b', 'request_id': 'req-f6985d24-2bde-43e6-9d56-d3516b93afbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.779808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] The token cache is being invalidate because project b0e594afa9b14d5180bfc2b46afada1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:29.780102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.781361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:29.789311 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1322/2648] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:29 2022] DELETE /identity/v3/projects/b0e594afa9b14d5180bfc2b46afada1b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:29.790572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.793708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:29.844405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.845512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.846630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.846927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.847813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.849812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:29.864339 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1327/2649] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:29.865518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:29.868751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:29.894442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:29.944871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:29.956403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6985d24-2bde-43e6-9d56-d3516b93afbb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:29.968415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:29.969071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:29.969356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:29.969633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] PATH_INFO: `/v3/users/d39dd5a23b89458c966ba9c93a3869a3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:29.977077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39dd5a23b89458c966ba9c93a3869a3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:29.977077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.029960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b0154-291d-4746-a10b-9932b1b0927e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.044247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.045258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.045701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.046213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.049800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.054582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39dd5a23b89458c966ba9c93a3869a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:30.075199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39dd5a23b89458c966ba9c93a3869a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:30.084507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39dd5a23b89458c966ba9c93a3869a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:30.099688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39dd5a23b89458c966ba9c93a3869a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:30.100861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ed0527f0-64a5-4269-b007-91f2988a72ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39dd5a23b89458c966ba9c93a3869a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:30.110169 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1323/2650] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:29 2022] DELETE /identity/v3/users/d39dd5a23b89458c966ba9c93a3869a3 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:30.112734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02d1f7-4ff9-4b4d-978b-2e92fc22c662 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.123470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02d1f7-4ff9-4b4d-978b-2e92fc22c662 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.127715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02d1f7-4ff9-4b4d-978b-2e92fc22c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.142670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:30.144632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.159521 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1328/2651] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:52:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:52:30.160468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:30.162475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.181269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52c55-b16d-4a50-bcf3-325475d09eda service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.186549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5e02d1f7-4ff9-4b4d-978b-2e92fc22c662 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:30.190949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5e02d1f7-4ff9-4b4d-978b-2e92fc22c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.292995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e721d-0dca-48c9-b411-b36bcf5cc9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.294760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e721d-0dca-48c9-b411-b36bcf5cc9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.295313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e721d-0dca-48c9-b411-b36bcf5cc9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.297050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e721d-0dca-48c9-b411-b36bcf5cc9a4 admin admin] PATH_INFO: `/v3/projects/d1894536ca864c60911b1465e8d6a8a4/users/a6653e696f084be886124bc2ae88eea8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.301793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e721d-0dca-48c9-b411-b36bcf5cc9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1894536ca864c60911b1465e8d6a8a4, user_id=a6653e696f084be886124bc2ae88eea8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:30.309615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e721d-0dca-48c9-b411-b36bcf5cc9a4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.336582 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1329/2652] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:30 2022] PUT /identity/v3/projects/d1894536ca864c60911b1465e8d6a8a4/users/a6653e696f084be886124bc2ae88eea8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:30.347560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.347560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.347853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.350498 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1324/2653] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:30.352386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8448e-f6ba-4f4c-b72b-33f5a2ea460d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.352682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8448e-f6ba-4f4c-b72b-33f5a2ea460d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.353062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8448e-f6ba-4f4c-b72b-33f5a2ea460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.382452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7bb8448e-f6ba-4f4c-b72b-33f5a2ea460d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:30.386033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7bb8448e-f6ba-4f4c-b72b-33f5a2ea460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.430981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] MFA Rules not processed for user `a6653e696f084be886124bc2ae88eea8`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:30.433934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.505162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:30.527498 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1325/2654] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:30.529315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8448e-f6ba-4f4c-b72b-33f5a2ea460d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:30.562465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:30.563276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8448e-f6ba-4f4c-b72b-33f5a2ea460d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.593857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.596216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.600621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.602257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.603243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:30.605497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:30.605706 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1330/2655] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:30.609063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.620150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:30.623866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:30.636743 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1326/2656] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:30.641295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:30.666769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe2b70-be84-40e7-bf7e-1c4f9dc51b23 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.685391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.689858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.689858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.689858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.689858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:30.692852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.768684 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1327/2657] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:30 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:30.771236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e096f8-8ab0-4916-925b-3cd8581e0bb6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.773367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:30.885127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a046-0efd-4f3b-b123-9a32c502b78e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.898810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.900342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.900917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.901197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.901876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:30.903825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.916667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db603430-6ec7-4f8d-90f9-9fa193b3a3d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.917313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db603430-6ec7-4f8d-90f9-9fa193b3a3d3 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.917686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db603430-6ec7-4f8d-90f9-9fa193b3a3d3 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.918051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db603430-6ec7-4f8d-90f9-9fa193b3a3d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.938330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1328/2658] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:30 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:30.940199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:30.954589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c037-4809-4b42-8d10-5405b5c5c3b2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:30.970856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:30.972019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:30.972374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:30.972795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:30.973756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:30.975867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:30.989857 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1329/2659] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:30.995403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:31.011958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006f305-0532-4ba3-bf0c-17a93570c1b7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:31.020003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db603430-6ec7-4f8d-90f9-9fa193b3a3d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:31.024464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:31.025350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db603430-6ec7-4f8d-90f9-9fa193b3a3d3 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:31.025575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.026045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.026468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.027478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:31.029328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:31.047338 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1330/2660] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:31.052357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:31.067115 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1331/2661] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:31.095485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3c6b6-5889-4912-b144-c354df1023a0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:31.103696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:31.104831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.105282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.105722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] PATH_INFO: `/v3/projects/7a742224229c4aee831699a7c4002d81/users/eb7ecd0dde2241eab1f5a52a6f0ec036/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.151958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a742224229c4aee831699a7c4002d81, user_id=eb7ecd0dde2241eab1f5a52a6f0ec036, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:31.164067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:31.185543 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1331/2662] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:31 2022] PUT /identity/v3/projects/7a742224229c4aee831699a7c4002d81/users/eb7ecd0dde2241eab1f5a52a6f0ec036/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:31.198196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c0785-dee2-447b-8c98-54e9b2cf19eb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.198814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c0785-dee2-447b-8c98-54e9b2cf19eb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.198814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c0785-dee2-447b-8c98-54e9b2cf19eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.272965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:31.276436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:31.302124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:31.313248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-605c0785-dee2-447b-8c98-54e9b2cf19eb None None] MFA Rules not processed for user `eb7ecd0dde2241eab1f5a52a6f0ec036`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:31.315608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-605c0785-dee2-447b-8c98-54e9b2cf19eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:31.448062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5139f-f68a-4fd4-b530-ad5c99c3f2f6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:31.476643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:31.477739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.478052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.478336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] PATH_INFO: `/v3/projects/0a8671221cb54eb3a2a4247f6318d2f2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.484420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8671221cb54eb3a2a4247f6318d2f2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:31.488277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:31.511836 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1332/2663] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:31.512914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:31.569956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8671221cb54eb3a2a4247f6318d2f2', 'request_id': 'req-472ce758-46a3-4882-b0cf-4414a5a1f001'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:31.577697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8671221cb54eb3a2a4247f6318d2f2', 'request_id': 'req-472ce758-46a3-4882-b0cf-4414a5a1f001'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:31.586071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8671221cb54eb3a2a4247f6318d2f2', 'request_id': 'req-472ce758-46a3-4882-b0cf-4414a5a1f001'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:31.607994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] The token cache is being invalidate because project 0a8671221cb54eb3a2a4247f6318d2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:31.608370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:31.609019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-472ce758-46a3-4882-b0cf-4414a5a1f001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:31.612491 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1332/2664] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:31 2022] DELETE /identity/v3/projects/0a8671221cb54eb3a2a4247f6318d2f2 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:31.621138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.621637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.622059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.665987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:31.667729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:31.779201 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1333/2665] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:31.783426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a50e01-91ec-4b56-9e62-aa39ecca49c9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.783817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a50e01-91ec-4b56-9e62-aa39ecca49c9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.784102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a50e01-91ec-4b56-9e62-aa39ecca49c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.801345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:31.809268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:31.824112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-07a50e01-91ec-4b56-9e62-aa39ecca49c9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:31.825344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-07a50e01-91ec-4b56-9e62-aa39ecca49c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:31.850686 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1333/2666] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:31.854756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a50e01-91ec-4b56-9e62-aa39ecca49c9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:31.867607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a50e01-91ec-4b56-9e62-aa39ecca49c9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:31.879530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:31.880147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.880429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.880703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.881578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:31.881910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:31.884165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:31.910282 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1334/2667] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:52:31.914306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:31.936318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa955b3-8901-49cc-b675-f5e77baf426d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:31.943134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b368bd7-81e7-4d5c-b18a-8d178e032f3e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:31.947931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:31.948255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.948609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.949442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.950864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:31.953318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:31.961603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:31.962346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:31.962687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:31.963020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:31.968609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:32.016143 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1335/2668] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:31 2022] POST /identity/v3/projects => generated 395 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:32.021201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.085905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:32.154807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.155900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4dadc-04e6-43f6-ab0f-14d25b1718f7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:32.156443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.162816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:32.163565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.163916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.164257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.164957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.167103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.192938 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1334/2669] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:32.225362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1336/2670] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:32 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:32.228726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.246665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0ea15-909c-4679-a7cf-4c8e86ac377b service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:32.256947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:32.257632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.257969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.258394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.259348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.261556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.276374 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1335/2671] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:32.285007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.290640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:32.298862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.327544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c78bb-44bc-484c-b9dc-4eb165e8297a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:32.333596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d8941-d676-4144-a802-bd66f71929be admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:32.335356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:32.336356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.336936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.337472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] PATH_INFO: `/v3/users/eb7ecd0dde2241eab1f5a52a6f0ec036` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.338992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7ecd0dde2241eab1f5a52a6f0ec036)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.343117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.344246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:32.345185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.345477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.345762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.347045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.349434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.389864 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1336/2672] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:32.395314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.421765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bdf51-1bbe-4e40-9fbe-0e0d96f6a1df admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:32.434456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d63bdd7-54e6-4423-b391-8bbca1cf525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:32.435284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63bdd7-54e6-4423-b391-8bbca1cf525b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.436389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63bdd7-54e6-4423-b391-8bbca1cf525b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.437943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d63bdd7-54e6-4423-b391-8bbca1cf525b admin admin] PATH_INFO: `/v3/projects/6a7fcfb238474035ba488033b7c5706b/users/69aaa52883684baf98b06a23101c494b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.442160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:32.458634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7ecd0dde2241eab1f5a52a6f0ec036'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:32.486807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7ecd0dde2241eab1f5a52a6f0ec036'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:32.504677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7ecd0dde2241eab1f5a52a6f0ec036'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:32.519470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7ecd0dde2241eab1f5a52a6f0ec036'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:32.520492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1b30cdf6-77b5-457f-a1d6-b8e9c497f57d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7ecd0dde2241eab1f5a52a6f0ec036'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:32.538599 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1337/2673] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:32 2022] DELETE /identity/v3/users/eb7ecd0dde2241eab1f5a52a6f0ec036 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:32.538894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63bdd7-54e6-4423-b391-8bbca1cf525b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fcfb238474035ba488033b7c5706b, user_id=69aaa52883684baf98b06a23101c494b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.540809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.541164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.541333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d63bdd7-54e6-4423-b391-8bbca1cf525b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.541582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.564237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:32.565716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:32.578885 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1337/2674] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:32 2022] PUT /identity/v3/projects/6a7fcfb238474035ba488033b7c5706b/users/69aaa52883684baf98b06a23101c494b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:32.594311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.595992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.596660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.652489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] MFA Rules not processed for user `69aaa52883684baf98b06a23101c494b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:32.653956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:32.696562 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1338/2675] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:32.703048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.708203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:32.859461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1380dec-551e-41c9-a1be-af6018c641f9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:32.860500 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1338/2676] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:32.862085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.864002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:32.870101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:32.870725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.871003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.871271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.881118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479e49d-e4c2-459b-a2ce-e79ca5ef21ee None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:32.890496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.893085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.908265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1339/2677] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:32.909506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:32.909892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:32.910724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:32.911176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:32.911418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:32.911761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] PATH_INFO: `/v3/users/c582bea406ea46498d7f0c175a6d19e6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:32.913120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c582bea406ea46498d7f0c175a6d19e6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:32.915290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:32.995608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c582bea406ea46498d7f0c175a6d19e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:33.002951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:33.010199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c582bea406ea46498d7f0c175a6d19e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:33.012554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c582bea406ea46498d7f0c175a6d19e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:33.036711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c582bea406ea46498d7f0c175a6d19e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:33.037965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9144acfd-918a-47d1-99e0-cb402f328241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c582bea406ea46498d7f0c175a6d19e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:33.046459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6298fd4-5943-4d24-89fb-52be9a22c87c service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:33.057380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10160e-0d23-45df-9a77-a450085b044c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:33.058047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10160e-0d23-45df-9a77-a450085b044c service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:33.058322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10160e-0d23-45df-9a77-a450085b044c service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:33.058603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10160e-0d23-45df-9a77-a450085b044c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:33.060977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ad10160e-0d23-45df-9a77-a450085b044c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:33.067618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:33.078105 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1339/2678] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:32 2022] DELETE /identity/v3/users/c582bea406ea46498d7f0c175a6d19e6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:33.082345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10160e-0d23-45df-9a77-a450085b044c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:33.082345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10160e-0d23-45df-9a77-a450085b044c service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:33.088262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:33.088584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:33.089280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:33.103404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1340/2679] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:32 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:33.117771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:33.119217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:33.174086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:33.174381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:33.174664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:33.207856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] MFA Rules not processed for user `69aaa52883684baf98b06a23101c494b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:33.212622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:33.295749 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1340/2680] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:33.310914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:33.312576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:33.328366 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1341/2681] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:33.439706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f8fbf-c0d1-45ef-b4c4-6350cf2bde44 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:33.446985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27a670-ca92-443a-aa91-cb16987210e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:33.449506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27a670-ca92-443a-aa91-cb16987210e2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:33.449811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27a670-ca92-443a-aa91-cb16987210e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:33.450197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27a670-ca92-443a-aa91-cb16987210e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:33.467305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27a670-ca92-443a-aa91-cb16987210e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:33.468839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27a670-ca92-443a-aa91-cb16987210e2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:33.495756 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1341/2682] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:33.799217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:33.802440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:33.827434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fff0c0-5122-45fc-9e83-e42c5af4de73 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:33.833946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:33.834471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:33.834813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:33.835225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] PATH_INFO: `/v3/projects/7a742224229c4aee831699a7c4002d81` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:33.836829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a742224229c4aee831699a7c4002d81)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:33.844783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:33.986988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a742224229c4aee831699a7c4002d81', 'request_id': 'req-8c862026-6ae3-466b-b229-3192e0aff03e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:33.995222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a742224229c4aee831699a7c4002d81', 'request_id': 'req-8c862026-6ae3-466b-b229-3192e0aff03e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.004192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a742224229c4aee831699a7c4002d81', 'request_id': 'req-8c862026-6ae3-466b-b229-3192e0aff03e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.026343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] The token cache is being invalidate because project 7a742224229c4aee831699a7c4002d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:34.026860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.027716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.039818 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1342/2683] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:33 2022] DELETE /identity/v3/projects/7a742224229c4aee831699a7c4002d81 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:34.042127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd854-459e-4605-8bc7-f9f6697283a1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.042127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd854-459e-4605-8bc7-f9f6697283a1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.042127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd854-459e-4605-8bc7-f9f6697283a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.051867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.053783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.065802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-66fdd854-459e-4605-8bc7-f9f6697283a1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:34.067295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-66fdd854-459e-4605-8bc7-f9f6697283a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.195182 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1342/2684] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:34.195375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c862026-6ae3-466b-b229-3192e0aff03e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.198232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f1e4-e62b-4746-9303-a97e7d7b8281 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.198365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f1e4-e62b-4746-9303-a97e7d7b8281 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.198634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9f1e4-e62b-4746-9303-a97e7d7b8281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.211683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:34.212474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.212728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.212999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] PATH_INFO: `/v3/users/69aaa52883684baf98b06a23101c494b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.214487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69aaa52883684baf98b06a23101c494b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:34.216435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:34.242895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bff9f1e4-e62b-4746-9303-a97e7d7b8281 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:34.247115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:34.268815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bff9f1e4-e62b-4746-9303-a97e7d7b8281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.298425 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1343/2685] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:34.303654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f1e4-e62b-4746-9303-a97e7d7b8281 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.307188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69aaa52883684baf98b06a23101c494b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.330646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69aaa52883684baf98b06a23101c494b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.331099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9f1e4-e62b-4746-9303-a97e7d7b8281 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.342677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:34.343299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.343582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.343855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.344510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:34.344896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:34.346621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69aaa52883684baf98b06a23101c494b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.346820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:34.360102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69aaa52883684baf98b06a23101c494b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.362859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-84145933-de08-48ae-9fda-3b3420f59972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69aaa52883684baf98b06a23101c494b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.375458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1343/2686] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:34 2022] DELETE /identity/v3/users/69aaa52883684baf98b06a23101c494b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:34.381144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.381569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.381974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.382145 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1344/2687] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:34.383088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.389140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.427217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595f4ca-a2f1-46fa-a424-4c0416172200 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.435112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:34.440650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.478218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:34.584173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:34.585342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.585693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.586096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.586979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:34.590142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:34.595840 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1344/2688] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:34.597234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.598630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.612690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761879-95bd-4f6a-84ea-3c84fe5d2d61 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.617623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:34.618293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.618610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.618958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] PATH_INFO: `/v3/projects/c76057169d15499ca9b00409f8657818` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.620226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76057169d15499ca9b00409f8657818)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:34.622362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:34.633640 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1345/2689] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:34 2022] POST /identity/v3/projects => generated 405 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:34.634749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.637061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.707188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76057169d15499ca9b00409f8657818', 'request_id': 'req-76685d94-cee9-485c-b901-a82de5c36e08'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.713857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76057169d15499ca9b00409f8657818', 'request_id': 'req-76685d94-cee9-485c-b901-a82de5c36e08'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.724531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76057169d15499ca9b00409f8657818', 'request_id': 'req-76685d94-cee9-485c-b901-a82de5c36e08'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.726101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92088bb9-54d3-47a3-a483-9ace9bbd13d0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.760198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] The token cache is being invalidate because project c76057169d15499ca9b00409f8657818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:34.760574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.761333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:34.767181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1345/2690] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:34 2022] DELETE /identity/v3/projects/c76057169d15499ca9b00409f8657818 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:34.768763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.770274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.803714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:34.863638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:34.864273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.864715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.865120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.867169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.872573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76685d94-cee9-485c-b901-a82de5c36e08 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.883271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:34.886039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:34.887196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:34.888048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.889577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.889738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.890626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:34.897763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:34.904569 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1346/2691] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:34.906818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.909344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:34.931954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9e72-639e-4326-acd2-170ef2cf8e48 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.942101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:34.942825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:34.943426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:34.943543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] PATH_INFO: `/v3/users/a6653e696f084be886124bc2ae88eea8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:34.945199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6653e696f084be886124bc2ae88eea8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:34.950051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:34.963180 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1346/2692] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:34 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:34.967535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:34.990303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209dd6a-421e-48c3-b5cf-9cc6dcb62c1d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:34.999704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:35.000665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:35.001761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:35.002162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:35.003503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:35.008375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:35.024221 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1347/2693] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:35.028965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:35.042458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e9201-436b-43ae-ac26-4c6b0556bdb7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:35.057538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6653e696f084be886124bc2ae88eea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:35.068395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:35.071114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6653e696f084be886124bc2ae88eea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:35.072545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6653e696f084be886124bc2ae88eea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:35.076168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:35.084326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:35.084577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:35.084900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:35.085029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6653e696f084be886124bc2ae88eea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:35.085592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:35.086289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-485ac713-af6b-4ec5-904d-acc23dd473b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6653e696f084be886124bc2ae88eea8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:35.087491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:35.120493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1348/2694] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:35.123628 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1347/2695] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:34 2022] DELETE /identity/v3/users/a6653e696f084be886124bc2ae88eea8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:35.130524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:35.132838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:35.150086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b763b7-0c8e-499b-91c7-e7eb0272dc38 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:35.220910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:35.221969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:35.222257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:35.222534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] PATH_INFO: `/v3/projects/afb37d675816472db726662b9f0c5994/users/e7392a7b7d474216b89feb16c75e2f63/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:35.243930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb37d675816472db726662b9f0c5994, user_id=e7392a7b7d474216b89feb16c75e2f63, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:35.247875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:35.263371 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1349/2696] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:35 2022] PUT /identity/v3/projects/afb37d675816472db726662b9f0c5994/users/e7392a7b7d474216b89feb16c75e2f63/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:35.269012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:35.269385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:35.269729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:35.294578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:35.295836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:35.362203 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1348/2697] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:35.372145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:35.374021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:35.445063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d6e9b-1f80-4220-a2dc-bd445c412176 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:35.448829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be32729-f5a5-47e3-90da-44d6e586111f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:35.449446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be32729-f5a5-47e3-90da-44d6e586111f service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:35.449935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be32729-f5a5-47e3-90da-44d6e586111f service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:35.450208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be32729-f5a5-47e3-90da-44d6e586111f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:35.463415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be32729-f5a5-47e3-90da-44d6e586111f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:35.464856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be32729-f5a5-47e3-90da-44d6e586111f service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:35.476632 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1350/2698] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:35.981872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:35.983532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:36.003149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ba4ec-d658-4d66-a86f-20bb0096ef7f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.015947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.017669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.018117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.018398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] PATH_INFO: `/v3/projects/d1894536ca864c60911b1465e8d6a8a4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.019737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1894536ca864c60911b1465e8d6a8a4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.022899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.098471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1894536ca864c60911b1465e8d6a8a4', 'request_id': 'req-b58988d6-5413-4169-8479-87fa88490a10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.106457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1894536ca864c60911b1465e8d6a8a4', 'request_id': 'req-b58988d6-5413-4169-8479-87fa88490a10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.116330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1894536ca864c60911b1465e8d6a8a4', 'request_id': 'req-b58988d6-5413-4169-8479-87fa88490a10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.131572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] The token cache is being invalidate because project d1894536ca864c60911b1465e8d6a8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:36.131878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.133969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b58988d6-5413-4169-8479-87fa88490a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.135381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1349/2699] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:35 2022] DELETE /identity/v3/projects/d1894536ca864c60911b1465e8d6a8a4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:36.140330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a2630-2ee5-471d-928d-bb65f4e4471b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.140816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a2630-2ee5-471d-928d-bb65f4e4471b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.141024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a2630-2ee5-471d-928d-bb65f4e4471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.166361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-441a2630-2ee5-471d-928d-bb65f4e4471b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:36.167479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-441a2630-2ee5-471d-928d-bb65f4e4471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:36.172494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:36.274078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1351/2700] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:36.277203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ada7d-3378-489f-bb44-97a17ac8d52c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.277591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ada7d-3378-489f-bb44-97a17ac8d52c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.277972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ada7d-3378-489f-bb44-97a17ac8d52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.305272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e84ada7d-3378-489f-bb44-97a17ac8d52c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:36.306698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e84ada7d-3378-489f-bb44-97a17ac8d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:36.317872 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1350/2701] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:36.323424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a2630-2ee5-471d-928d-bb65f4e4471b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.336356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a2630-2ee5-471d-928d-bb65f4e4471b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.352876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.354666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.354959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.355259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.356107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:36.356417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.358102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.374210 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1352/2702] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:36.381729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ada7d-3378-489f-bb44-97a17ac8d52c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.394158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ada7d-3378-489f-bb44-97a17ac8d52c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.401478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.403124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.403660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.404165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.405817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.410568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.448763 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1351/2703] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:36 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:36.453537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.455904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.458091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:36.468963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d24e1-9f2b-41f1-9da3-055f9c935455 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.535473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:36.583871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34123c59-597e-4839-8fa1-5a99e2a07c15 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.588939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.589645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.590034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.590471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.590586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.591935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.592276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.592561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] PATH_INFO: `/v3/projects/6a7fcfb238474035ba488033b7c5706b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.592671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.594782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7fcfb238474035ba488033b7c5706b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.595868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.597377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.661864 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1353/2704] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:36 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:36.666160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.691002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997edb4-362f-4e59-b841-4923477ac1fd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.706075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.706654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.706886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.707334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.707999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.709809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.713102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7fcfb238474035ba488033b7c5706b', 'request_id': 'req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.723463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7fcfb238474035ba488033b7c5706b', 'request_id': 'req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.726068 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1354/2705] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:36.729691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.748519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7fcfb238474035ba488033b7c5706b', 'request_id': 'req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.749445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5caf0c-a173-4828-897d-b9f83d73bf0a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.763148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] The token cache is being invalidate because project 6a7fcfb238474035ba488033b7c5706b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:36.763884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.765459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a0ef7f8-1a9b-4043-a221-eb81a9f1c2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:36.771611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1352/2706] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:36 2022] DELETE /identity/v3/projects/6a7fcfb238474035ba488033b7c5706b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:36.773246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b9fa5-171f-4e2f-b40f-e3d7434cb2e3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.773804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b9fa5-171f-4e2f-b40f-e3d7434cb2e3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.774227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b9fa5-171f-4e2f-b40f-e3d7434cb2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.814818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-028b9fa5-171f-4e2f-b40f-e3d7434cb2e3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:36.817045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-028b9fa5-171f-4e2f-b40f-e3d7434cb2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:36.884186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.884903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.885309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.885675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.886452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.888281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.906834 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1355/2707] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:36.911141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.913169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:36.930368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c81141-bd79-4eb3-9171-6f1229b39b59 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:36.933269 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1353/2708] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:36.934857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730192e-769b-4136-8a54-2f58cc32856f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.935145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730192e-769b-4136-8a54-2f58cc32856f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.935425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730192e-769b-4136-8a54-2f58cc32856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.937352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179e698-da15-43b7-8e86-399de2b0585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:36.938258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179e698-da15-43b7-8e86-399de2b0585d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:36.939477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179e698-da15-43b7-8e86-399de2b0585d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:36.939822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179e698-da15-43b7-8e86-399de2b0585d admin admin] PATH_INFO: `/v3/projects/58f8ffc98342434fae63ef658e695fc3/users/d61d5d5c997748acb8163b0fdf2f8607/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:36.951302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:36.960356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3730192e-769b-4136-8a54-2f58cc32856f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:36.962338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3730192e-769b-4136-8a54-2f58cc32856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:36.977909 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1354/2709] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:36.978885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730192e-769b-4136-8a54-2f58cc32856f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:36.982104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179e698-da15-43b7-8e86-399de2b0585d admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f8ffc98342434fae63ef658e695fc3, user_id=d61d5d5c997748acb8163b0fdf2f8607, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:36.986676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179e698-da15-43b7-8e86-399de2b0585d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:36.996020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730192e-769b-4136-8a54-2f58cc32856f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.003799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.004794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.005919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.006241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.017989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:37.019493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.022098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.028783 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1356/2710] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:36 2022] PUT /identity/v3/projects/58f8ffc98342434fae63ef658e695fc3/users/d61d5d5c997748acb8163b0fdf2f8607/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:37.039448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0c7dc-bf90-4d7b-8bd3-04834eb8a285 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.039706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0c7dc-bf90-4d7b-8bd3-04834eb8a285 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.039985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0c7dc-bf90-4d7b-8bd3-04834eb8a285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.047987 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1355/2711] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:37.050718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.069692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c439c-0acd-4c85-9621-ee49f6bdae69 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.092933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3cc0c7dc-bf90-4d7b-8bd3-04834eb8a285 None None] MFA Rules not processed for user `d61d5d5c997748acb8163b0fdf2f8607`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:37.094230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3cc0c7dc-bf90-4d7b-8bd3-04834eb8a285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:37.130276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:37.160459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.161150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.161962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.162311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.163885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.166018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.239427 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1356/2712] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:37 2022] POST /identity/v3/projects => generated 397 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:37.243508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.274680 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1357/2713] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:37.283158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0c7dc-bf90-4d7b-8bd3-04834eb8a285 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.296440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0c7dc-bf90-4d7b-8bd3-04834eb8a285 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.373799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.374693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.375211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.375467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.376925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.382059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.433314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543ccca-dc4a-4516-b0f4-8ffd9d4c2535 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.441515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1358/2714] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:37 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:37.442933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.508332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.509414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.509856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.510457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.511196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.517526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.543289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd00a-8ddc-495c-b306-1657fcc67ea1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.552094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.553460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.553882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.554235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.556092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.558851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.564770 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1357/2715] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:37 2022] POST /identity/v3/users => generated 299 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:37.572155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.573872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:37.587847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f8a7e-4248-4c72-beae-c55e64c95600 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.594139 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1359/2716] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:37 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:37.598588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.601671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.603822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.604045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.604299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.604960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.610760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.624095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225fbc4-6574-4fda-8625-79e32ba0de62 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.624980 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1358/2717] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:37.631321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.639399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.642220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.642474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.642764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.643437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.645023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc401ad-d0d2-4706-aa8a-4448108ec5ae admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.645161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.650402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.651454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.651918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.652291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.653156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.655353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.663247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1360/2718] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:37.670339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.681115 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1359/2719] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:37.684110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ebd3e-5d50-4007-81a4-5f6a86cfafcd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.685608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.699736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.699967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c680f1-e3fe-4207-8745-52deeb5c2e9b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.700727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.701113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.701486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] PATH_INFO: `/v3/projects/4f2c1ba23d6d484eb05aed6c93739fc6/users/4b0f1319695b45fba114a0a476a3f224/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.709561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2639427-455c-4927-a803-f141d5459580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.710270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2639427-455c-4927-a803-f141d5459580 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.711044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2639427-455c-4927-a803-f141d5459580 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.711346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2639427-455c-4927-a803-f141d5459580 admin admin] PATH_INFO: `/v3/projects/52dc7565b41b48b7a442810f0f21cc01/users/e01d3916c09a484eb3a687ba732162b1/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.733555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2639427-455c-4927-a803-f141d5459580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52dc7565b41b48b7a442810f0f21cc01, user_id=e01d3916c09a484eb3a687ba732162b1, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.740728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2639427-455c-4927-a803-f141d5459580 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.740892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f2c1ba23d6d484eb05aed6c93739fc6, user_id=4b0f1319695b45fba114a0a476a3f224, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.749129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.761515 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1360/2720] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:37 2022] PUT /identity/v3/projects/52dc7565b41b48b7a442810f0f21cc01/users/e01d3916c09a484eb3a687ba732162b1/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:37.766724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.767342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.767912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.782515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1361/2721] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:37 2022] PUT /identity/v3/projects/4f2c1ba23d6d484eb05aed6c93739fc6/users/4b0f1319695b45fba114a0a476a3f224/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:37.786216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.796785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:37.799003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:37.820251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:37.889586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9274dc-fd86-47da-91e1-71cc3f92d9bc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:37.893970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1361/2722] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:37.916621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:37.918265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:37.924745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:37.926934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:37.928993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:37.929434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:37.931288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:37.937404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:37.985882 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1362/2723] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:37.997403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:38.036347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfba5a-14e4-4f4b-a7fe-ae1a683f3755 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:38.041717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab89ee4-e15f-491d-ae39-a59b7f6314a2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:38.045356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b7c2d-4655-4a36-8190-fd994527c537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:38.046617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b7c2d-4655-4a36-8190-fd994527c537 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:38.047512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b7c2d-4655-4a36-8190-fd994527c537 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:38.049854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b7c2d-4655-4a36-8190-fd994527c537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:38.055267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:38.056919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:38.057735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:38.060430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] PATH_INFO: `/v3/projects/4f2c1ba23d6d484eb05aed6c93739fc6/users/4b0f1319695b45fba114a0a476a3f224/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:38.064141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f2c1ba23d6d484eb05aed6c93739fc6, user_id=4b0f1319695b45fba114a0a476a3f224, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:38.068681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:38.072480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b7c2d-4655-4a36-8190-fd994527c537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:38.076779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b7c2d-4655-4a36-8190-fd994527c537 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:38.101732 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1363/2724] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:37 2022] PUT /identity/v3/projects/4f2c1ba23d6d484eb05aed6c93739fc6/users/4b0f1319695b45fba114a0a476a3f224/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:38.107152 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1362/2725] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:38.109498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:38.113802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:38.114129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:38.114409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:38.118901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:38.180160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] MFA Rules not processed for user `4b0f1319695b45fba114a0a476a3f224`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:38.181879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:38.255278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d7c16-99b4-47d7-aa1c-70ddec446583 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:38.265417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:38.266275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:38.267441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:38.267808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] PATH_INFO: `/v3/users/7721fd95d802434d905b50d23eaad9b7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:38.269620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7721fd95d802434d905b50d23eaad9b7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:38.272097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:38.333170 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1363/2726] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:38.335896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:SELECT=90,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:38.346922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:38.347307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7721fd95d802434d905b50d23eaad9b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:38.351029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:38.381667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7721fd95d802434d905b50d23eaad9b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:38.394715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7721fd95d802434d905b50d23eaad9b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:38.419148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7721fd95d802434d905b50d23eaad9b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:38.420378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e3ef299-da1a-46ea-a851-b503c5838b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7721fd95d802434d905b50d23eaad9b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:38.444419 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1364/2727] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:38 2022] DELETE /identity/v3/users/7721fd95d802434d905b50d23eaad9b7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:38.456418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:38.456718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:38.456992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:38.505580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:38.527566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:38.529325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:38.565995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf6e06-88c6-4537-89ac-4ecc96d979d4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:38.609248 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1365/2728] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:38.632628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:38.634110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:38.683054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45165c-443f-4a80-9e68-c9b08db003bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:38.683867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45165c-443f-4a80-9e68-c9b08db003bc service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:38.686000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45165c-443f-4a80-9e68-c9b08db003bc service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:38.686962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45165c-443f-4a80-9e68-c9b08db003bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:38.690653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4e45165c-443f-4a80-9e68-c9b08db003bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:38.765507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e27937-0f6e-4768-a620-91ff399f82bf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:38.774363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:38.777076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:38.777723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:38.778252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:38.795327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:38.797257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:38.836629 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1366/2729] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:38.847631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45165c-443f-4a80-9e68-c9b08db003bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:38.850363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45165c-443f-4a80-9e68-c9b08db003bc service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:38.866773 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1364/2730] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:39.595031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:39.596867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:39.613979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cd976-fecb-497e-838f-31e178ea2a93 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:39.621354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:39.622470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:39.622961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:39.623444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] PATH_INFO: `/v3/users/d61d5d5c997748acb8163b0fdf2f8607` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:39.626486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61d5d5c997748acb8163b0fdf2f8607)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:39.628975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:39.757266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61d5d5c997748acb8163b0fdf2f8607'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:39.769380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:39.794081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61d5d5c997748acb8163b0fdf2f8607'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:39.806138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61d5d5c997748acb8163b0fdf2f8607'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:39.820154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61d5d5c997748acb8163b0fdf2f8607'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:39.823951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61d5d5c997748acb8163b0fdf2f8607'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:39.841227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1367/2731] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:39 2022] DELETE /identity/v3/users/d61d5d5c997748acb8163b0fdf2f8607 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:39.845533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:39.846271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:39.846271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:39.885743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:39.891238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:40.082197 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1365/2732] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:40.102851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:40.104811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:40.281282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b76c7-b653-4fde-93a5-483ec30e0f85 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:40.293375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:40.296495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:40.297046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:40.297430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:40.324612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:40.326326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:40.356013 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1368/2733] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:41.060650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:41.062279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:41.086060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d083b7-7b1b-4d5d-a425-dd7d29617ac6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:41.091402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:41.092390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:41.092893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:41.093403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] PATH_INFO: `/v3/projects/c82fb214ece04574bf521a45ba055bdf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:41.097589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82fb214ece04574bf521a45ba055bdf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:41.104788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:41.133116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:41.223944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82fb214ece04574bf521a45ba055bdf', 'request_id': 'req-8d89705d-2a98-41dd-8499-8c06303f5a94'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.237928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82fb214ece04574bf521a45ba055bdf', 'request_id': 'req-8d89705d-2a98-41dd-8499-8c06303f5a94'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.252403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82fb214ece04574bf521a45ba055bdf', 'request_id': 'req-8d89705d-2a98-41dd-8499-8c06303f5a94'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.267957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] The token cache is being invalidate because project c82fb214ece04574bf521a45ba055bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:41.268765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.269890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.285715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:41.314566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1366/2734] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:41 2022] DELETE /identity/v3/projects/c82fb214ece04574bf521a45ba055bdf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:41.320195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755854a-b3ad-44bb-835d-b53c8659f7b0 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:41.454110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:41.454773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:41.455055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:41.455332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] PATH_INFO: `/v3/users/3e95489068c843aaad1a10314d58520a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:41.457124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e95489068c843aaad1a10314d58520a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:41.459448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:41.524377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:41.526541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:41.537172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e95489068c843aaad1a10314d58520a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.554150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89705d-2a98-41dd-8499-8c06303f5a94 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:41.555541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e95489068c843aaad1a10314d58520a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.567701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e95489068c843aaad1a10314d58520a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.569731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:41.570779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:41.571370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:41.571829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] PATH_INFO: `/v3/projects/58f8ffc98342434fae63ef658e695fc3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:41.573345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:41.575075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f8ffc98342434fae63ef658e695fc3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:41.582641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:41.582798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e95489068c843aaad1a10314d58520a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.584319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e95489068c843aaad1a10314d58520a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.596852 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1369/2735] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:41 2022] DELETE /identity/v3/users/3e95489068c843aaad1a10314d58520a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:41.828172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f8ffc98342434fae63ef658e695fc3', 'request_id': 'req-6c667371-535a-4caa-80f8-1510741cc1cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.851717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f8ffc98342434fae63ef658e695fc3', 'request_id': 'req-6c667371-535a-4caa-80f8-1510741cc1cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.868203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f8ffc98342434fae63ef658e695fc3', 'request_id': 'req-6c667371-535a-4caa-80f8-1510741cc1cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.889590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] The token cache is being invalidate because project 58f8ffc98342434fae63ef658e695fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:41.890986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.892782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:41.895960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1367/2736] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:41 2022] DELETE /identity/v3/projects/58f8ffc98342434fae63ef658e695fc3 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:41.901491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:41.903954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:42.046978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c2ec-e364-4321-aee4-4b6e33ec446c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:42.182802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:42.184676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:42.186543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:42.186933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] PATH_INFO: `/v3/users/4b0f1319695b45fba114a0a476a3f224` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:42.190002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0f1319695b45fba114a0a476a3f224)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:42.193985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:42.326409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0f1319695b45fba114a0a476a3f224'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:42.377186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0f1319695b45fba114a0a476a3f224'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:42.397507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0f1319695b45fba114a0a476a3f224'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:42.411618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0f1319695b45fba114a0a476a3f224'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:42.412253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de37ccbf-940b-469e-8e93-3f9326863045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0f1319695b45fba114a0a476a3f224'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:42.422265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1370/2737] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:41 2022] DELETE /identity/v3/users/4b0f1319695b45fba114a0a476a3f224 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:43.022343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:43.025598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:43.051768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c667371-535a-4caa-80f8-1510741cc1cb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:43.165762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:43.166698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:43.167502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:43.168810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] PATH_INFO: `/v3/projects/5abd6c399336495e87515a95e895282e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:43.172237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5abd6c399336495e87515a95e895282e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:43.202311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:43.265212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:43.297633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5abd6c399336495e87515a95e895282e', 'request_id': 'req-9aae3144-cbe0-4b6e-be34-96ccf56613c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:43.306750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5abd6c399336495e87515a95e895282e', 'request_id': 'req-9aae3144-cbe0-4b6e-be34-96ccf56613c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:43.322010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5abd6c399336495e87515a95e895282e', 'request_id': 'req-9aae3144-cbe0-4b6e-be34-96ccf56613c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:43.343568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] The token cache is being invalidate because project 5abd6c399336495e87515a95e895282e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:43.344317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:43.346214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9aae3144-cbe0-4b6e-be34-96ccf56613c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:43.350806 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1368/2738] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:43 2022] DELETE /identity/v3/projects/5abd6c399336495e87515a95e895282e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:43.355995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8b35e-30aa-4125-bc5a-d1c783d3632d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:43.356518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8b35e-30aa-4125-bc5a-d1c783d3632d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:43.356619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8b35e-30aa-4125-bc5a-d1c783d3632d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:43.380912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fdc8b35e-30aa-4125-bc5a-d1c783d3632d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:43.382797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fdc8b35e-30aa-4125-bc5a-d1c783d3632d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:43.404012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:43.475780 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1371/2739] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:43.483258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6e9b-861c-4d64-af34-f4e035823716 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:43.483695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6e9b-861c-4d64-af34-f4e035823716 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:43.484072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6e9b-861c-4d64-af34-f4e035823716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:43.524763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6cff6e9b-861c-4d64-af34-f4e035823716 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:43.526185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6cff6e9b-861c-4d64-af34-f4e035823716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:43.545869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1369/2740] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:43.553339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8b35e-30aa-4125-bc5a-d1c783d3632d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:43.580155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8b35e-30aa-4125-bc5a-d1c783d3632d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:43.591463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:43.592502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:43.592946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:43.593505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:43.594859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:43.595519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:43.597632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:43.645652 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1372/2741] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:43.649418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff6e9b-861c-4d64-af34-f4e035823716 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:43.660320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff6e9b-861c-4d64-af34-f4e035823716 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:43.674364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:43.675283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:43.675816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:43.676195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:43.686790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:43.689536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:43.755787 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1370/2742] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:43 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:43.760483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:43.773496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199b260-7bb9-4a35-a720-03cf8de2c234 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:43.801668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:43.804004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:43.943731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:43.946193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:43.946913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:43.947210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:43.948138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:43.954866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:43.989766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ded85-046c-405e-aba4-bb0c1163acf8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.007658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:44.015577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.015899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.016195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] PATH_INFO: `/v3/projects/4f2c1ba23d6d484eb05aed6c93739fc6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.021021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1373/2743] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:43 2022] POST /identity/v3/users => generated 302 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:44.021170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f2c1ba23d6d484eb05aed6c93739fc6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:44.023622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:44.025626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:44.044259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563bbe6-a5e8-4dd6-b794-7df7d8004301 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.056160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:44.057076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.057513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.057991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.058951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:44.061599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:44.090794 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1374/2744] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:44.118927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:44.153024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930b124-9bb1-49ab-ac78-0230a6a2a587 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.174876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f2c1ba23d6d484eb05aed6c93739fc6', 'request_id': 'req-125ac78c-b4fa-41e3-8c9c-31abaca65b67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:44.206626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f2c1ba23d6d484eb05aed6c93739fc6', 'request_id': 'req-125ac78c-b4fa-41e3-8c9c-31abaca65b67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:44.232033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f2c1ba23d6d484eb05aed6c93739fc6', 'request_id': 'req-125ac78c-b4fa-41e3-8c9c-31abaca65b67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:44.275025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] The token cache is being invalidate because project 4f2c1ba23d6d484eb05aed6c93739fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:44.278831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:44.280747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-125ac78c-b4fa-41e3-8c9c-31abaca65b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:44.287618 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1371/2745] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:43 2022] DELETE /identity/v3/projects/4f2c1ba23d6d484eb05aed6c93739fc6 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:44.302656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8193593-ead6-43d9-a6dd-f71b10573e58 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.302656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8193593-ead6-43d9-a6dd-f71b10573e58 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.305968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8193593-ead6-43d9-a6dd-f71b10573e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.400671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e8193593-ead6-43d9-a6dd-f71b10573e58 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:44.408892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e8193593-ead6-43d9-a6dd-f71b10573e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:44.419429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:44.421519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.423830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.424414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.426001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:44.429606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:44.430083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1372/2746] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:44.439405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72faa1-03cf-4b91-9931-0e2553a2a030 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.439767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72faa1-03cf-4b91-9931-0e2553a2a030 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.439958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72faa1-03cf-4b91-9931-0e2553a2a030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.444547 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1375/2747] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:44.448347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:44.452915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:44.472228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3f72faa1-03cf-4b91-9931-0e2553a2a030 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:44.473962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3f72faa1-03cf-4b91-9931-0e2553a2a030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:44.482020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbbc1ef-aed7-4945-bd1d-4a43f960cc43 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.489785 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1373/2748] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:44.493583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f08b79-a08d-4572-8b62-4b34a56263fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:44.494549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f08b79-a08d-4572-8b62-4b34a56263fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.495596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f08b79-a08d-4572-8b62-4b34a56263fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.496105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f08b79-a08d-4572-8b62-4b34a56263fc admin admin] PATH_INFO: `/v3/projects/de87134dfa234cefabb413dbe523ae4c/users/d7c632adbf784d28ab306c9cb60b2a3f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.500025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72faa1-03cf-4b91-9931-0e2553a2a030 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:44.506766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:44.515126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72faa1-03cf-4b91-9931-0e2553a2a030 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.526914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:44.528065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.528467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.528850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.530280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:44.531062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:44.533331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:44.536077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f08b79-a08d-4572-8b62-4b34a56263fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=de87134dfa234cefabb413dbe523ae4c, user_id=d7c632adbf784d28ab306c9cb60b2a3f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:44.542251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f08b79-a08d-4572-8b62-4b34a56263fc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:44.550019 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1374/2749] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:52:44.554252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:44.571778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19f56e-4881-45ab-9ae7-77d00ce798b3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.573048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:44.595895 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1376/2750] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:44 2022] PUT /identity/v3/projects/de87134dfa234cefabb413dbe523ae4c/users/d7c632adbf784d28ab306c9cb60b2a3f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:44.600321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137440-4aca-4d14-a297-da6475fc348d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.601869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137440-4aca-4d14-a297-da6475fc348d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.602312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137440-4aca-4d14-a297-da6475fc348d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.682019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f8137440-4aca-4d14-a297-da6475fc348d None None] MFA Rules not processed for user `d7c632adbf784d28ab306c9cb60b2a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:44.683560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f8137440-4aca-4d14-a297-da6475fc348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:44.727505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:44.728761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:44.729986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:44.730628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:44.731569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:44.736379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:44.828485 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1375/2751] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:44 2022] POST /identity/v3/projects => generated 409 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:44.831120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:44.846329 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1377/2752] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:44.848145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8137440-4aca-4d14-a297-da6475fc348d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:44.868826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8137440-4aca-4d14-a297-da6475fc348d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.991703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484710f9-b8e6-40d1-a50b-0b3da04f7f87 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:44.999021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:44.999775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.000056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.000328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.000490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea944e-bf26-4089-bc80-638e6c0f9012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.001180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea944e-bf26-4089-bc80-638e6c0f9012 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.001532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea944e-bf26-4089-bc80-638e6c0f9012 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.001981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.002097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea944e-bf26-4089-bc80-638e6c0f9012 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.002810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea944e-bf26-4089-bc80-638e6c0f9012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.004029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.005478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea944e-bf26-4089-bc80-638e6c0f9012 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.043714 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1378/2753] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:44 2022] POST /identity/v3/projects => generated 403 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:45.045109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6a37-7386-4bad-918d-902169c271ec None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.045436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6a37-7386-4bad-918d-902169c271ec None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.045711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6a37-7386-4bad-918d-902169c271ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.050834 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1376/2754] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:44 2022] POST /identity/v3/users => generated 305 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:45.051748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.074226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd3890-4fa6-4e16-978f-3d6aa34de19b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.087657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a40c6a37-7386-4bad-918d-902169c271ec None None] MFA Rules not processed for user `e7392a7b7d474216b89feb16c75e2f63`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:45.089453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a40c6a37-7386-4bad-918d-902169c271ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:45.147873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.149027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.149203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.149547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.150272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.152451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.166847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:45.192900 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1377/2755] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:45 2022] POST /identity/v3/users => generated 308 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:45.194258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.210875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197f091-b4f1-43a9-92cf-c476b295fa8f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.216406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.217177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.217602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.218516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.218930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.221658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.238921 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1379/2756] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:45.240031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6a37-7386-4bad-918d-902169c271ec None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.247984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1378/2757] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:45.249411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.254687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:45.268902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217ebd0-1d92-4786-b192-2a0d3308fa03 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.273720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6a37-7386-4bad-918d-902169c271ec None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.274441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.275197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.275578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.275854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.276493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.278256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.285050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.286558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.290614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.291372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.292375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.294844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.299672 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1379/2758] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:45.299672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.300331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:45.311309 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1380/2759] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:45.312593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.332600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302d5a-6c4d-48d8-ac8a-4341d85084e0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.346850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.347537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.347819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.348098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] PATH_INFO: `/v3/projects/c35198fec11a4f43b49e60a5f92623aa/users/1a4133b199784c499f3a8f1a02e50dec/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.387480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35198fec11a4f43b49e60a5f92623aa, user_id=1a4133b199784c499f3a8f1a02e50dec, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.390221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.416838 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1381/2760] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:45 2022] PUT /identity/v3/projects/c35198fec11a4f43b49e60a5f92623aa/users/1a4133b199784c499f3a8f1a02e50dec/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:45.417646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.438716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478132e0-1570-4d67-9d8c-23cc3047a7ba admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.572402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21e57-b473-4e22-9a30-bf76fff6c879 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.578579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb9c8-40ec-49fc-98b7-a9df1bdab4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.580097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb9c8-40ec-49fc-98b7-a9df1bdab4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.580384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb9c8-40ec-49fc-98b7-a9df1bdab4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.580665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb9c8-40ec-49fc-98b7-a9df1bdab4f2 admin admin] PATH_INFO: `/v3/projects/d75ae76f6fec4a688e774d7f0af40f2c/users/e06f0e2458704f908f7302866c3787b1/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.605151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.605151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.605552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.605687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.624646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcb9c8-40ec-49fc-98b7-a9df1bdab4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75ae76f6fec4a688e774d7f0af40f2c, user_id=e06f0e2458704f908f7302866c3787b1, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.628077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcb9c8-40ec-49fc-98b7-a9df1bdab4f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.666515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1382/2761] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:45 2022] PUT /identity/v3/projects/d75ae76f6fec4a688e774d7f0af40f2c/users/e06f0e2458704f908f7302866c3787b1/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:45.670770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8c18c-4b3a-4bfc-8c1e-7216399d745e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.671058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8c18c-4b3a-4bfc-8c1e-7216399d745e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.671346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8c18c-4b3a-4bfc-8c1e-7216399d745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.746376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6cd8c18c-4b3a-4bfc-8c1e-7216399d745e None None] MFA Rules not processed for user `1a4133b199784c499f3a8f1a02e50dec`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:45.747617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6cd8c18c-4b3a-4bfc-8c1e-7216399d745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:45.783641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.785371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.799051 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1380/2762] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:45 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:45.800162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.812529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e42439-27a8-4d01-a568-ff9b2d55bbd7 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.824485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:45.934994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.935823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.936106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:45.936382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:45.937028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:45.938990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:45.947613 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1383/2763] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:45.952643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8c18c-4b3a-4bfc-8c1e-7216399d745e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.964225 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1381/2764] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:45.971693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:45.980251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8c18c-4b3a-4bfc-8c1e-7216399d745e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:45.992896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf42a8-9a65-47a6-8d88-4f935d7c5408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:45.993460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf42a8-9a65-47a6-8d88-4f935d7c5408 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:45.993728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf42a8-9a65-47a6-8d88-4f935d7c5408 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.004599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45659065-518b-4c66-8df9-8fccf35c216d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.005490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf42a8-9a65-47a6-8d88-4f935d7c5408 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.006244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf42a8-9a65-47a6-8d88-4f935d7c5408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.008345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf42a8-9a65-47a6-8d88-4f935d7c5408 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.022546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.023224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.023503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.023778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] PATH_INFO: `/v3/projects/d75ae76f6fec4a688e774d7f0af40f2c/users/e06f0e2458704f908f7302866c3787b1/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.028978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75ae76f6fec4a688e774d7f0af40f2c, user_id=e06f0e2458704f908f7302866c3787b1, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.031630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.070688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1382/2765] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:45 2022] PUT /identity/v3/projects/d75ae76f6fec4a688e774d7f0af40f2c/users/e06f0e2458704f908f7302866c3787b1/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:46.078509 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1384/2766] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:45 2022] POST /identity/v3/projects => generated 409 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:46.079927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:46.082231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.082515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.082791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.155319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] MFA Rules not processed for user `e06f0e2458704f908f7302866c3787b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:46.157293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:46.177015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:46.229378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427ffa3-391e-4e86-a0b1-c8e7dec5496d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.339299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1385/2767] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:46.350339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:46.351853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:46.397026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.397776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.398173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.399054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.399777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.407824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.441320 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1383/2768] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:46 2022] POST /identity/v3/users => generated 311 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:46.442649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:46.457885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56123287-dc30-466f-bd7e-624a415d5178 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.465482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.466307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.466725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.467109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.468145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.471905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.493068 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1384/2769] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:52:46.498381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:46.510574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee0f6c-e7f6-4199-8a5b-7bb49ff9105d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.514409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22029956-bed2-454a-87ab-6ac88ea74c5f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.516845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.517807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.518299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.518608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] PATH_INFO: `/v3/projects/452cae40661345769a7c731371d0ba89/users/bf675e107ee647789d7efbe898704cc0/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.528308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.528953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.529240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.529520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.542377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.544205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.557407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:46.562809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452cae40661345769a7c731371d0ba89, user_id=bf675e107ee647789d7efbe898704cc0, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.565379 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1386/2770] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:46 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:46.567380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.601487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:46.612610 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1385/2771] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:46 2022] PUT /identity/v3/projects/452cae40661345769a7c731371d0ba89/users/bf675e107ee647789d7efbe898704cc0/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:46.624872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca66b6-77ad-4095-961b-58dbdfc78320 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.835258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde4802-6d3d-4bd7-928c-225d7b42e0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.836134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4802-6d3d-4bd7-928c-225d7b42e0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.836647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4802-6d3d-4bd7-928c-225d7b42e0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.837142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde4802-6d3d-4bd7-928c-225d7b42e0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.838065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde4802-6d3d-4bd7-928c-225d7b42e0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.841088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde4802-6d3d-4bd7-928c-225d7b42e0a6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:46.863881 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1387/2772] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:46.869135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:46.894958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c7bae-1253-423f-99ff-cd41440c5cb6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:46.906526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:46.909919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:46.910230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:46.910512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] PATH_INFO: `/v3/projects/452cae40661345769a7c731371d0ba89/users/bf675e107ee647789d7efbe898704cc0/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:46.918378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452cae40661345769a7c731371d0ba89, user_id=bf675e107ee647789d7efbe898704cc0, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:46.930374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:47.012629 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1386/2773] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:46 2022] PUT /identity/v3/projects/452cae40661345769a7c731371d0ba89/users/bf675e107ee647789d7efbe898704cc0/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:47.013665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7c7dd-894e-4991-837f-0d9a7c37c755 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:47.013989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7c7dd-894e-4991-837f-0d9a7c37c755 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:47.014277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7c7dd-894e-4991-837f-0d9a7c37c755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:47.075521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f0d7c7dd-894e-4991-837f-0d9a7c37c755 None None] MFA Rules not processed for user `bf675e107ee647789d7efbe898704cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:47.076766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f0d7c7dd-894e-4991-837f-0d9a7c37c755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:47.122252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:47.276020 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1388/2774] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:47 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:52:47.365949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:47.505218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c93956-4ca2-45fb-8811-e4553c8c36c4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:47.513634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:47.514296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:47.514574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:47.518133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:47.529639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:47.531237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:47.571934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1387/2775] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:47 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:47.900821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7c7dd-894e-4991-837f-0d9a7c37c755 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:47.930712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7c7dd-894e-4991-837f-0d9a7c37c755 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:47.938380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:47.939402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:47.941245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:47.941734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:48.058725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:48.060605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:48.087071 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1389/2776] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:47 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:48.326729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:48.349495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfca45-e65d-4a6d-b0e4-e91529eccecf service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:48.355594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:48.356280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:48.356627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:48.357040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:48.465639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:48.469789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:48.492925 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1388/2777] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:48 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:48.523168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:48.663433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0734b-ac54-4034-b4eb-9402fafece4c service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:48.673017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:48.673729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:48.674052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:48.675557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] PATH_INFO: `/v3/users/d7c632adbf784d28ab306c9cb60b2a3f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:48.676841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c632adbf784d28ab306c9cb60b2a3f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:48.680248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:48.771368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c632adbf784d28ab306c9cb60b2a3f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:48.784576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:48.804168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c632adbf784d28ab306c9cb60b2a3f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:48.833682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c632adbf784d28ab306c9cb60b2a3f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:48.834023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:48.847412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c632adbf784d28ab306c9cb60b2a3f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:48.847839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b502d04-c045-4d5f-b65e-519a8c667cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c632adbf784d28ab306c9cb60b2a3f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:48.865577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13214957-5e9b-4410-b372-f91a5e5b3f10 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:48.871030 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1390/2778] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:48 2022] DELETE /identity/v3/users/d7c632adbf784d28ab306c9cb60b2a3f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:48.872686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:48.873193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:48.873784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:48.926949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:48.930316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.036026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:49.037558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:49.037982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:49.038433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] PATH_INFO: `/v3/users/1a4133b199784c499f3a8f1a02e50dec` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:49.052986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4133b199784c499f3a8f1a02e50dec)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:49.058152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:49.061403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:49.097494 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1391/2779] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:49.173960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:49.175783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.226450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4133b199784c499f3a8f1a02e50dec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.250213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4133b199784c499f3a8f1a02e50dec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.250707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4133b199784c499f3a8f1a02e50dec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.270504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4133b199784c499f3a8f1a02e50dec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.271257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4133b199784c499f3a8f1a02e50dec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.305932 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1389/2780] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:48 2022] DELETE /identity/v3/users/1a4133b199784c499f3a8f1a02e50dec => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:49.307051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:49.311359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.371720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8d03-5ebf-4011-bde1-b892170a1767 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:49.526591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b7ea8-8be7-4875-9037-b6dba331a60b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:49.538731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:49.539544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:49.541019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:49.541494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] PATH_INFO: `/v3/users/3f0ba978c04048a29bf4132a1e7379b9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:49.542891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0ba978c04048a29bf4132a1e7379b9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:49.545108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:49.553098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191320d-fab5-4ecb-aa8b-930558a7b4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:49.554972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191320d-fab5-4ecb-aa8b-930558a7b4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:49.555289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191320d-fab5-4ecb-aa8b-930558a7b4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:49.556521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191320d-fab5-4ecb-aa8b-930558a7b4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:49.562084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2191320d-fab5-4ecb-aa8b-930558a7b4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.589590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191320d-fab5-4ecb-aa8b-930558a7b4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:49.591091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191320d-fab5-4ecb-aa8b-930558a7b4a0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:49.625006 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1392/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:52:49.628515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:49.628759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:49.629016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:49.669442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0ba978c04048a29bf4132a1e7379b9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.690085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0ba978c04048a29bf4132a1e7379b9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.691461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0ba978c04048a29bf4132a1e7379b9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.692000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:49.695730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.711734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0ba978c04048a29bf4132a1e7379b9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.712814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0ba978c04048a29bf4132a1e7379b9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:49.721265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1393/2782] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:49.739604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:49.753975 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1390/2783] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:49 2022] DELETE /identity/v3/users/3f0ba978c04048a29bf4132a1e7379b9 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:49.756402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:49.760112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:49.821108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d759-3f03-4acf-86ec-1c91c5e45f79 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:49.847354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:49.926598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6534a-2746-42b1-910c-a2da261d55b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:49.927910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6534a-2746-42b1-910c-a2da261d55b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:49.928569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6534a-2746-42b1-910c-a2da261d55b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:49.928906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6534a-2746-42b1-910c-a2da261d55b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:49.934373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1be6534a-2746-42b1-910c-a2da261d55b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:50.064696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6534a-2746-42b1-910c-a2da261d55b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:50.067690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6534a-2746-42b1-910c-a2da261d55b5 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:50.087411 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1394/2784] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:50.289892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:50.291585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:50.314552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9ffe7-03d1-4b72-9825-a682bc99b593 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:50.337617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:50.338315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:50.338968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:50.338968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] PATH_INFO: `/v3/projects/a7933149ce16427e84de63058fff1142` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:50.340069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7933149ce16427e84de63058fff1142)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:50.342162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:50.442693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7933149ce16427e84de63058fff1142', 'request_id': 'req-856d802e-2751-4d5a-b921-db616266bdae'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:50.456943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7933149ce16427e84de63058fff1142', 'request_id': 'req-856d802e-2751-4d5a-b921-db616266bdae'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:50.474257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7933149ce16427e84de63058fff1142', 'request_id': 'req-856d802e-2751-4d5a-b921-db616266bdae'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:50.503561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] The token cache is being invalidate because project a7933149ce16427e84de63058fff1142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:50.503945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:50.506142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-856d802e-2751-4d5a-b921-db616266bdae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:50.513839 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1391/2785] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:50 2022] DELETE /identity/v3/projects/a7933149ce16427e84de63058fff1142 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:50.519415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392a4f-1e68-4b65-8e70-7f4a57e2840c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:50.520076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392a4f-1e68-4b65-8e70-7f4a57e2840c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:50.520520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392a4f-1e68-4b65-8e70-7f4a57e2840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:50.556401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d392a4f-1e68-4b65-8e70-7f4a57e2840c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:50.557641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d392a4f-1e68-4b65-8e70-7f4a57e2840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:50.707011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a88c7-6950-4aab-8225-03d2f80d5f1a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:50.707574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a88c7-6950-4aab-8225-03d2f80d5f1a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:50.707574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a88c7-6950-4aab-8225-03d2f80d5f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:50.707811 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1395/2786] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:50.742337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-401a88c7-6950-4aab-8225-03d2f80d5f1a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:50.747788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-401a88c7-6950-4aab-8225-03d2f80d5f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:50.782309 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1392/2787] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:50.786335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392a4f-1e68-4b65-8e70-7f4a57e2840c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:50.799874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392a4f-1e68-4b65-8e70-7f4a57e2840c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:50.818022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:50.819082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:50.819734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:50.820307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:50.821638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:50.822362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:50.824299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:50.860694 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1396/2788] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:50.862120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a88c7-6950-4aab-8225-03d2f80d5f1a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:50.877534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a88c7-6950-4aab-8225-03d2f80d5f1a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:50.886071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:50.888469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:50.888977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:50.889703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:50.890582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:50.894589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:50.943677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:50.951920 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1393/2789] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:50 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:50.956890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece48c75-d83f-4afd-acfc-dac288439df2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:51.057257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:51.058362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:51.058362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:51.058820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:51.059554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:51.061864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:51.074342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:51.105642 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1397/2790] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:50 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:51.110334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:51.122177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21631d60-69fb-4075-baf5-ba988f99fc83 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:51.129946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:51.130725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:51.131106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:51.131479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:51.132258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:51.134030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:51.150974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:51.158748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1394/2791] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:51.163182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd375-ccdb-4a23-a7a6-abdbb3eac38e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:51.169767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f683cb-c84e-4c18-8ffd-d4220e1ce239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:51.170705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f683cb-c84e-4c18-8ffd-d4220e1ce239 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:51.171125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f683cb-c84e-4c18-8ffd-d4220e1ce239 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:51.171516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f683cb-c84e-4c18-8ffd-d4220e1ce239 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:51.173489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f683cb-c84e-4c18-8ffd-d4220e1ce239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:51.174708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f683cb-c84e-4c18-8ffd-d4220e1ce239 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:51.192739 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1398/2792] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:51.197128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:51.208283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a8a95-51cf-4237-993d-5ca2edf0475f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:51.217204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:51.218313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:51.218838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:51.220551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] PATH_INFO: `/v3/projects/a45a58e14da346efbb24a4bfed8b0b69/users/5053eb274d2c4e8180aa1cbcb680a94f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:51.240000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45a58e14da346efbb24a4bfed8b0b69, user_id=5053eb274d2c4e8180aa1cbcb680a94f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:51.243355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:51.267138 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1395/2793] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:51 2022] PUT /identity/v3/projects/a45a58e14da346efbb24a4bfed8b0b69/users/5053eb274d2c4e8180aa1cbcb680a94f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:51.287226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:51.287676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:51.287925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:51.382926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:51.384243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:51.485600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:51.488908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:51.552391 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1399/2794] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:51.554166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:51.555650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:51.578056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf91b-cce6-44d4-8fd0-2de7dc413fa4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:51.588160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:51.589012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:51.589447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:51.590099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] PATH_INFO: `/v3/projects/de87134dfa234cefabb413dbe523ae4c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:51.592732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de87134dfa234cefabb413dbe523ae4c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:51.597499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:51.670955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3db81-3a92-4487-89d7-81c794791775 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:51.686838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:51.687765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:51.688718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:51.689565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] PATH_INFO: `/v3/projects/c35198fec11a4f43b49e60a5f92623aa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:51.697659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35198fec11a4f43b49e60a5f92623aa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:51.700072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:51.745257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:51.769629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de87134dfa234cefabb413dbe523ae4c', 'request_id': 'req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.787089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de87134dfa234cefabb413dbe523ae4c', 'request_id': 'req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.795609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35198fec11a4f43b49e60a5f92623aa', 'request_id': 'req-f740c2ea-db62-41c2-a7ee-a4a4cc978673'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.808472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de87134dfa234cefabb413dbe523ae4c', 'request_id': 'req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.841171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35198fec11a4f43b49e60a5f92623aa', 'request_id': 'req-f740c2ea-db62-41c2-a7ee-a4a4cc978673'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.841813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] The token cache is being invalidate because project de87134dfa234cefabb413dbe523ae4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:51.842144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.843873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.856068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35198fec11a4f43b49e60a5f92623aa', 'request_id': 'req-f740c2ea-db62-41c2-a7ee-a4a4cc978673'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.859287 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1400/2795] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:51 2022] DELETE /identity/v3/projects/de87134dfa234cefabb413dbe523ae4c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:51.860967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:51.862453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:51.873733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] The token cache is being invalidate because project c35198fec11a4f43b49e60a5f92623aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:51.874151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.876086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:51.882430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1396/2796] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:51 2022] DELETE /identity/v3/projects/c35198fec11a4f43b49e60a5f92623aa => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:51.883799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:51.902720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:52.008164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8323a-5da8-46fc-a1a7-f076a25a4a4e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:52.029164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:52.068960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740c2ea-db62-41c2-a7ee-a4a4cc978673 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:52.081739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:52.083607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:52.085708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:52.086130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] PATH_INFO: `/v3/users/e06f0e2458704f908f7302866c3787b1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:52.090259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06f0e2458704f908f7302866c3787b1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:52.094713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:52.102299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:52.103195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:52.103484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:52.103766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:52.107545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:52.135472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:52.137007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:52.152669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1401/2797] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:52.153563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:52.154974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:52.169305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccf668-5528-4cef-9485-1938ceb633cb service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:52.183888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:52.189614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:52.190294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:52.190790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] PATH_INFO: `/v3/users/bf675e107ee647789d7efbe898704cc0` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:52.194898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf675e107ee647789d7efbe898704cc0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:52.198676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:52.217328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06f0e2458704f908f7302866c3787b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.241993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06f0e2458704f908f7302866c3787b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.242723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06f0e2458704f908f7302866c3787b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.259608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06f0e2458704f908f7302866c3787b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.260246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06f0e2458704f908f7302866c3787b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.306234 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1397/2798] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:51 2022] DELETE /identity/v3/users/e06f0e2458704f908f7302866c3787b1 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:52.326916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf675e107ee647789d7efbe898704cc0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.343407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf675e107ee647789d7efbe898704cc0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.351471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf675e107ee647789d7efbe898704cc0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.364507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf675e107ee647789d7efbe898704cc0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.365448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4839796-6e56-41a4-b544-aa737ede0b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf675e107ee647789d7efbe898704cc0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:52.377602 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1402/2799] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:52:52 2022] DELETE /identity/v3/users/bf675e107ee647789d7efbe898704cc0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:53.270386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:53.272758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:53.319774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:53.376363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb27559-1789-4ad5-b4dc-d7a8f972affd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:53.449221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:53.450077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.450432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.450777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] PATH_INFO: `/v3/projects/d75ae76f6fec4a688e774d7f0af40f2c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.452025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75ae76f6fec4a688e774d7f0af40f2c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:53.455940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:53.532784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75ae76f6fec4a688e774d7f0af40f2c', 'request_id': 'req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.543862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75ae76f6fec4a688e774d7f0af40f2c', 'request_id': 'req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.566420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75ae76f6fec4a688e774d7f0af40f2c', 'request_id': 'req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.588222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] The token cache is being invalidate because project d75ae76f6fec4a688e774d7f0af40f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:53.588836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.589687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.591972 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1398/2800] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:53 2022] DELETE /identity/v3/projects/d75ae76f6fec4a688e774d7f0af40f2c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:53.596950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c4d7d-ce19-4ce1-9e03-7531979f37a8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.598059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c4d7d-ce19-4ce1-9e03-7531979f37a8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.598779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c4d7d-ce19-4ce1-9e03-7531979f37a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.630805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:53.634169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:53.645403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-110c4d7d-ce19-4ce1-9e03-7531979f37a8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:53.646677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-110c4d7d-ce19-4ce1-9e03-7531979f37a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:53.825261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1403/2801] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:53.834124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2254774-eb87-49ec-8090-7924ef0eeb8c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.834410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2254774-eb87-49ec-8090-7924ef0eeb8c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.834780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2254774-eb87-49ec-8090-7924ef0eeb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.838288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcdf799-83ad-4d2b-9bb8-3f2eaf284356 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:53.855930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:53.856580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.856870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.857147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] PATH_INFO: `/v3/projects/452cae40661345769a7c731371d0ba89` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.861065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] RBAC: Authorizing `identity:delete_project(project_id=452cae40661345769a7c731371d0ba89)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:53.866534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:53.867696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d2254774-eb87-49ec-8090-7924ef0eeb8c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:53.872194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2254774-eb87-49ec-8090-7924ef0eeb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:53.886958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:53.897627 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1404/2802] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:53.901569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2254774-eb87-49ec-8090-7924ef0eeb8c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:53.914331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2254774-eb87-49ec-8090-7924ef0eeb8c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:53.922247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:53.924035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:53.924663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:53.925046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:53.929526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:53.929994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:53.932326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:53.950612 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1405/2803] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:52:53.957114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '452cae40661345769a7c731371d0ba89', 'request_id': 'req-b52db798-6e89-46f4-b5e3-af107b8b984d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.957571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:53.972482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '452cae40661345769a7c731371d0ba89', 'request_id': 'req-b52db798-6e89-46f4-b5e3-af107b8b984d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.975670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79ec6a-fba0-499e-8906-5cea02c27b8e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:53.984361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '452cae40661345769a7c731371d0ba89', 'request_id': 'req-b52db798-6e89-46f4-b5e3-af107b8b984d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:53.997810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:54.011405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] The token cache is being invalidate because project 452cae40661345769a7c731371d0ba89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:52:54.012165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:54.013763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b52db798-6e89-46f4-b5e3-af107b8b984d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:52:54.017706 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1399/2804] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:52:53 2022] DELETE /identity/v3/projects/452cae40661345769a7c731371d0ba89 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:54.026076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.028549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.029620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.089595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:54.092466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.096926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a37f6-837b-465b-8473-fcd76f167e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.097618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a37f6-837b-465b-8473-fcd76f167e3d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.098567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a37f6-837b-465b-8473-fcd76f167e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.098758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a37f6-837b-465b-8473-fcd76f167e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.099610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a37f6-837b-465b-8473-fcd76f167e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.102994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a37f6-837b-465b-8473-fcd76f167e3d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.110548 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1400/2805] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:54.169862 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1406/2806] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:53 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:54.178054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.184674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.208762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027904d-d9bd-47c1-906c-60236c63d9be None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.209202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027904d-d9bd-47c1-906c-60236c63d9be None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.209320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027904d-d9bd-47c1-906c-60236c63d9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.232761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ca171-b803-46fc-9dcc-506c42579a4b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.255239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3027904d-d9bd-47c1-906c-60236c63d9be None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:54.258131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3027904d-d9bd-47c1-906c-60236c63d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.350103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.352123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.352620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.353081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.353943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.373663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.385037 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1407/2807] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:54.390317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027904d-d9bd-47c1-906c-60236c63d9be None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.401656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027904d-d9bd-47c1-906c-60236c63d9be None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.409222 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1401/2808] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:54.412894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.414673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.415303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.416016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.416301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.417144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:52:54.417648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.419547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.430012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09df1e8-bd4f-4436-bf36-ad30d6ef6151 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.438558 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1408/2809] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:52:54.440199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.451774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ecedd-d3b2-4193-80f4-bfee60f6ea6e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.452261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.453084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.454170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.454350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.454888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.456660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.466846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.468068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.468358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.468630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.469435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.474264 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1402/2810] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:52:54.474438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.475853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.489922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb5e0-951f-4a82-9328-60d81b1268a6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.511188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1409/2811] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/projects => generated 355 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:54.515209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.523082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.524832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.525163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.525602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.526968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.530179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.564221 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1403/2812] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:54.568023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.586844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa9f54-ecdb-47f2-8df0-3c71f6afd397 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.598334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3206-03d0-4cad-a6d8-0a9565bfb7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.601670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582f050-5aa9-4a33-8ab3-c337b4b27f39 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.602240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3206-03d0-4cad-a6d8-0a9565bfb7da admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.602240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3206-03d0-4cad-a6d8-0a9565bfb7da admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.603235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3206-03d0-4cad-a6d8-0a9565bfb7da admin admin] PATH_INFO: `/v3/projects/ad160a4b6a31471a82c069d1c6844203/users/8c59c2cab6eb46edb2b2482f70813740/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.608138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.609291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.611165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.611669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.612507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.615873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.630114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:54.640640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3206-03d0-4cad-a6d8-0a9565bfb7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad160a4b6a31471a82c069d1c6844203, user_id=8c59c2cab6eb46edb2b2482f70813740, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.647273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3206-03d0-4cad-a6d8-0a9565bfb7da admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.653573 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1410/2813] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/users => generated 278 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:52:54.659051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.670692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7d2b8-d797-45ec-b79c-ae3fe960beb5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.681180 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1404/2814] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:54 2022] PUT /identity/v3/projects/ad160a4b6a31471a82c069d1c6844203/users/8c59c2cab6eb46edb2b2482f70813740/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:54.690045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.690367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.690969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.718601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:54.720581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.758484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.759338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.759810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.760308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.761205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.763751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.778721 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1411/2815] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:54.783791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.801390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aad8a1-538f-49c0-8ad9-bf8bcbb016f2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.812201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:54.819422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.820619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.820619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.820619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.821227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.822871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.845923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1412/2816] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:54.847304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.875052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662b2f8-9433-4190-9aa0-4e10355a40f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:54.877653 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1405/2817] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:54.887153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb2e4f-602c-48a3-be2b-2fd80310f616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:54.887766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb2e4f-602c-48a3-be2b-2fd80310f616 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.888050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb2e4f-602c-48a3-be2b-2fd80310f616 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.888334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb2e4f-602c-48a3-be2b-2fd80310f616 admin admin] PATH_INFO: `/v3/projects/7a8797f8deb14dd38be2421e17032a5d/users/ce9139872ad54cc6a2fc98872381ef90/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:54.904095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:54.906398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:54.929899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb2e4f-602c-48a3-be2b-2fd80310f616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8797f8deb14dd38be2421e17032a5d, user_id=ce9139872ad54cc6a2fc98872381ef90, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:54.935403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb2e4f-602c-48a3-be2b-2fd80310f616 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:54.976999 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1413/2818] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:54 2022] PUT /identity/v3/projects/7a8797f8deb14dd38be2421e17032a5d/users/ce9139872ad54cc6a2fc98872381ef90/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:54.983191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daf33e-ce81-4e18-8cf8-bc47998b3bd2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:54.986163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daf33e-ce81-4e18-8cf8-bc47998b3bd2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:54.987078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daf33e-ce81-4e18-8cf8-bc47998b3bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.074628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14f3da-fd8b-491e-8f27-ebfd71c13dce None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:55.089199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-70daf33e-ce81-4e18-8cf8-bc47998b3bd2 None None] MFA Rules not processed for user `e01d3916c09a484eb3a687ba732162b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:55.093396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-70daf33e-ce81-4e18-8cf8-bc47998b3bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:55.191281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:55.191907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.192243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.192475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.254983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1414/2819] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:55.256343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95827-ec81-4d0f-b400-743de50282b4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.256634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95827-ec81-4d0f-b400-743de50282b4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.256905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95827-ec81-4d0f-b400-743de50282b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.302168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-caa95827-ec81-4d0f-b400-743de50282b4 None None] MFA Rules not processed for user `ce9139872ad54cc6a2fc98872381ef90`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:55.304137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-caa95827-ec81-4d0f-b400-743de50282b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:55.395528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:55.408967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:55.433446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:55.441374 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1406/2820] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:55.442304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:55.447558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:55.510780 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1415/2821] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:55.515802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa95827-ec81-4d0f-b400-743de50282b4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:55.533720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa95827-ec81-4d0f-b400-743de50282b4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:55.539579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:55.540211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.540499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.540774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.541988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:55.543940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:55.549767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:55.590182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5712d3-9956-40e7-ad72-59c4ef8b0e28 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:55.599537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:55.600567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.601075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.601801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.612625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:55.614042 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1416/2822] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:52:55 2022] POST /identity/v3/projects => generated 361 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:55.615105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:55.617755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:55.631471 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1407/2823] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:55 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:52:55.765101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa5e05-466f-4764-9172-98500aa77fdd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:55.777671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:55.778699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.780535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.780974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.783112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:55.787236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:55.838288 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1417/2824] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:52:55 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:52:55.842529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:55.858029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8eae6b-849e-4e3b-a52a-e090f281aabc service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:55.874040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:55.876265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.876762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.884721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:55.888518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:55.890305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:55.916596 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1408/2825] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:55.921041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:55.937108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba1c7f-8392-444f-928c-43c4145a8b31 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:55.950362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:55.952481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:55.953330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:55.953330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] PATH_INFO: `/v3/projects/1103575e884f41a0a08dc997fbc390c9/users/2363ca9a1a0e4af49c8794214075c9f9/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:56.003609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1103575e884f41a0a08dc997fbc390c9, user_id=2363ca9a1a0e4af49c8794214075c9f9, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:56.019323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:56.057739 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1418/2826] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:55 2022] PUT /identity/v3/projects/1103575e884f41a0a08dc997fbc390c9/users/2363ca9a1a0e4af49c8794214075c9f9/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:52:56.061568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:56.084148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365dddb3-68f2-4638-9341-a3b4579e4137 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:56.290718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fd1dd-c0c2-4b99-9d96-9dbee28962b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:56.291354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fd1dd-c0c2-4b99-9d96-9dbee28962b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:56.291641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fd1dd-c0c2-4b99-9d96-9dbee28962b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:56.291932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fd1dd-c0c2-4b99-9d96-9dbee28962b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:56.292578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fd1dd-c0c2-4b99-9d96-9dbee28962b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:56.294388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fd1dd-c0c2-4b99-9d96-9dbee28962b7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:56.331013 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1409/2827] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:52:56.331352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:56.346253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed81f47-96c6-435f-90c5-87bfce76fd9b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:56.371046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:56.372170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:56.372683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:56.374231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] PATH_INFO: `/v3/projects/1103575e884f41a0a08dc997fbc390c9/users/2363ca9a1a0e4af49c8794214075c9f9/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:56.384003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1103575e884f41a0a08dc997fbc390c9, user_id=2363ca9a1a0e4af49c8794214075c9f9, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:56.394729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:56.434266 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1419/2828] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:52:56 2022] PUT /identity/v3/projects/1103575e884f41a0a08dc997fbc390c9/users/2363ca9a1a0e4af49c8794214075c9f9/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:52:56.439336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ac7a4-39ca-42a3-919b-77ab6e02c26e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:56.440420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ac7a4-39ca-42a3-919b-77ab6e02c26e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:56.441013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ac7a4-39ca-42a3-919b-77ab6e02c26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:56.508161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1a3ac7a4-39ca-42a3-919b-77ab6e02c26e None None] MFA Rules not processed for user `2363ca9a1a0e4af49c8794214075c9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:52:56.511982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a3ac7a4-39ca-42a3-919b-77ab6e02c26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:56.676771 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1410/2829] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:52:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:52:57.135262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:52:57.139471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:52:57.250667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:52:57.261745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538959e-f9f6-4413-8ef2-3b925a8b2aea admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:52:57.270424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59539b1e-bf4b-47ba-aaad-9b6061cc8462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:52:57.271058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59539b1e-bf4b-47ba-aaad-9b6061cc8462 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:52:57.271343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59539b1e-bf4b-47ba-aaad-9b6061cc8462 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:52:57.271729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59539b1e-bf4b-47ba-aaad-9b6061cc8462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:52:57.281783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59539b1e-bf4b-47ba-aaad-9b6061cc8462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:52:57.283821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59539b1e-bf4b-47ba-aaad-9b6061cc8462 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:52:57.299471 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1420/2830] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:52:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:00.001601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ac7a4-39ca-42a3-919b-77ab6e02c26e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:00.104617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:00.137182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ac7a4-39ca-42a3-919b-77ab6e02c26e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:00.144619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:00.145617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:00.146197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:00.146697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] PATH_INFO: `/v3/users/ce9139872ad54cc6a2fc98872381ef90` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:00.148101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9139872ad54cc6a2fc98872381ef90)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:00.157028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:00.286384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9139872ad54cc6a2fc98872381ef90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:00.335705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9139872ad54cc6a2fc98872381ef90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:00.349545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9139872ad54cc6a2fc98872381ef90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:00.362825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9139872ad54cc6a2fc98872381ef90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:00.365949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9139872ad54cc6a2fc98872381ef90'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:00.419301 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1411/2831] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:00 2022] DELETE /identity/v3/users/ce9139872ad54cc6a2fc98872381ef90 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:00.420774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:00.421667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:00.422371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:00.464693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:00.472577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:00.627164 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1421/2832] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:00.671597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:00.674267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:00.793322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534638b-f979-4e8d-a1f7-1eaffbd9442c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:00.797716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:00.798402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:00.798888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:00.799176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:00.815479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:00.817277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:00.837546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1412/2833] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:02.628122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:02.634104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:02.654867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c02cb-a349-4f99-8a58-b4869df977b9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:02.661168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:02.662178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:02.662739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:02.663253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] PATH_INFO: `/v3/projects/7a8797f8deb14dd38be2421e17032a5d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:02.664971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8797f8deb14dd38be2421e17032a5d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:02.667812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:02.784098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8797f8deb14dd38be2421e17032a5d', 'request_id': 'req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:02.794592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8797f8deb14dd38be2421e17032a5d', 'request_id': 'req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:02.818260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8797f8deb14dd38be2421e17032a5d', 'request_id': 'req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:02.839012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] The token cache is being invalidate because project 7a8797f8deb14dd38be2421e17032a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:02.839367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:02.841299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1dd9b6f9-59c5-40b0-8ed2-58f1b36f418f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:02.853887 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1422/2834] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:02 2022] DELETE /identity/v3/projects/7a8797f8deb14dd38be2421e17032a5d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:02.857742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:02.870806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbab972-2f2c-47de-af6b-9aa7437fa517 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:03.016609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:03.066341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:03.066989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:03.067276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:03.067558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] PATH_INFO: `/v3/users/2363ca9a1a0e4af49c8794214075c9f9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:03.080617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2363ca9a1a0e4af49c8794214075c9f9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:03.082647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:03.145722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:03.146579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:03.147010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:03.183419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] MFA Rules not processed for user `dd431fc2d9834bea8df807edb91771ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:03.187399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:03.282611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2363ca9a1a0e4af49c8794214075c9f9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:03.302608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2363ca9a1a0e4af49c8794214075c9f9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:03.303565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2363ca9a1a0e4af49c8794214075c9f9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:03.324568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2363ca9a1a0e4af49c8794214075c9f9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:03.325964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b2441a5-34f1-4fcf-909d-520897ab9745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2363ca9a1a0e4af49c8794214075c9f9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:03.353253 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1423/2835] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:03.377443 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1413/2836] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:02 2022] DELETE /identity/v3/users/2363ca9a1a0e4af49c8794214075c9f9 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:53:05.030403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:05.034509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:05.109057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973de704-94e6-4fa1-9b3f-8add34acba5a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:05.178263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:05.320184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:05.320858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:05.321141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:05.321417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] PATH_INFO: `/v3/projects/1103575e884f41a0a08dc997fbc390c9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:05.332636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] RBAC: Authorizing `identity:delete_project(project_id=1103575e884f41a0a08dc997fbc390c9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:05.334664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:05.424623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1103575e884f41a0a08dc997fbc390c9', 'request_id': 'req-6f54f92c-7479-4d65-b57e-d1ac8659deef'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:05.436429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1103575e884f41a0a08dc997fbc390c9', 'request_id': 'req-6f54f92c-7479-4d65-b57e-d1ac8659deef'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:05.447467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1103575e884f41a0a08dc997fbc390c9', 'request_id': 'req-6f54f92c-7479-4d65-b57e-d1ac8659deef'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:05.481123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] The token cache is being invalidate because project 1103575e884f41a0a08dc997fbc390c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:05.482555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:05.483703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f54f92c-7479-4d65-b57e-d1ac8659deef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:05.486858 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1424/2837] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:05 2022] DELETE /identity/v3/projects/1103575e884f41a0a08dc997fbc390c9 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:05.491920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be739cb9-65ac-4f70-9275-f9c27c3149d7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:05.492833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be739cb9-65ac-4f70-9275-f9c27c3149d7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:05.493262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be739cb9-65ac-4f70-9275-f9c27c3149d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:05.567848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-be739cb9-65ac-4f70-9275-f9c27c3149d7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:05.570221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be739cb9-65ac-4f70-9275-f9c27c3149d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:05.799576 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1414/2838] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:05.808643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f24cc8-6de8-4b72-b45b-06440f8df0b0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:05.809107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f24cc8-6de8-4b72-b45b-06440f8df0b0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:05.809396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f24cc8-6de8-4b72-b45b-06440f8df0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:05.878499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a8f24cc8-6de8-4b72-b45b-06440f8df0b0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:05.879753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a8f24cc8-6de8-4b72-b45b-06440f8df0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:05.904964 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1425/2839] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:05.909593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be739cb9-65ac-4f70-9275-f9c27c3149d7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:05.930203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be739cb9-65ac-4f70-9275-f9c27c3149d7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:05.946631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:05.947696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:05.947989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:05.948266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:05.948925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:53:05.949231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:05.952496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:06.005922 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1415/2840] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:53:06.010669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f24cc8-6de8-4b72-b45b-06440f8df0b0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:06.024007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f24cc8-6de8-4b72-b45b-06440f8df0b0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:06.051377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:06.052040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:06.052281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:06.058696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:06.061931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:06.070779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:06.167665 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1426/2841] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:06 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:06.172363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:06.186715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad522a-ca06-4cd8-8172-d1cabb56c9da admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:06.288188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:06.288819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:06.289155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:06.289435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:06.291777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:06.295540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:06.317706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:06.342273 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1416/2842] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:06 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:06.347332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:06.363922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5141f-196d-4f91-b59f-16c2e4ec9528 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:06.376102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:06.379462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:06.379972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:06.380600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:06.382933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:06.386738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:06.413485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1427/2843] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:06.418107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:06.432849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dfb11-959f-40b2-ba8e-f17f963688f0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:06.448221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb167c-1a22-4f3e-88ba-73400ef5f835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:06.449626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb167c-1a22-4f3e-88ba-73400ef5f835 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:06.449626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb167c-1a22-4f3e-88ba-73400ef5f835 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:06.449626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb167c-1a22-4f3e-88ba-73400ef5f835 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:06.450149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb167c-1a22-4f3e-88ba-73400ef5f835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:06.457988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb167c-1a22-4f3e-88ba-73400ef5f835 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:06.481961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1417/2844] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:06.484896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:06.497603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01d0e6-9e08-4682-8700-dd26ca590d44 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:06.504713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:06.506737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:06.507217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:06.507656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] PATH_INFO: `/v3/projects/aa06d42811af49009e65a61d918ac49b/users/bdda19c789334355a1cf6160d172ef2c/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:06.548004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa06d42811af49009e65a61d918ac49b, user_id=bdda19c789334355a1cf6160d172ef2c, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:06.554608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:06.603366 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1428/2845] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:06 2022] PUT /identity/v3/projects/aa06d42811af49009e65a61d918ac49b/users/bdda19c789334355a1cf6160d172ef2c/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:06.616385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb0b29-898b-45d0-9b76-9a3dc3fb3352 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:06.616832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb0b29-898b-45d0-9b76-9a3dc3fb3352 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:06.616936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb0b29-898b-45d0-9b76-9a3dc3fb3352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:06.711924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c8cb0b29-898b-45d0-9b76-9a3dc3fb3352 None None] MFA Rules not processed for user `bdda19c789334355a1cf6160d172ef2c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:06.713442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8cb0b29-898b-45d0-9b76-9a3dc3fb3352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:06.916300 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1418/2846] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:06.920373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:06.999020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:07.020306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9b4f2-6769-4995-ac58-619916a3130b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:07.035750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:07.036502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:07.040215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:07.041395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:07.042664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:07.044871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:07.135635 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1429/2847] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:06 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:53:07.140266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb0b29-898b-45d0-9b76-9a3dc3fb3352 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:07.163436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb0b29-898b-45d0-9b76-9a3dc3fb3352 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:07.340374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:07.341972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:07.343231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:07.344321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:07.345766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:07.353686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:07.413641 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1419/2848] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:07 2022] POST /identity/v3/users => generated 300 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:07.415965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:07.441430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dce64-f8ac-49b6-8b30-0995eee4bbac admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:07.456525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:07.458429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:07.459834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:07.462288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:07.463818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:07.471973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:07.511870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1430/2849] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:07.513928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:07.540611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3d27c-4e28-4e27-9963-0c7c85ded55f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:07.555756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:07.556625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:07.556919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:07.557795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] PATH_INFO: `/v3/projects/781e788e55f744ee8715ec6f47da62ac/users/c118c91327a749788a97c6cde5591a53/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:07.601104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781e788e55f744ee8715ec6f47da62ac, user_id=c118c91327a749788a97c6cde5591a53, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:07.604313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:07.643046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1420/2850] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:07 2022] PUT /identity/v3/projects/781e788e55f744ee8715ec6f47da62ac/users/c118c91327a749788a97c6cde5591a53/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:07.648362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:07.664562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1cdf6-cb3c-4482-a766-a5e0104edb6a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:07.846969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a131021-873e-467c-865c-0af79e5edee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:07.847610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a131021-873e-467c-865c-0af79e5edee4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:07.847899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a131021-873e-467c-865c-0af79e5edee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:07.848179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a131021-873e-467c-865c-0af79e5edee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:07.848840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a131021-873e-467c-865c-0af79e5edee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:07.850503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a131021-873e-467c-865c-0af79e5edee4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:07.865200 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1431/2851] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:07.870115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:07.900655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd9db7b-3b41-4572-9d63-1d6df6170832 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:07.917519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:07.918370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:07.920278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:07.921616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] PATH_INFO: `/v3/projects/781e788e55f744ee8715ec6f47da62ac/users/c118c91327a749788a97c6cde5591a53/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:07.928421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781e788e55f744ee8715ec6f47da62ac, user_id=c118c91327a749788a97c6cde5591a53, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:07.940687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:07.991281 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1421/2852] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:07 2022] PUT /identity/v3/projects/781e788e55f744ee8715ec6f47da62ac/users/c118c91327a749788a97c6cde5591a53/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:08.008919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569c473-8636-4fe6-8dca-a22565487082 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:08.009911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569c473-8636-4fe6-8dca-a22565487082 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:08.010351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569c473-8636-4fe6-8dca-a22565487082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:08.103126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2569c473-8636-4fe6-8dca-a22565487082 None None] MFA Rules not processed for user `c118c91327a749788a97c6cde5591a53`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:08.109757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2569c473-8636-4fe6-8dca-a22565487082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:08.323488 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1432/2853] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:08.335585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:08.337571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:08.393653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:08.475803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a861be-e601-41b5-a1ec-2faa53a6bce0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:08.487010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:08.488774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:08.490131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:08.491840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:08.531737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:08.531737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:08.598088 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1422/2854] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:08.612592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:08.612882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:08.613199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:08.637299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] MFA Rules not processed for user `870c4c66a15b4546bd981cfb5932b214`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:08.639140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:08.713914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:08.766896 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1433/2855] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:09.539214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:09.707194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca092ac-c278-43b9-b23b-54169fa5b091 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:09.715870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:09.717618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:09.736874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:09.737527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] PATH_INFO: `/v3/users/bdda19c789334355a1cf6160d172ef2c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:09.745705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdda19c789334355a1cf6160d172ef2c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:09.751476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:09.871001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:09.876298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:09.925576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdda19c789334355a1cf6160d172ef2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:09.926579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ba81-2f8f-4364-9393-844d72d70590 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:09.941688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:09.942383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:09.945542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:09.945895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] PATH_INFO: `/v3/users/e7392a7b7d474216b89feb16c75e2f63` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:09.946524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdda19c789334355a1cf6160d172ef2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:09.947253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdda19c789334355a1cf6160d172ef2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:09.947745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7392a7b7d474216b89feb16c75e2f63)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:09.950421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:09.957185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdda19c789334355a1cf6160d172ef2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:09.958286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdda19c789334355a1cf6160d172ef2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.025779 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1423/2856] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:09 2022] DELETE /identity/v3/users/bdda19c789334355a1cf6160d172ef2c => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:10.029076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:10.035727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.124121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7392a7b7d474216b89feb16c75e2f63'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.163276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7392a7b7d474216b89feb16c75e2f63'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.174864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7392a7b7d474216b89feb16c75e2f63'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.187878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7392a7b7d474216b89feb16c75e2f63'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.189399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1431ecf9-3084-4ff1-92c8-b4996a1e85ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7392a7b7d474216b89feb16c75e2f63'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.201012 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1434/2857] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:09 2022] DELETE /identity/v3/users/e7392a7b7d474216b89feb16c75e2f63 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:53:10.204469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.204757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.208816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.217121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:10.291705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:10.294196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.330309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac23e3-53ce-436c-b885-eaab471d3043 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:10.486430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:10.490280 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1435/2858] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:10.490280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:10.491955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.492949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.494475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.496085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.497543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.645133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:10.684194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19419574-a23a-4764-b513-dd79a3274b2b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:10.696500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:10.698982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.699295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.699579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.705162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.713132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:10.715710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:10.733103 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1424/2859] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:10.733495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:10.735430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:10.737885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:10.740875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.767350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee3f1b-3422-4c21-a547-45f32ce2b075 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:10.768005 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1436/2860] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:10.768983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:10.770502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:10.783473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515c70-91ce-4189-afd4-ba8b4451d01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:10.784957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515c70-91ce-4189-afd4-ba8b4451d01c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.786260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515c70-91ce-4189-afd4-ba8b4451d01c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.786718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515c70-91ce-4189-afd4-ba8b4451d01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.788278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d4ff5-3b64-4e9d-91f5-2b2ce610264a service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:10.793706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:10.794553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:10.794856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:10.795202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] PATH_INFO: `/v3/projects/afb37d675816472db726662b9f0c5994` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:10.797307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb37d675816472db726662b9f0c5994)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:10.799483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:10.815229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515c70-91ce-4189-afd4-ba8b4451d01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:10.817450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515c70-91ce-4189-afd4-ba8b4451d01c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:10.844692 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1425/2861] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:10.908964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb37d675816472db726662b9f0c5994', 'request_id': 'req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.933470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb37d675816472db726662b9f0c5994', 'request_id': 'req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:10.951652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb37d675816472db726662b9f0c5994', 'request_id': 'req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.000986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] The token cache is being invalidate because project afb37d675816472db726662b9f0c5994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:11.001302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.002114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1f57cbe0-adf6-4b40-bb2f-6521c733eb4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.004499 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1437/2862] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:10 2022] DELETE /identity/v3/projects/afb37d675816472db726662b9f0c5994 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:11.012535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984330c-605e-4a0d-9eb2-4169c19d1546 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.012848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984330c-605e-4a0d-9eb2-4169c19d1546 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.013123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984330c-605e-4a0d-9eb2-4169c19d1546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.050585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3984330c-605e-4a0d-9eb2-4169c19d1546 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:11.051989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3984330c-605e-4a0d-9eb2-4169c19d1546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:11.239430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6accd-e0e8-4aca-bd8b-ac51e7fe368b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.239757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6accd-e0e8-4aca-bd8b-ac51e7fe368b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.240033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6accd-e0e8-4aca-bd8b-ac51e7fe368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.241038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1426/2863] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:11.323828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-36c6accd-e0e8-4aca-bd8b-ac51e7fe368b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:11.325087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-36c6accd-e0e8-4aca-bd8b-ac51e7fe368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:11.360307 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1438/2864] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:11.361790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984330c-605e-4a0d-9eb2-4169c19d1546 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.375473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984330c-605e-4a0d-9eb2-4169c19d1546 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:11.393507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:11.394235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.394923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.395409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.396445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:53:11.397500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:11.400319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:11.421028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:11.434983 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1427/2865] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:53:11.436364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6accd-e0e8-4aca-bd8b-ac51e7fe368b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.451340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6accd-e0e8-4aca-bd8b-ac51e7fe368b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:11.468398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:11.469367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.470111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.470364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.471015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:11.473363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:11.503564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.505998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:11.540681 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1439/2866] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:11 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:11.542763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.548925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b79e6-5bb3-405e-b9f3-44454add7f8b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:11.708635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140c037-2812-49f3-b46a-5977a07c9472 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:11.726561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:11.730817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.731169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.731443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.737603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:11.743379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:11.754961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:11.755631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.755923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.756229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] PATH_INFO: `/v3/projects/aa06d42811af49009e65a61d918ac49b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.759190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa06d42811af49009e65a61d918ac49b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:11.761683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:11.787847 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1440/2867] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:11 2022] POST /identity/v3/users => generated 296 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:11.792042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.805947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc57985-1757-4d97-bb69-20bc67051172 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:11.812493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:11.813575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.813993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.815017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.815916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:11.818541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:11.832805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1441/2868] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:11.837417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.860920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23333d5-8669-4f81-abf9-a108dc963154 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:11.862854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa06d42811af49009e65a61d918ac49b', 'request_id': 'req-32fbcd88-500a-4109-a5d3-942a6810ebc7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.882069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa06d42811af49009e65a61d918ac49b', 'request_id': 'req-32fbcd88-500a-4109-a5d3-942a6810ebc7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.894524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa06d42811af49009e65a61d918ac49b', 'request_id': 'req-32fbcd88-500a-4109-a5d3-942a6810ebc7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.902081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:11.930748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] The token cache is being invalidate because project aa06d42811af49009e65a61d918ac49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:11.932577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.934236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:11.939287 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1428/2869] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:11 2022] DELETE /identity/v3/projects/aa06d42811af49009e65a61d918ac49b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:53:11.954239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.967450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:11.976226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:11.976226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:11.976226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:11.976226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:11.976975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:11.976975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:11.993105 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1442/2870] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:53:11.997178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:11.999275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.002298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbcd88-500a-4109-a5d3-942a6810ebc7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:12.009846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:12.012035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.012416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.012691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] PATH_INFO: `/v3/users/c118c91327a749788a97c6cde5591a53` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.014555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c118c91327a749788a97c6cde5591a53)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:12.017467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:12.019582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8f936-8827-46f5-8b82-8c08f8a8603e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:12.034763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103f05a-7782-423c-b556-8b0ef86eb440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:12.035580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103f05a-7782-423c-b556-8b0ef86eb440 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.035961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103f05a-7782-423c-b556-8b0ef86eb440 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.036476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103f05a-7782-423c-b556-8b0ef86eb440 admin admin] PATH_INFO: `/v3/projects/f4b1732129f3414fbc3f26fff6d039ec/users/d8b0a160808f4adda7be57cd535f0631/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.059731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103f05a-7782-423c-b556-8b0ef86eb440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b1732129f3414fbc3f26fff6d039ec, user_id=d8b0a160808f4adda7be57cd535f0631, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:12.065770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103f05a-7782-423c-b556-8b0ef86eb440 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:12.097660 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1443/2871] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:11 2022] PUT /identity/v3/projects/f4b1732129f3414fbc3f26fff6d039ec/users/d8b0a160808f4adda7be57cd535f0631/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:12.099900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f54b4-3729-4a84-bcd3-f3a2e0f479e7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.100431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f54b4-3729-4a84-bcd3-f3a2e0f479e7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.101390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f54b4-3729-4a84-bcd3-f3a2e0f479e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.137103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c118c91327a749788a97c6cde5591a53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:12.149233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d9f54b4-3729-4a84-bcd3-f3a2e0f479e7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:12.153216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c118c91327a749788a97c6cde5591a53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:12.154001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d9f54b4-3729-4a84-bcd3-f3a2e0f479e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.154505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c118c91327a749788a97c6cde5591a53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:12.168816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c118c91327a749788a97c6cde5591a53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:12.169753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c118c91327a749788a97c6cde5591a53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:12.265160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1429/2872] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:11 2022] DELETE /identity/v3/users/c118c91327a749788a97c6cde5591a53 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:12.349164 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1444/2873] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:12.371195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:12.372845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.379568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:12.527748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f54b4-3729-4a84-bcd3-f3a2e0f479e7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:12.604389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4763031-93ee-46d6-8aef-286d93055abe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:12.618571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:12.619223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.619508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.624604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.644909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f54b4-3729-4a84-bcd3-f3a2e0f479e7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:12.653941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdd735-34c3-4098-88f7-77582f65a5ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:12.654651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdd735-34c3-4098-88f7-77582f65a5ae service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:12.654935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdd735-34c3-4098-88f7-77582f65a5ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:12.655216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdd735-34c3-4098-88f7-77582f65a5ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:12.659472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-18bdd735-34c3-4098-88f7-77582f65a5ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:12.804358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:12.855145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:12.857379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:12.875492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdd735-34c3-4098-88f7-77582f65a5ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:12.877479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdd735-34c3-4098-88f7-77582f65a5ae service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:12.882786 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1430/2874] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:12.903908 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1445/2875] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:13.282702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:13.284468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:13.310738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1fd4-f70c-46bf-8196-f045fcd540c4 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:13.322053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:13.322712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:13.323000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:13.323291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] PATH_INFO: `/v3/projects/781e788e55f744ee8715ec6f47da62ac` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:13.327102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=781e788e55f744ee8715ec6f47da62ac)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:13.330418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:13.340406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:13.341426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:13.342099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:13.450929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] MFA Rules not processed for user `5053eb274d2c4e8180aa1cbcb680a94f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:13.456757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:13.535240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781e788e55f744ee8715ec6f47da62ac', 'request_id': 'req-b352d109-51b5-4725-a3ca-5931cd16d69e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:13.555742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781e788e55f744ee8715ec6f47da62ac', 'request_id': 'req-b352d109-51b5-4725-a3ca-5931cd16d69e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:13.581518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781e788e55f744ee8715ec6f47da62ac', 'request_id': 'req-b352d109-51b5-4725-a3ca-5931cd16d69e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:13.604612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] The token cache is being invalidate because project 781e788e55f744ee8715ec6f47da62ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:13.607025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:13.609770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b352d109-51b5-4725-a3ca-5931cd16d69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:13.614514 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1431/2876] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:13 2022] DELETE /identity/v3/projects/781e788e55f744ee8715ec6f47da62ac => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:13.620446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707c8e-cf9c-4407-9a12-9b73987dde2f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:13.621229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707c8e-cf9c-4407-9a12-9b73987dde2f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:13.622136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5707c8e-cf9c-4407-9a12-9b73987dde2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:13.671751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f5707c8e-cf9c-4407-9a12-9b73987dde2f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:13.673127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f5707c8e-cf9c-4407-9a12-9b73987dde2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:13.703866 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1446/2877] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:13.719621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:13.721736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:13.747843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:13.753319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506a0a1-835c-47aa-b1a5-05e2dd43378d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:13.810545 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1432/2878] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:13.818924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff037c6-c109-414e-9128-2d4250cdcea3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:13.819316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff037c6-c109-414e-9128-2d4250cdcea3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:13.819631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff037c6-c109-414e-9128-2d4250cdcea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:13.879799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4ff037c6-c109-414e-9128-2d4250cdcea3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:13.881351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4ff037c6-c109-414e-9128-2d4250cdcea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:13.894188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1433/2879] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:13.899339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff037c6-c109-414e-9128-2d4250cdcea3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:13.934283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff037c6-c109-414e-9128-2d4250cdcea3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:13.943619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:13.944256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:13.944762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:13.945104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:13.947955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:13.950265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:13.951468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:13.954215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:13.955401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:53:13.956539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:13.965631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:14.007905 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1434/2880] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:53:14.012520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:14.040174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609fcf4-fd84-405a-81fc-26c7343b746d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:14.056567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:14.059509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.060964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.061793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.063718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:14.065089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:14.069291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:14.073165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:14.076422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:14.121765 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1447/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:14.155448 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1435/2882] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:14 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:53:14.160245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:14.181276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68844efa-c4b3-40cb-a557-8684f16d8643 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:14.406138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:14.407141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.407141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.407426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.408079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:14.412208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:14.465984 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1448/2883] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:14 2022] POST /identity/v3/users => generated 295 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:14.468531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:14.492458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1394b0-fb18-418b-b38a-f00c3849b4c1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:14.504323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:14.506435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.506727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.507043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.509175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:14.513248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:14.539660 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1436/2884] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:14.540540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:14.555892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839deff-55ab-4461-a3e3-80fd1d7c1a4c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:14.574280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5050ce-47ab-4365-9132-2d6ed69c3800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:14.574280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5050ce-47ab-4365-9132-2d6ed69c3800 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.574280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5050ce-47ab-4365-9132-2d6ed69c3800 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.575320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5050ce-47ab-4365-9132-2d6ed69c3800 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.575660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5050ce-47ab-4365-9132-2d6ed69c3800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:14.577360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5050ce-47ab-4365-9132-2d6ed69c3800 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:14.615438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1449/2885] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:14.616675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:14.629620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aee0bc-21f1-4a17-9d90-f02cef23f226 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:14.647431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:14.648137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.648404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.648728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] PATH_INFO: `/v3/projects/70fb988b59aa4806b77ca61de1d6d57e/users/f340cba48e624334bf7da93b5240f3b9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.682875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fb988b59aa4806b77ca61de1d6d57e, user_id=f340cba48e624334bf7da93b5240f3b9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:14.685626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:14.714402 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1437/2886] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:14 2022] PUT /identity/v3/projects/70fb988b59aa4806b77ca61de1d6d57e/users/f340cba48e624334bf7da93b5240f3b9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:53:14.725695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c8588-48c2-458f-bdae-3aec06952ae5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:14.726064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c8588-48c2-458f-bdae-3aec06952ae5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:14.726351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c8588-48c2-458f-bdae-3aec06952ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:14.788910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-381c8588-48c2-458f-bdae-3aec06952ae5 None None] MFA Rules not processed for user `f340cba48e624334bf7da93b5240f3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:14.801851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-381c8588-48c2-458f-bdae-3aec06952ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:15.119330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1450/2887] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:15.121744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:15.277426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecd6ec-cf1e-4193-b7e4-72b6ca474ab9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:15.301009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:15.303347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:15.303705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:15.304273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:15.305209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:15.310844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:15.385676 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1438/2888] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:15 2022] POST /identity/v3/projects => generated 391 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:15.389568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c8588-48c2-458f-bdae-3aec06952ae5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:15.411286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c8588-48c2-458f-bdae-3aec06952ae5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:15.618541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:15.619321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:15.619474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:15.619695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:15.620372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:15.627007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:15.666066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:15.678152 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1451/2889] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:15 2022] POST /identity/v3/users => generated 302 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:15.679426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:15.700400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785b55f-18ad-4b90-b2ac-48678e92bc32 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:15.721217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699693c1-44dd-4e84-b3b3-b4af6055b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:15.725795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699693c1-44dd-4e84-b3b3-b4af6055b32c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:15.727128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699693c1-44dd-4e84-b3b3-b4af6055b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:15.727516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699693c1-44dd-4e84-b3b3-b4af6055b32c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:15.728304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699693c1-44dd-4e84-b3b3-b4af6055b32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:15.736883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699693c1-44dd-4e84-b3b3-b4af6055b32c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:15.802452 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1439/2890] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:53:15.807699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:15.833494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a0be2-3092-4247-80c4-35718cc22d8e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:15.847511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:15.849805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:15.850140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:15.851815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] PATH_INFO: `/v3/projects/219ffc0c09a9455f8a0e022ffcbee9a4/users/af5454f868414e729a3654cd9be1e30a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:15.872952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc8fd-5f2f-42cd-8857-ad684b8d38b2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:15.873645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc8fd-5f2f-42cd-8857-ad684b8d38b2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:15.874293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc8fd-5f2f-42cd-8857-ad684b8d38b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:15.916255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=219ffc0c09a9455f8a0e022ffcbee9a4, user_id=af5454f868414e729a3654cd9be1e30a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:15.923955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:15.964533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-41bfc8fd-5f2f-42cd-8857-ad684b8d38b2 None None] MFA Rules not processed for user `8c59c2cab6eb46edb2b2482f70813740`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:15.966305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-41bfc8fd-5f2f-42cd-8857-ad684b8d38b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:15.970557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1452/2891] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:15 2022] PUT /identity/v3/projects/219ffc0c09a9455f8a0e022ffcbee9a4/users/af5454f868414e729a3654cd9be1e30a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:15.972049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:16.042343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:16.143268 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1440/2892] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:16.146023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e5581-9fd4-4151-a24d-b4701597812e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:16.154180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e5581-9fd4-4151-a24d-b4701597812e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:16.154180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e5581-9fd4-4151-a24d-b4701597812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:16.178764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c86e5581-9fd4-4151-a24d-b4701597812e None None] MFA Rules not processed for user `5053eb274d2c4e8180aa1cbcb680a94f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:16.180127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c86e5581-9fd4-4151-a24d-b4701597812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:16.180364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76da154-8790-4f19-aa8e-aa3782b82951 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:16.192631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:16.197882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:16.197882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:16.197882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:16.197882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:16.198900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:16.226338 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1453/2893] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:53:16.227797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:16.289611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1441/2894] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:16.296059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e5581-9fd4-4151-a24d-b4701597812e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:16.341621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e5581-9fd4-4151-a24d-b4701597812e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:16.356726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6e413-caae-4201-a261-12f13caa18d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:16.357392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6e413-caae-4201-a261-12f13caa18d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:16.357691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6e413-caae-4201-a261-12f13caa18d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:16.359477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6e413-caae-4201-a261-12f13caa18d0 admin admin] PATH_INFO: `/v3/projects/219ffc0c09a9455f8a0e022ffcbee9a4/users/af5454f868414e729a3654cd9be1e30a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:16.365581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6e413-caae-4201-a261-12f13caa18d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=219ffc0c09a9455f8a0e022ffcbee9a4, user_id=af5454f868414e729a3654cd9be1e30a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:16.368819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6e413-caae-4201-a261-12f13caa18d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:16.386023 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1442/2895] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:16 2022] PUT /identity/v3/projects/219ffc0c09a9455f8a0e022ffcbee9a4/users/af5454f868414e729a3654cd9be1e30a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:16.398668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f27402-0866-49e7-9290-07c15286d82b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:16.399239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f27402-0866-49e7-9290-07c15286d82b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:16.400049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f27402-0866-49e7-9290-07c15286d82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:16.433816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe1279-0974-44ae-ba63-c03a260d3db5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:16.473055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-70f27402-0866-49e7-9290-07c15286d82b None None] MFA Rules not processed for user `af5454f868414e729a3654cd9be1e30a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:16.490057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70f27402-0866-49e7-9290-07c15286d82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:16.639520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:16.641086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:16.641506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:16.642589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:16.654841 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1443/2896] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:16 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:53:16.655078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:16.766764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:16.770590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:16.791179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1454/2897] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:17.832047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f27402-0866-49e7-9290-07c15286d82b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:17.834959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70f27402-0866-49e7-9290-07c15286d82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:17.991468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:18.061710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f27402-0866-49e7-9290-07c15286d82b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:18.088181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:18.088856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:18.089137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:18.090014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] PATH_INFO: `/v3/users/e01d3916c09a484eb3a687ba732162b1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:18.098460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e01d3916c09a484eb3a687ba732162b1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:18.103937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:18.318710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e01d3916c09a484eb3a687ba732162b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:18.330696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e01d3916c09a484eb3a687ba732162b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:18.332990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e01d3916c09a484eb3a687ba732162b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:18.352339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e01d3916c09a484eb3a687ba732162b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:18.353249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e01d3916c09a484eb3a687ba732162b1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:18.395849 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1444/2898] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:17 2022] DELETE /identity/v3/users/e01d3916c09a484eb3a687ba732162b1 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:18.740228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:18.757268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:18.835263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8fc98-a5d2-4737-bfd0-d8b278bf72ce service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:19.085579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:19.086168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:19.086456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:19.086732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:19.089523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:19.315241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:19.329515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:19.422140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1455/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:19.709377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:19.711363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:19.886035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b403862-7c22-4f18-b745-bc3f4fa16f18 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:19.894815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:19.895920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:19.896202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:19.896477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] PATH_INFO: `/v3/projects/52dc7565b41b48b7a442810f0f21cc01` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:19.899062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52dc7565b41b48b7a442810f0f21cc01)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:19.903264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:20.007267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:20.009717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:20.019638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52dc7565b41b48b7a442810f0f21cc01', 'request_id': 'req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:20.047437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52dc7565b41b48b7a442810f0f21cc01', 'request_id': 'req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:20.071423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52dc7565b41b48b7a442810f0f21cc01', 'request_id': 'req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:20.073041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:20.114756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] The token cache is being invalidate because project 52dc7565b41b48b7a442810f0f21cc01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:20.115345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:20.116698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f06fd8b5-a1f7-4b66-b1b2-6bd0f82c5657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:20.125378 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1445/2900] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:19 2022] DELETE /identity/v3/projects/52dc7565b41b48b7a442810f0f21cc01 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:20.129148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7592-8e10-4718-b2c3-5db6b78cac9d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:20.129442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7592-8e10-4718-b2c3-5db6b78cac9d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:20.129717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7592-8e10-4718-b2c3-5db6b78cac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:20.176558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:20.190564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2e9f7592-8e10-4718-b2c3-5db6b78cac9d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:20.194329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2e9f7592-8e10-4718-b2c3-5db6b78cac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:20.204790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973da396-8cab-478d-a057-badc5436afa9 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:20.441964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:20.442966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:20.443487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:20.444045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:20.446142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:20.465065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:20.467086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:20.478422 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1446/2901] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:20.480817 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1456/2902] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:20.557448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886a57f-34b9-4655-b9ef-f541a7fe74c2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:20.557805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886a57f-34b9-4655-b9ef-f541a7fe74c2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:20.558261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886a57f-34b9-4655-b9ef-f541a7fe74c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:20.581772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c886a57f-34b9-4655-b9ef-f541a7fe74c2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:20.583427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c886a57f-34b9-4655-b9ef-f541a7fe74c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:20.621349 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1447/2903] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:20.623017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:20.656703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818058b-67be-4ead-8f72-d5a64463e94b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:20.669115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:20.670349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:20.671581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:20.672231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:20.673201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:53:20.673791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:20.675918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:20.694680 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1457/2904] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:53:20.698449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886a57f-34b9-4655-b9ef-f541a7fe74c2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:20.713290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886a57f-34b9-4655-b9ef-f541a7fe74c2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:20.725306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:20.749871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:20.750411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:20.750695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:20.751398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:20.770697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:20.898194 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1448/2905] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:20 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:20.901663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:20.921001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833ee7-c934-413b-83b7-faf3920b818d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:21.157755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:21.158525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.158721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.158994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.159646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:21.186160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:21.246787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1458/2906] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:20 2022] POST /identity/v3/users => generated 294 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:21.251536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:21.287818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e129-6f0a-408b-baba-3065a05a7fff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:21.310199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:21.314209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.314765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.315157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.316173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:21.326902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:21.382389 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1449/2907] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:21.391285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:21.410436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540c09-6be7-491a-9518-3ba639fa7baf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:21.425390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e6b39-bef2-472c-8705-98afe9ee77c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:21.427337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e6b39-bef2-472c-8705-98afe9ee77c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.427699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e6b39-bef2-472c-8705-98afe9ee77c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.427911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e6b39-bef2-472c-8705-98afe9ee77c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.431681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e6b39-bef2-472c-8705-98afe9ee77c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:21.436585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e6b39-bef2-472c-8705-98afe9ee77c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:21.475578 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1459/2908] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:21.479555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:21.495257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ad5fd-3f62-41fc-a905-83807a4b2c65 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:21.504644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:21.505341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.505629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.505960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] PATH_INFO: `/v3/projects/bbee44bfe21341d295b8c2290f692085/users/5b0c267722f14e7fa87395f796551ab5/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.548986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbee44bfe21341d295b8c2290f692085, user_id=5b0c267722f14e7fa87395f796551ab5, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:21.551783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:21.574932 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1450/2909] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:21 2022] PUT /identity/v3/projects/bbee44bfe21341d295b8c2290f692085/users/5b0c267722f14e7fa87395f796551ab5/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:21.581076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:21.581370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:21.581644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:21.625990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:21.627351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:21.894508 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1460/2910] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:22.019464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:22.024765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:22.199024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914503bb-16b4-4802-9a08-915c679cf51a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:22.211267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:22.212285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:22.212834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:22.213227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:22.238306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:22.239961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:22.269558 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1451/2911] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:24.293033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:24.295581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:24.318831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:24.537538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34674e9-34f0-44ca-936c-9a4cd17d789c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:24.554397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9b00a-5417-438f-862f-d5d7cad955c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:24.554748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9b00a-5417-438f-862f-d5d7cad955c0 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:24.555029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9b00a-5417-438f-862f-d5d7cad955c0 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:24.555313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9b00a-5417-438f-862f-d5d7cad955c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:24.560141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-16c9b00a-5417-438f-862f-d5d7cad955c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:24.693374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9b00a-5417-438f-862f-d5d7cad955c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:24.695137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9b00a-5417-438f-862f-d5d7cad955c0 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:24.712779 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1461/2912] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:24 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:26.442802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:26.486240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47245680-fc43-4e35-a088-a7f13d4402b1 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:26.500953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b81c4-842b-4aee-abba-5386507adb1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:26.501581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b81c4-842b-4aee-abba-5386507adb1f service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:26.501939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b81c4-842b-4aee-abba-5386507adb1f service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:26.502219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b81c4-842b-4aee-abba-5386507adb1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:26.504041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8c5b81c4-842b-4aee-abba-5386507adb1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:26.530888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b81c4-842b-4aee-abba-5386507adb1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:26.532529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b81c4-842b-4aee-abba-5386507adb1f service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:26.564324 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1452/2913] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:28.492713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:28.493229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:28.493317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:28.568100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] MFA Rules not processed for user `d8b0a160808f4adda7be57cd535f0631`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:28.569364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:28.719312 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1462/2914] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:28.729353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:28.730280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:28.730892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:28.764052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] MFA Rules not processed for user `d8b0a160808f4adda7be57cd535f0631`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:28.767545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:28.779298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1453/2915] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:30.535651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:30.537133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:30.558297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68833ce-ff8f-4f95-bc86-721920385b03 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:30.568710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7097d3-1a68-4102-ae64-c32678a579a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:30.569663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7097d3-1a68-4102-ae64-c32678a579a8 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:30.570172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7097d3-1a68-4102-ae64-c32678a579a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:30.570642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7097d3-1a68-4102-ae64-c32678a579a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:30.572620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fd7097d3-1a68-4102-ae64-c32678a579a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:30.602900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7097d3-1a68-4102-ae64-c32678a579a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:30.604816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7097d3-1a68-4102-ae64-c32678a579a8 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:30.623597 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1463/2916] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:30.968588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:30.970927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:30.997524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253ea76-92e8-4db5-acf7-9732e5345a28 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:31.016591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b19f31-bfb6-45ae-9ea4-1a8e84de020c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:31.017373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19f31-bfb6-45ae-9ea4-1a8e84de020c service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:31.017751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19f31-bfb6-45ae-9ea4-1a8e84de020c service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:31.018165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19f31-bfb6-45ae-9ea4-1a8e84de020c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:31.038199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:31.054077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b19f31-bfb6-45ae-9ea4-1a8e84de020c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:31.054077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b19f31-bfb6-45ae-9ea4-1a8e84de020c service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:31.085695 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1454/2917] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:34.439248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56362c24-3105-4810-94ea-619ae6b3cfee None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:34.439842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56362c24-3105-4810-94ea-619ae6b3cfee None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:34.439842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56362c24-3105-4810-94ea-619ae6b3cfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:34.496616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-56362c24-3105-4810-94ea-619ae6b3cfee None None] MFA Rules not processed for user `d8b0a160808f4adda7be57cd535f0631`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:34.499826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-56362c24-3105-4810-94ea-619ae6b3cfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:34.532051 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1464/2918] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:36.362411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1a62-7056-4ad7-9c35-8ecc384b8c4b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:36.362411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1a62-7056-4ad7-9c35-8ecc384b8c4b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:36.362411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1a62-7056-4ad7-9c35-8ecc384b8c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:36.441506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5b3b1a62-7056-4ad7-9c35-8ecc384b8c4b None None] MFA Rules not processed for user `5b0c267722f14e7fa87395f796551ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:36.442861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b3b1a62-7056-4ad7-9c35-8ecc384b8c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:36.586354 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1455/2919] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:37.395488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56362c24-3105-4810-94ea-619ae6b3cfee None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:37.432405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56362c24-3105-4810-94ea-619ae6b3cfee None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:37.466599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:37.467233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:37.467629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:37.468045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:37.475187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:37.529099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:37.534480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:37.547635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:37.583234 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1465/2920] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:37.713041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b1a62-7056-4ad7-9c35-8ecc384b8c4b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:37.735957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b1a62-7056-4ad7-9c35-8ecc384b8c4b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:37.754594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c4ed8-e399-4c1c-9321-a62dc837a5e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:37.755924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c4ed8-e399-4c1c-9321-a62dc837a5e3 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:37.756420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c4ed8-e399-4c1c-9321-a62dc837a5e3 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:37.757103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c4ed8-e399-4c1c-9321-a62dc837a5e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:37.788695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c4ed8-e399-4c1c-9321-a62dc837a5e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:37.790500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c4ed8-e399-4c1c-9321-a62dc837a5e3 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:37.821265 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1456/2921] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:41.091547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:41.093726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:41.129800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a8cc4-26e2-4b60-ba34-6226cb64f67a service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:41.154737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:41.155593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:41.155974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:41.156248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] PATH_INFO: `/v3/users/f340cba48e624334bf7da93b5240f3b9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:41.163578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f340cba48e624334bf7da93b5240f3b9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:41.168289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:41.283961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f340cba48e624334bf7da93b5240f3b9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:41.317260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f340cba48e624334bf7da93b5240f3b9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:41.330393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f340cba48e624334bf7da93b5240f3b9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:41.357426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f340cba48e624334bf7da93b5240f3b9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:41.359716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f340cba48e624334bf7da93b5240f3b9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:41.370168 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1466/2922] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:41 2022] DELETE /identity/v3/users/f340cba48e624334bf7da93b5240f3b9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:41.375908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070923-0d61-439c-bc2e-f73680e31b94 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:41.376507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070923-0d61-439c-bc2e-f73680e31b94 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:41.376507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070923-0d61-439c-bc2e-f73680e31b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:41.429993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d0070923-0d61-439c-bc2e-f73680e31b94 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:41.431206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d0070923-0d61-439c-bc2e-f73680e31b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:41.596782 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1457/2923] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:41.643329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:41.646597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:41.792744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dae05f-72ea-4967-90fb-02fffcfd9eb6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:41.804390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec16c6-e60f-4fcb-b3e4-4815164daf0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:41.805203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec16c6-e60f-4fcb-b3e4-4815164daf0d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:41.805508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec16c6-e60f-4fcb-b3e4-4815164daf0d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:41.805753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec16c6-e60f-4fcb-b3e4-4815164daf0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:41.835736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec16c6-e60f-4fcb-b3e4-4815164daf0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:41.838082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec16c6-e60f-4fcb-b3e4-4815164daf0d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:41.855110 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1467/2924] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:42.129924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0f2b6-a620-4e88-b637-96b60160d0eb None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:42.130420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0f2b6-a620-4e88-b637-96b60160d0eb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:42.130897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0f2b6-a620-4e88-b637-96b60160d0eb None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:42.133046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1458/2925] 10.222.0.33 () {54 vars in 921 bytes} [Sat Sep 10 20:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:53:42.147041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:42.147719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:42.147719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:42.179548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:42.181894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:42.297034 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1468/2926] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 20:53:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:42.319586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0f2b6-a620-4e88-b637-96b60160d0eb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:42.339129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0f2b6-a620-4e88-b637-96b60160d0eb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:42.349378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:42.350406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:42.350845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:42.351296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:42.367998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:42.370626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:42.385193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:42.387420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:42.388828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1459/2927] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:42.400750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:42.414943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b02408-9d40-4aa3-b756-40debd068baa None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:42.428602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:42.429659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:42.430139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:42.430639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] PATH_INFO: `/v3/projects/70fb988b59aa4806b77ca61de1d6d57e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:42.432482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fb988b59aa4806b77ca61de1d6d57e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:42.435692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:42.550128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fb988b59aa4806b77ca61de1d6d57e', 'request_id': 'req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:42.560111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fb988b59aa4806b77ca61de1d6d57e', 'request_id': 'req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:42.583290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fb988b59aa4806b77ca61de1d6d57e', 'request_id': 'req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:42.630739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] The token cache is being invalidate because project 70fb988b59aa4806b77ca61de1d6d57e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:42.631040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:42.635867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-967b17a8-4064-4e36-a0d2-3a1e9f4f7028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:42.645805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1469/2928] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:42 2022] DELETE /identity/v3/projects/70fb988b59aa4806b77ca61de1d6d57e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:53:42.650622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:42.671432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e27b7-da4f-43c1-8e49-dacd33f8313e service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:42.872437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:42.874178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:42.875463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:42.875995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] PATH_INFO: `/v3/users/af5454f868414e729a3654cd9be1e30a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:42.878460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5454f868414e729a3654cd9be1e30a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:42.881971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:42.888272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:43.033287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5454f868414e729a3654cd9be1e30a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:43.051355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5454f868414e729a3654cd9be1e30a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:43.052616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5454f868414e729a3654cd9be1e30a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:43.082837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5454f868414e729a3654cd9be1e30a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:43.083634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5454f868414e729a3654cd9be1e30a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:43.147157 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1460/2929] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:53:42 2022] DELETE /identity/v3/users/af5454f868414e729a3654cd9be1e30a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:43.952410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa147a-1074-42b9-a7b1-c1e56e9bbaab None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:43.954332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa147a-1074-42b9-a7b1-c1e56e9bbaab None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:43.954939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa147a-1074-42b9-a7b1-c1e56e9bbaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:44.011033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-21fa147a-1074-42b9-a7b1-c1e56e9bbaab None None] MFA Rules not processed for user `5b0c267722f14e7fa87395f796551ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:44.012291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-21fa147a-1074-42b9-a7b1-c1e56e9bbaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:44.186516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1470/2930] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:44.549134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:44.551884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:44.727991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba9636-d4aa-41aa-be47-1451036dc7bf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:44.736776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:44.738101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:44.739799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:44.740474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] PATH_INFO: `/v3/projects/219ffc0c09a9455f8a0e022ffcbee9a4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:44.742446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=219ffc0c09a9455f8a0e022ffcbee9a4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:44.745262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:44.943230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '219ffc0c09a9455f8a0e022ffcbee9a4', 'request_id': 'req-565cae55-1bca-4c39-90f3-fa34530916e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:44.977236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '219ffc0c09a9455f8a0e022ffcbee9a4', 'request_id': 'req-565cae55-1bca-4c39-90f3-fa34530916e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:45.015837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '219ffc0c09a9455f8a0e022ffcbee9a4', 'request_id': 'req-565cae55-1bca-4c39-90f3-fa34530916e5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:45.041356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] The token cache is being invalidate because project 219ffc0c09a9455f8a0e022ffcbee9a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:53:45.043492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:45.044625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-565cae55-1bca-4c39-90f3-fa34530916e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:53:45.049989 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1461/2931] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:53:44 2022] DELETE /identity/v3/projects/219ffc0c09a9455f8a0e022ffcbee9a4 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:45.055850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf509fc-908f-425d-9b8c-ce6fd88b65c0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:45.056899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf509fc-908f-425d-9b8c-ce6fd88b65c0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:45.057359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf509fc-908f-425d-9b8c-ce6fd88b65c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:45.109115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-acf509fc-908f-425d-9b8c-ce6fd88b65c0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:45.110521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-acf509fc-908f-425d-9b8c-ce6fd88b65c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:45.487884 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1471/2932] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:45.496042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5dc42-1e70-4718-a55b-c3bfea74e5a5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:45.496559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5dc42-1e70-4718-a55b-c3bfea74e5a5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:45.497005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5dc42-1e70-4718-a55b-c3bfea74e5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:45.537102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f9a5dc42-1e70-4718-a55b-c3bfea74e5a5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:45.541027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f9a5dc42-1e70-4718-a55b-c3bfea74e5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:45.591129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1462/2933] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:45.595797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf509fc-908f-425d-9b8c-ce6fd88b65c0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:45.624363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf509fc-908f-425d-9b8c-ce6fd88b65c0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:45.669673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:45.680014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:45.681507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:45.683383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:45.684138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:53:45.684451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:45.699282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:45.776828 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1472/2934] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:53:45.781853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5dc42-1e70-4718-a55b-c3bfea74e5a5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:45.786912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:45.797801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5dc42-1e70-4718-a55b-c3bfea74e5a5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:45.814776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:45.815660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:45.816019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:45.816370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:45.817317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:45.821348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:45.914080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:45.914373 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1463/2935] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:45 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:45.936912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28466-3ab6-48e0-b3eb-0de6cda25122 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:46.002913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:46.118882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:46.120375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.120925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.121726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:46.122679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:46.126118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:46.208247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1473/2936] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:45 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:46.211971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:46.230054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00826ed-5643-4546-8ff6-157b3ccc5d2d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:46.246158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:46.246784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.247064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.247408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:46.248799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:46.258552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:46.277928 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1464/2937] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:46.280350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:46.324145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8ec3-c43a-4f20-85b3-9ac5cf1e721b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:46.341361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d245fda-3de9-4a3f-b36e-bf8f071059b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:46.343458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d245fda-3de9-4a3f-b36e-bf8f071059b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.344222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d245fda-3de9-4a3f-b36e-bf8f071059b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.344740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d245fda-3de9-4a3f-b36e-bf8f071059b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:46.345718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d245fda-3de9-4a3f-b36e-bf8f071059b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:46.349526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d245fda-3de9-4a3f-b36e-bf8f071059b2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:46.368424 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1474/2938] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:46.370583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:46.391666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6806b-fb8f-49d1-b376-c3f246a858dc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:46.397790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:46.397790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.397790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.398208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] PATH_INFO: `/v3/projects/f2836de68f8546259ebb0649f7567c4f/users/eff0636f25bb459cb92e0c733bb51df8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:46.432596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2836de68f8546259ebb0649f7567c4f, user_id=eff0636f25bb459cb92e0c733bb51df8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:46.440237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:46.479998 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1465/2939] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:46 2022] PUT /identity/v3/projects/f2836de68f8546259ebb0649f7567c4f/users/eff0636f25bb459cb92e0c733bb51df8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:46.486221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.486536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.486981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:46.529354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:46.531524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:46.752708 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1475/2940] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:46.792111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:46.797259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:46.948140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f973c-175e-403a-848d-563b6f5e5380 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:46.974389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:46.976486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:46.978138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:46.978404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:47.018589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:47.020920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:47.039947 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1466/2941] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:48.209044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:48.212625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:48.388285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b3297-88ab-4364-8586-7fdb54967a92 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:48.402267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:48.403224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:48.403538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:48.403820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:48.406642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:48.438208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:48.445029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:48.464331 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1476/2942] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:52.898644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:52.903285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:52.941882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6d1ac-12d7-4da0-acc7-6ec04fdd3e14 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:52.947915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:52.958370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:52.958980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:52.959504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:52.972441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:53.030839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:53.036533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:53.093646 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1467/2943] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:56.751128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:56.761771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f6e02-6c41-45dc-9769-43f0a4ba97d0 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:56.768835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476640ba-61b8-433b-b294-1e99e8723cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:56.769485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476640ba-61b8-433b-b294-1e99e8723cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:56.769770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476640ba-61b8-433b-b294-1e99e8723cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:56.773379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476640ba-61b8-433b-b294-1e99e8723cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:56.778326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-476640ba-61b8-433b-b294-1e99e8723cd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:56.806063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476640ba-61b8-433b-b294-1e99e8723cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:56.808814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476640ba-61b8-433b-b294-1e99e8723cd5 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:56.842911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1477/2944] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:57.524534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:57.536366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcf653-baec-45e9-9cf5-4cfa81edb4f8 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:57.549252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:57.552511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:57.552847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:57.553120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:57.557405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:57.694176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:57.695960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:57.716492 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1468/2945] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:53:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:53:58.775695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198df0c-6713-4f91-9393-59f45040db50 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:58.779785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198df0c-6713-4f91-9393-59f45040db50 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:58.780240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198df0c-6713-4f91-9393-59f45040db50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:58.838843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:58.915361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8198df0c-6713-4f91-9393-59f45040db50 None None] MFA Rules not processed for user `eff0636f25bb459cb92e0c733bb51df8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:53:58.919269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8198df0c-6713-4f91-9393-59f45040db50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:53:59.300687 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1478/2946] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:53:59.304249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:59.335916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da4771-c322-4f78-ac98-18419eafa7ed service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:59.368423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:59.393957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:59.394525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:59.397183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:59.398204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:59.419242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:59.529132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1469/2947] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:53:59 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:53:59.534105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198df0c-6713-4f91-9393-59f45040db50 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:59.562644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198df0c-6713-4f91-9393-59f45040db50 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:59.722254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:59.723198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:59.723703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:59.724164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:59.725017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:59.733305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:59.780191 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1479/2948] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:53:59 2022] POST /identity/v3/users => generated 299 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:53:59.784471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:59.803261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023a587-5adb-4ac5-9704-aec92dcd1dc8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:59.812076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:59.827073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:59.828342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:59.829067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:59.831567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:59.833977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:59.860888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:53:59.871976 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1470/2949] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:53:59.877357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:53:59.896424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0cdd3-759d-43e7-83b2-8badefd15ac5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:53:59.912602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:53:59.914211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:53:59.914503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:53:59.914919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] PATH_INFO: `/v3/projects/8d8536a456e94f5b842c0f9be5caf0ff/users/5fa4298166524e10a0cf869cdec9df34/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:53:59.956538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d8536a456e94f5b842c0f9be5caf0ff, user_id=5fa4298166524e10a0cf869cdec9df34, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:53:59.959693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:53:59.995264 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1480/2950] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:53:59 2022] PUT /identity/v3/projects/8d8536a456e94f5b842c0f9be5caf0ff/users/5fa4298166524e10a0cf869cdec9df34/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:53:59.999090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:00.024595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2159d-d33d-4628-8cb4-59e922bfaba2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:00.227328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:00.228165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:00.228630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:00.229075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:00.229959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:00.233231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:00.250784 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1471/2951] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:00.250925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:00.267314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6db293-0913-464c-a4c9-80297041fc1b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:00.280487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:00.286838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:00.287736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:00.288831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] PATH_INFO: `/v3/projects/8d8536a456e94f5b842c0f9be5caf0ff/users/5fa4298166524e10a0cf869cdec9df34/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:00.296516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d8536a456e94f5b842c0f9be5caf0ff, user_id=5fa4298166524e10a0cf869cdec9df34, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:00.307737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:00.338030 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1481/2952] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:54:00 2022] PUT /identity/v3/projects/8d8536a456e94f5b842c0f9be5caf0ff/users/5fa4298166524e10a0cf869cdec9df34/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:54:02.343132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:02.349464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:02.373633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae0f62-23ee-4b9d-b371-058f82e90fa0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:02.515658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:02.516704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:02.517867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:02.518311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] PATH_INFO: `/v3/users/8c59c2cab6eb46edb2b2482f70813740` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:02.520058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c59c2cab6eb46edb2b2482f70813740)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:02.522290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:02.641307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c59c2cab6eb46edb2b2482f70813740'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:02.657006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c59c2cab6eb46edb2b2482f70813740'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:02.658030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c59c2cab6eb46edb2b2482f70813740'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:02.673750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c59c2cab6eb46edb2b2482f70813740'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:02.674651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-641a7505-1c0d-49f5-8a6c-24da5a100140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c59c2cab6eb46edb2b2482f70813740'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:02.706027 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1472/2953] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:54:02 2022] DELETE /identity/v3/users/8c59c2cab6eb46edb2b2482f70813740 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:03.957293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:03.960604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:04.004838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a7dd7-4a74-41e3-b273-e4427280d20d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:04.070180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:04.083180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:04.084257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:04.084693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:04.085096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] PATH_INFO: `/v3/projects/ad160a4b6a31471a82c069d1c6844203` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:04.086622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad160a4b6a31471a82c069d1c6844203)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:04.089321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:04.159636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad160a4b6a31471a82c069d1c6844203', 'request_id': 'req-ff8684f3-869c-407b-9950-ddc600a428db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.167245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad160a4b6a31471a82c069d1c6844203', 'request_id': 'req-ff8684f3-869c-407b-9950-ddc600a428db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.176959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad160a4b6a31471a82c069d1c6844203', 'request_id': 'req-ff8684f3-869c-407b-9950-ddc600a428db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.195458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] The token cache is being invalidate because project ad160a4b6a31471a82c069d1c6844203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:54:04.196182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.197250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff8684f3-869c-407b-9950-ddc600a428db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.202087 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1482/2954] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:54:03 2022] DELETE /identity/v3/projects/ad160a4b6a31471a82c069d1c6844203 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:04.204929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:04.205389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:04.205491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:04.227027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:04.228260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:04.240613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:04.339865 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1473/2955] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:04.348954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb10dd-2d73-40f5-bfb0-94ef749945e4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:04.349247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb10dd-2d73-40f5-bfb0-94ef749945e4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:04.349519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb10dd-2d73-40f5-bfb0-94ef749945e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:04.383595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d2fb10dd-2d73-40f5-bfb0-94ef749945e4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:04.384483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2fb10dd-2d73-40f5-bfb0-94ef749945e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:04.399354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:04.401885 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1483/2956] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:04.403787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb10dd-2d73-40f5-bfb0-94ef749945e4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:04.405694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:04.442099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d859-f831-4587-b727-e2cb63914eb1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:04.454257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:04.454257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:04.454257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:04.454257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] PATH_INFO: `/v3/users/5053eb274d2c4e8180aa1cbcb680a94f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:04.455086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5053eb274d2c4e8180aa1cbcb680a94f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:04.458478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:04.478192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb10dd-2d73-40f5-bfb0-94ef749945e4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:04.496194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:04.499467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:04.500615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:04.502216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:04.503140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:54:04.504354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:04.511113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:04.545282 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1484/2957] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:54:04.549559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:04.579044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88308d92-3620-4367-995d-04a7150d0496 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:04.596064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:04.596374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5053eb274d2c4e8180aa1cbcb680a94f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.598600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:04.599322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:04.601692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:04.602581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:04.609214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:04.630611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5053eb274d2c4e8180aa1cbcb680a94f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.631476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5053eb274d2c4e8180aa1cbcb680a94f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.648588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5053eb274d2c4e8180aa1cbcb680a94f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.649051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5053eb274d2c4e8180aa1cbcb680a94f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:04.687456 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1485/2958] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:54:04 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:54:04.691830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:04.703259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:04.710677 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1474/2959] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:54:04 2022] DELETE /identity/v3/users/5053eb274d2c4e8180aa1cbcb680a94f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:04.822324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a837e1-9f2f-405a-877f-397894d7b6a3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:04.889407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:04.914397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:04.915192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:04.915755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:04.916286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:04.917031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:04.919644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:04.992363 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1486/2960] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:54:04 2022] POST /identity/v3/users => generated 291 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:54:04.996912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:05.014010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71af30-b44b-4933-bd9e-cd3013f0d942 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:05.027914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:05.028622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:05.029667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:05.029997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:05.031336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:05.035413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:05.074790 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1475/2961] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:05.079763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:05.102773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388d021-ff87-4f07-bdb5-a81eaf33d269 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:05.117497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6d8ef-eb7e-44fa-ad65-90aad6ea429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:05.118384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d8ef-eb7e-44fa-ad65-90aad6ea429f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:05.125162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d8ef-eb7e-44fa-ad65-90aad6ea429f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:05.125999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6d8ef-eb7e-44fa-ad65-90aad6ea429f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:05.126951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d8ef-eb7e-44fa-ad65-90aad6ea429f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:05.129545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6d8ef-eb7e-44fa-ad65-90aad6ea429f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:05.171690 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1487/2962] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:05.176135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:05.193256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e602-1f04-44f5-a832-847614a60fda admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:05.204116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:05.205789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:05.206380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:05.207814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] PATH_INFO: `/v3/projects/566dd302b2664b6e993ba2d639d86a5d/users/8b615ba669104c3b92734b838a026594/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:05.242051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=566dd302b2664b6e993ba2d639d86a5d, user_id=8b615ba669104c3b92734b838a026594, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:05.245136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:05.267499 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1476/2963] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:54:05 2022] PUT /identity/v3/projects/566dd302b2664b6e993ba2d639d86a5d/users/8b615ba669104c3b92734b838a026594/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:05.352657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2237d-df80-4c0a-af80-ad29a9ecc79e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:05.352968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2237d-df80-4c0a-af80-ad29a9ecc79e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:05.353236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2237d-df80-4c0a-af80-ad29a9ecc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:05.388119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dda2237d-df80-4c0a-af80-ad29a9ecc79e None None] MFA Rules not processed for user `8b615ba669104c3b92734b838a026594`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:05.389479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dda2237d-df80-4c0a-af80-ad29a9ecc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:05.580491 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1488/2964] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:05.591071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:05.594434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:05.701817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb0437-9289-4023-88e5-4a1683e6fbb3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:05.707079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:05.708534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:05.709076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:05.721974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:05.731963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:05.737818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:05.741037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:05.785347 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1477/2965] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:06.055012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:06.055593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:06.055593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:06.094166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:06.098519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:06.206694 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1489/2966] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:06.276200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:06.279634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:06.386323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cc356-79cf-472f-8dca-05825b0d4215 service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:06.394459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c92fd-aa91-44b6-97e4-9cfeff32760b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:06.395089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c92fd-aa91-44b6-97e4-9cfeff32760b service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:06.395429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c92fd-aa91-44b6-97e4-9cfeff32760b service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:06.395853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c92fd-aa91-44b6-97e4-9cfeff32760b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:06.413767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c92fd-aa91-44b6-97e4-9cfeff32760b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:06.415772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c92fd-aa91-44b6-97e4-9cfeff32760b service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:06.430611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1478/2967] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:54:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:54:07.193990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:07.196352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:07.316454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5578e8-3754-4809-8ada-0a5ae27abb82 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:07.322415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:07.323475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:07.324038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:07.324640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] PATH_INFO: `/v3/projects/a45a58e14da346efbb24a4bfed8b0b69` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:07.326113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a45a58e14da346efbb24a4bfed8b0b69)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:07.328452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:07.330545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:07.412154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a45a58e14da346efbb24a4bfed8b0b69', 'request_id': 'req-bdb15980-fda9-433a-8b06-1971b3888b33'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:07.423959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a45a58e14da346efbb24a4bfed8b0b69', 'request_id': 'req-bdb15980-fda9-433a-8b06-1971b3888b33'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:07.432954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a45a58e14da346efbb24a4bfed8b0b69', 'request_id': 'req-bdb15980-fda9-433a-8b06-1971b3888b33'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:07.457363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] The token cache is being invalidate because project a45a58e14da346efbb24a4bfed8b0b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:54:07.458148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:07.460327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdb15980-fda9-433a-8b06-1971b3888b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:07.462998 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1490/2968] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:54:07 2022] DELETE /identity/v3/projects/a45a58e14da346efbb24a4bfed8b0b69 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:07.469292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebeb33-957b-4190-82ad-7365a9da5b5b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:07.469970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebeb33-957b-4190-82ad-7365a9da5b5b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:07.469970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebeb33-957b-4190-82ad-7365a9da5b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:07.501970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-64ebeb33-957b-4190-82ad-7365a9da5b5b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:07.504229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64ebeb33-957b-4190-82ad-7365a9da5b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:07.605701 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1479/2969] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:07.614594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49338c-69f7-41b0-8f3c-810b81628b6f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:07.614941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49338c-69f7-41b0-8f3c-810b81628b6f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:07.615228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49338c-69f7-41b0-8f3c-810b81628b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:07.711292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cf49338c-69f7-41b0-8f3c-810b81628b6f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:07.715671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cf49338c-69f7-41b0-8f3c-810b81628b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:07.729735 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1491/2970] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:07.731026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebeb33-957b-4190-82ad-7365a9da5b5b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:07.756457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebeb33-957b-4190-82ad-7365a9da5b5b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:07.764390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:07.765224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:07.765627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:07.766056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:07.766952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:54:07.767288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:07.768990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:07.803240 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1480/2971] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:54:07.807827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49338c-69f7-41b0-8f3c-810b81628b6f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:07.834973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49338c-69f7-41b0-8f3c-810b81628b6f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:07.849651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:07.850312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:07.850595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:07.852358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:07.853647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:07.855538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:07.934080 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1492/2972] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:54:07 2022] POST /identity/v3/projects => generated 397 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:54:07.935526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:07.950915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f789fc-1511-46da-a58b-37458efcf30f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:08.115599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:08.116602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.116989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.117385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.118167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:08.120185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:08.155407 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1481/2973] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:54:07 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:54:08.162112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:08.194649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86131ec3-c761-4ec8-ab28-511f63b76cb6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:08.225930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:08.227419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.231784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.232691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.236648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:08.238435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:08.260123 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1493/2974] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:08.265053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:08.271885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:08.280240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84288-ab00-47cf-b7ee-8bdb4160b55a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:08.298431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f2fb-a353-42c5-96f7-1d32bf165028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:08.299072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f2fb-a353-42c5-96f7-1d32bf165028 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.299334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f2fb-a353-42c5-96f7-1d32bf165028 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.299785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f2fb-a353-42c5-96f7-1d32bf165028 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.300622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f2fb-a353-42c5-96f7-1d32bf165028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:08.302492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f2fb-a353-42c5-96f7-1d32bf165028 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:08.323868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1482/2975] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:08.330432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:08.345316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12fb59-2f78-4446-8c30-e1dde47d442d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:08.363045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:08.363801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.364089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.364365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] PATH_INFO: `/v3/projects/471f82eb46074bdabaedf6f837f6b94c/users/7ef231141009462d9819a123ea16d8c4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.395615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=471f82eb46074bdabaedf6f837f6b94c, user_id=7ef231141009462d9819a123ea16d8c4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:08.398332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:08.431739 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1494/2976] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:54:08 2022] PUT /identity/v3/projects/471f82eb46074bdabaedf6f837f6b94c/users/7ef231141009462d9819a123ea16d8c4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:08.432899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d98fe4-79ce-4d8e-b60e-8ac02cbe2204 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.433195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d98fe4-79ce-4d8e-b60e-8ac02cbe2204 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.434078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d98fe4-79ce-4d8e-b60e-8ac02cbe2204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.481300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e0d98fe4-79ce-4d8e-b60e-8ac02cbe2204 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:08.482611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e0d98fe4-79ce-4d8e-b60e-8ac02cbe2204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:08.650917 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1483/2977] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:54:08.677630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:08.679416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:08.792867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e03fa-6aaa-4744-b43c-a71d12ee3d44 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:08.807561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:08.809311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:08.809633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:08.811247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:08.835397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:08.838113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:08.866334 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1495/2978] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:14.922088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52704b9e-3117-4200-a50e-0994d5bfa4fb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:14.922647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52704b9e-3117-4200-a50e-0994d5bfa4fb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:14.922647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52704b9e-3117-4200-a50e-0994d5bfa4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:15.048019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-52704b9e-3117-4200-a50e-0994d5bfa4fb None None] MFA Rules not processed for user `5fa4298166524e10a0cf869cdec9df34`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:15.050502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-52704b9e-3117-4200-a50e-0994d5bfa4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:15.194053 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1484/2979] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:15.960113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:15.962316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:16.045521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:16.105889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6c999-e10f-4775-b5b1-62f8e4f56f6e service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:16.114351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:16.115122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:16.116824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:16.117201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:16.119150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:16.203690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:16.205521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:16.217435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1496/2980] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:19.228347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:19.228871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:19.228871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:19.261766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:19.264007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:19.392716 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1485/2981] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:54:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:19.408352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:19.409879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:19.549007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d9d78-6c8f-4a90-858c-aae4040025ce service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:19.556789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2313-f664-4fc5-87a4-77f59c00091c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:19.557469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2313-f664-4fc5-87a4-77f59c00091c service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:19.557766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2313-f664-4fc5-87a4-77f59c00091c service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:19.558152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2313-f664-4fc5-87a4-77f59c00091c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:19.569322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2313-f664-4fc5-87a4-77f59c00091c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:19.570960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2313-f664-4fc5-87a4-77f59c00091c service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:19.584406 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1497/2982] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:54:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:54:19.890714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:19.893412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:19.921324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52fa2-8170-4fb9-8447-d35a76b982f9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:19.930806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:19.932414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:19.933196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:19.937148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] PATH_INFO: `/v3/users/8b615ba669104c3b92734b838a026594` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:19.939103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b615ba669104c3b92734b838a026594)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:19.941351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:19.979399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:20.059372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b615ba669104c3b92734b838a026594'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:20.092989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b615ba669104c3b92734b838a026594'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:20.095271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b615ba669104c3b92734b838a026594'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:20.116567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b615ba669104c3b92734b838a026594'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:20.117480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b615ba669104c3b92734b838a026594'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:20.180656 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1486/2983] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:54:19 2022] DELETE /identity/v3/users/8b615ba669104c3b92734b838a026594 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:20.186191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5bbd1-38ec-4582-9a3c-f80b32b1d85d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:20.186694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5bbd1-38ec-4582-9a3c-f80b32b1d85d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:20.188787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5bbd1-38ec-4582-9a3c-f80b32b1d85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:20.225124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-33b5bbd1-38ec-4582-9a3c-f80b32b1d85d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:20.232570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-33b5bbd1-38ec-4582-9a3c-f80b32b1d85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:20.331912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:20.349546 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1498/2984] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:20.415092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:20.417029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:20.581052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53eedda-e306-49f3-839a-93af69cade29 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:20.593359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:20.594822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:20.595738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:20.596237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:20.617788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:20.622902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:20.650798 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1487/2985] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:21.648659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee4dff-438e-4fd7-a6c6-2becb1c0bed9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:21.649140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee4dff-438e-4fd7-a6c6-2becb1c0bed9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:21.649274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee4dff-438e-4fd7-a6c6-2becb1c0bed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:21.687445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7cee4dff-438e-4fd7-a6c6-2becb1c0bed9 None None] MFA Rules not processed for user `eff0636f25bb459cb92e0c733bb51df8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:21.688893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7cee4dff-438e-4fd7-a6c6-2becb1c0bed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:21.783265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:21.785608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:21.827390 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1499/2986] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:21.847150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f684e0d-0edd-49ae-9b0e-06f6bb83e2b1 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:21.871749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:21.873971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:21.876120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:21.877347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] PATH_INFO: `/v3/projects/566dd302b2664b6e993ba2d639d86a5d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:21.880850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=566dd302b2664b6e993ba2d639d86a5d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:21.894637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:22.030356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566dd302b2664b6e993ba2d639d86a5d', 'request_id': 'req-d139774b-c474-4152-b80c-7039e52bca20'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:22.039079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566dd302b2664b6e993ba2d639d86a5d', 'request_id': 'req-d139774b-c474-4152-b80c-7039e52bca20'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:22.051114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566dd302b2664b6e993ba2d639d86a5d', 'request_id': 'req-d139774b-c474-4152-b80c-7039e52bca20'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:22.064092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] The token cache is being invalidate because project 566dd302b2664b6e993ba2d639d86a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:54:22.064733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:22.065811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d139774b-c474-4152-b80c-7039e52bca20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:22.067942 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1488/2987] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:54:21 2022] DELETE /identity/v3/projects/566dd302b2664b6e993ba2d639d86a5d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:22.072536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387696-988d-497f-9176-d96af811b6c0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.072827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387696-988d-497f-9176-d96af811b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.073129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387696-988d-497f-9176-d96af811b6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.108363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058504f-796a-4e72-be23-e2ea110ed1ed None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.108941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058504f-796a-4e72-be23-e2ea110ed1ed None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.109396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058504f-796a-4e72-be23-e2ea110ed1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.137419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-50387696-988d-497f-9176-d96af811b6c0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:22.139336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-50387696-988d-497f-9176-d96af811b6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:22.188488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b058504f-796a-4e72-be23-e2ea110ed1ed None None] MFA Rules not processed for user `7ef231141009462d9819a123ea16d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:22.190213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b058504f-796a-4e72-be23-e2ea110ed1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:22.199748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:22.344950 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1500/2988] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:22.346020 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1489/2989] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:22.354898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99541-d357-46e3-add9-dcab4a2f87ba None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.355182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99541-d357-46e3-add9-dcab4a2f87ba None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.357696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefaa27-9167-403a-8760-a0ca8f089a04 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.357951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefaa27-9167-403a-8760-a0ca8f089a04 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.358286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefaa27-9167-403a-8760-a0ca8f089a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.366519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99541-d357-46e3-add9-dcab4a2f87ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.395615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0fefaa27-9167-403a-8760-a0ca8f089a04 None None] MFA Rules not processed for user `7ef231141009462d9819a123ea16d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:22.396897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0fefaa27-9167-403a-8760-a0ca8f089a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:22.407832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bad99541-d357-46e3-add9-dcab4a2f87ba None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:22.409038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bad99541-d357-46e3-add9-dcab4a2f87ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:22.412825 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1490/2990] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:22.422354 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1501/2991] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:22.426552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefaa27-9167-403a-8760-a0ca8f089a04 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:22.441723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefaa27-9167-403a-8760-a0ca8f089a04 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:22.455007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:22.455733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.456118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.456498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.457508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:54:22.458076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:22.462151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:22.493738 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1491/2992] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:54:22.494912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad99541-d357-46e3-add9-dcab4a2f87ba None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:22.507844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad99541-d357-46e3-add9-dcab4a2f87ba None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:22.550000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:22.550000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.550000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.550000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.550480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:22.550480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:22.603284 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1502/2993] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:54:22 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:54:22.608258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:22.625394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e909a-8074-4bed-8a57-4dd24cfdafbc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:22.744906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:22.746326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.746713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.746851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.748439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:22.759235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:22.819738 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1492/2994] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:54:22 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:54:22.820858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:22.837754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f7dda-b8cf-4ca8-ab7d-b096a1b041a8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:22.846044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:22.847813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.848102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.848379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.849037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:22.853943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:22.881523 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1503/2995] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:22.886841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:22.909748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:22.911564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:22.917399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10835-ffb7-43e4-8ec9-e57f46a5b07b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:22.931214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:22.931855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:22.932174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:22.932447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:22.933252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:22.935215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:22.979586 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1493/2996] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:54:22.986771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:23.013922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6abed-240e-46b7-b1ee-7f396e6eef45 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:23.024567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac8c05-7a1e-4bce-b3e0-f5cab3a167f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:23.028720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac8c05-7a1e-4bce-b3e0-f5cab3a167f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:23.029128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac8c05-7a1e-4bce-b3e0-f5cab3a167f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:23.029476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac8c05-7a1e-4bce-b3e0-f5cab3a167f0 admin admin] PATH_INFO: `/v3/projects/e8f7f9f3112545e9b40d88a7cc48e3fe/users/687eef270165450a97b4d260b2724b3e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:23.066788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:23.068579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac8c05-7a1e-4bce-b3e0-f5cab3a167f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f7f9f3112545e9b40d88a7cc48e3fe, user_id=687eef270165450a97b4d260b2724b3e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:23.071740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac8c05-7a1e-4bce-b3e0-f5cab3a167f0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:23.095588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a8be8-1f09-4bb3-ba67-166edecc4fad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:23.104252 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1494/2997] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:54:22 2022] PUT /identity/v3/projects/e8f7f9f3112545e9b40d88a7cc48e3fe/users/687eef270165450a97b4d260b2724b3e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:23.110548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:23.110931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:23.111277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:23.176576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:23.179125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:23.218192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:23.218830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:23.219115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:23.219392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:23.256362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:23.283654 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1495/2998] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:23.295797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:23.297298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:23.338328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:23.340118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:23.350719 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1504/2999] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:22 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:23.367281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcbb9f-ea1e-4667-b3a6-1ed6e6df48c5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:23.371274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab2622-0fd6-418a-bdbc-fc68d3ceccd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:23.371886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab2622-0fd6-418a-bdbc-fc68d3ceccd8 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:23.372965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab2622-0fd6-418a-bdbc-fc68d3ceccd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:23.373239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab2622-0fd6-418a-bdbc-fc68d3ceccd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:23.384815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab2622-0fd6-418a-bdbc-fc68d3ceccd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:23.386326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab2622-0fd6-418a-bdbc-fc68d3ceccd8 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:23.401675 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1496/3000] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:24.047138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:24.048670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:24.069190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70450349-f4cb-4b69-8d66-fe1178b6e8d1 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:24.078073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:24.079386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:24.079679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:24.079959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:24.081843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:24.190025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:24.191818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:24.226088 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1505/3001] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:24 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:25.339732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605923f-916f-48ec-b646-3cb31334c5f2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:25.341954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605923f-916f-48ec-b646-3cb31334c5f2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:25.342453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605923f-916f-48ec-b646-3cb31334c5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:25.380368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5605923f-916f-48ec-b646-3cb31334c5f2 None None] MFA Rules not processed for user `f74a2c8cd22c454fa8fdd7c2e1ef9230`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:25.381604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5605923f-916f-48ec-b646-3cb31334c5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:25.521680 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1497/3002] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:25.635771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:25.639255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:25.733853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:25.878446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa26e57-7793-4472-8d34-8d63c08a45ee service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:25.891923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8775317-1f95-4aac-955d-413239d7d5b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:25.892685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775317-1f95-4aac-955d-413239d7d5b1 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:25.893071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775317-1f95-4aac-955d-413239d7d5b1 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:25.893715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8775317-1f95-4aac-955d-413239d7d5b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:25.895830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8775317-1f95-4aac-955d-413239d7d5b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:25.918393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8775317-1f95-4aac-955d-413239d7d5b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:25.922048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8775317-1f95-4aac-955d-413239d7d5b1 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:25.938137 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1506/3003] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:26.868468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce8372-9c2a-4fce-8faa-0079056af510 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:26.869432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce8372-9c2a-4fce-8faa-0079056af510 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:26.870212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce8372-9c2a-4fce-8faa-0079056af510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:26.913806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-92ce8372-9c2a-4fce-8faa-0079056af510 None None] MFA Rules not processed for user `870c4c66a15b4546bd981cfb5932b214`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:26.915219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-92ce8372-9c2a-4fce-8faa-0079056af510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:27.050134 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1498/3004] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:35.930823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=18 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:37.032195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:38.050805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f977c-7b42-4d2c-a6f8-252db9fbecda None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:38.053479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f977c-7b42-4d2c-a6f8-252db9fbecda None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:38.054359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f977c-7b42-4d2c-a6f8-252db9fbecda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:38.099871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3f1f977c-7b42-4d2c-a6f8-252db9fbecda None None] MFA Rules not processed for user `687eef270165450a97b4d260b2724b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:38.102617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3f1f977c-7b42-4d2c-a6f8-252db9fbecda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:38.260045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1507/3005] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:38.276772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c170c-a125-4c49-9a50-ba8905f55bd4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:38.277328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c170c-a125-4c49-9a50-ba8905f55bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:38.277452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c170c-a125-4c49-9a50-ba8905f55bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:38.313608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-504c170c-a125-4c49-9a50-ba8905f55bd4 None None] MFA Rules not processed for user `687eef270165450a97b4d260b2724b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:38.318442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-504c170c-a125-4c49-9a50-ba8905f55bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:38.343615 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1499/3006] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:38.814081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f977c-7b42-4d2c-a6f8-252db9fbecda None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:38.836917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f977c-7b42-4d2c-a6f8-252db9fbecda None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:38.853369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcd78d-7418-458f-9c46-a949b9ec3832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:38.854439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcd78d-7418-458f-9c46-a949b9ec3832 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:38.854953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcd78d-7418-458f-9c46-a949b9ec3832 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:38.855479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcd78d-7418-458f-9c46-a949b9ec3832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:38.872911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcd78d-7418-458f-9c46-a949b9ec3832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:38.874921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcd78d-7418-458f-9c46-a949b9ec3832 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:38.900082 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1508/3007] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:43.430576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b0c82-4d97-4bd5-a583-14305e5251e2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:43.431055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b0c82-4d97-4bd5-a583-14305e5251e2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:43.431185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b0c82-4d97-4bd5-a583-14305e5251e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:43.467330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-600b0c82-4d97-4bd5-a583-14305e5251e2 None None] MFA Rules not processed for user `7ef231141009462d9819a123ea16d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:43.470493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-600b0c82-4d97-4bd5-a583-14305e5251e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:43.483091 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1500/3008] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:44.098508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6ce4-2997-4483-bdf4-6c900c5bb17b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:44.099451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6ce4-2997-4483-bdf4-6c900c5bb17b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:44.099926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6ce4-2997-4483-bdf4-6c900c5bb17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:44.131848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e3fd6ce4-2997-4483-bdf4-6c900c5bb17b None None] MFA Rules not processed for user `7ef231141009462d9819a123ea16d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:44.133287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e3fd6ce4-2997-4483-bdf4-6c900c5bb17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:44.146968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1509/3009] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:44.192704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4881a5-63aa-4c32-8c5d-2e08161c4a7f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:44.193933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4881a5-63aa-4c32-8c5d-2e08161c4a7f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:44.194499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4881a5-63aa-4c32-8c5d-2e08161c4a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:44.220357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cc4881a5-63aa-4c32-8c5d-2e08161c4a7f None None] MFA Rules not processed for user `7ef231141009462d9819a123ea16d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:44.222564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cc4881a5-63aa-4c32-8c5d-2e08161c4a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:44.237253 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1501/3010] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:44.273810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619dc90-9c38-4ab8-97ff-e221ff6dd905 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:44.274445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619dc90-9c38-4ab8-97ff-e221ff6dd905 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:44.275119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619dc90-9c38-4ab8-97ff-e221ff6dd905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:44.306370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8619dc90-9c38-4ab8-97ff-e221ff6dd905 None None] MFA Rules not processed for user `7ef231141009462d9819a123ea16d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:44.308266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8619dc90-9c38-4ab8-97ff-e221ff6dd905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:44.323419 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1510/3011] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:44.353628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:44.354077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:44.354457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:44.398926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] MFA Rules not processed for user `7ef231141009462d9819a123ea16d8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:54:44.401432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:44.439941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1502/3012] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:54:51.041328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619dc90-9c38-4ab8-97ff-e221ff6dd905 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:51.078173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619dc90-9c38-4ab8-97ff-e221ff6dd905 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:51.111992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:51.115302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:51.115594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:51.118931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:51.123291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:51.143159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:51.144836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:51.172079 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1511/3013] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:51.627335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:51.629561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:51.674348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f0fa0-1898-473c-9d91-d733cbafdc67 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:51.685658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:51.688052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:51.688348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:51.688791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:51.694702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:51.721953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:51.724469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:51.738762 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1503/3014] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:58.804460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:58.808216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:58.825219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8fc50-e0dd-43d9-94f7-a354989c3fe0 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:58.833042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:58.833687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:58.834019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:58.834294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:58.836208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:58.891300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:58.892929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:54:58.894208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:58.905261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:58.933620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048eea-19c4-43a1-85f5-585105da6e3d service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:58.941128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1512/3015] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:54:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:54:58.964371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:58.967199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:58.967802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:58.969200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] PATH_INFO: `/v3/users/7ef231141009462d9819a123ea16d8c4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:58.972486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef231141009462d9819a123ea16d8c4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:58.982260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:59.124506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef231141009462d9819a123ea16d8c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:59.145782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef231141009462d9819a123ea16d8c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:59.149660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef231141009462d9819a123ea16d8c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:59.163002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef231141009462d9819a123ea16d8c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:59.163952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-582e7101-306b-43df-906f-5a5553203026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef231141009462d9819a123ea16d8c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:59.206366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1504/3016] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:54:58 2022] DELETE /identity/v3/users/7ef231141009462d9819a123ea16d8c4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:54:59.773416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:54:59.791424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881878-77d9-42ed-85de-c5511b576cc6 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:54:59.870200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:54:59.870987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:54:59.871660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:54:59.871954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] PATH_INFO: `/v3/projects/471f82eb46074bdabaedf6f837f6b94c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:54:59.873120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=471f82eb46074bdabaedf6f837f6b94c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:54:59.875045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:54:59.951918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:54:59.976403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471f82eb46074bdabaedf6f837f6b94c', 'request_id': 'req-f276f927-10aa-42b9-b1a5-ad04793fef4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:59.986759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471f82eb46074bdabaedf6f837f6b94c', 'request_id': 'req-f276f927-10aa-42b9-b1a5-ad04793fef4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:54:59.999311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471f82eb46074bdabaedf6f837f6b94c', 'request_id': 'req-f276f927-10aa-42b9-b1a5-ad04793fef4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:00.019351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] The token cache is being invalidate because project 471f82eb46074bdabaedf6f837f6b94c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:55:00.019494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:00.020330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f276f927-10aa-42b9-b1a5-ad04793fef4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:00.022675 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1513/3017] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:54:59 2022] DELETE /identity/v3/projects/471f82eb46074bdabaedf6f837f6b94c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:00.030752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c2a88-273a-4237-8652-4259cb94f075 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.031289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c2a88-273a-4237-8652-4259cb94f075 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.031545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c2a88-273a-4237-8652-4259cb94f075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.053400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-de6c2a88-273a-4237-8652-4259cb94f075 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:00.054727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de6c2a88-273a-4237-8652-4259cb94f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:00.122018 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1505/3018] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:00.130361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c44c2-476b-488e-85d0-5c0ad83a8166 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.130716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c44c2-476b-488e-85d0-5c0ad83a8166 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.131012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c44c2-476b-488e-85d0-5c0ad83a8166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.153735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8f7c44c2-476b-488e-85d0-5c0ad83a8166 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:00.155032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8f7c44c2-476b-488e-85d0-5c0ad83a8166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:00.163936 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1514/3019] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:00.168204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c2a88-273a-4237-8652-4259cb94f075 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.180705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c2a88-273a-4237-8652-4259cb94f075 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.186622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.187211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.187495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.187771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.188418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:55:00.188724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.190384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.203481 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1506/3020] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:55:00.207624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c44c2-476b-488e-85d0-5c0ad83a8166 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.219013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c44c2-476b-488e-85d0-5c0ad83a8166 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.225665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.226438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.226822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.227191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.228071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.230427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.263387 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1515/3021] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:55:00 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:55:00.267559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.279544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d530-332b-41a7-9b49-7309760c9191 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.298585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:00.347764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.348380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.348664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.348939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.349599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.351593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.376986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1507/3022] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:55:00 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:55:00.381621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.402295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b57cc-26d6-4f75-ad8c-06d447f03c7c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.410007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.410820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.411107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.411483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.412319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.414556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.430897 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1516/3023] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:55:00.432626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.444648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86fe00-793c-4538-a744-bfe228f4ca46 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.451151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7cacc-4a6a-4118-a35f-54e2fc2fe54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.451923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7cacc-4a6a-4118-a35f-54e2fc2fe54e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.452317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7cacc-4a6a-4118-a35f-54e2fc2fe54e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.452688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7cacc-4a6a-4118-a35f-54e2fc2fe54e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.453456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7cacc-4a6a-4118-a35f-54e2fc2fe54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.455295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7cacc-4a6a-4118-a35f-54e2fc2fe54e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.468410 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1508/3024] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:55:00.473148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.489694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666f1dd-c744-40a1-ac62-6c51d81d1003 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.497095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.498100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.498649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.499004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] PATH_INFO: `/v3/projects/fcd2501efc564e63834afc0a474d486d/users/a7130068051c4688a7759ccbfbb134a9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.519199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd2501efc564e63834afc0a474d486d, user_id=a7130068051c4688a7759ccbfbb134a9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.521962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.540235 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1517/3025] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:55:00 2022] PUT /identity/v3/projects/fcd2501efc564e63834afc0a474d486d/users/a7130068051c4688a7759ccbfbb134a9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:00.545750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a919f3-806f-4e49-8413-05267fcbf073 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.546080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a919f3-806f-4e49-8413-05267fcbf073 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.546341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a919f3-806f-4e49-8413-05267fcbf073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.577469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c2a919f3-806f-4e49-8413-05267fcbf073 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:00.580639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c2a919f3-806f-4e49-8413-05267fcbf073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:00.653862 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1509/3026] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:00.664464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:00.666453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:00.739577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c74da-c339-47f4-98d1-ddb847a7bc17 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:00.743970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:00.744603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:00.744916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:00.745194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:00.757344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:00.758824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:00.771617 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1518/3027] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:05.297093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:05.297648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:05.297648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:05.346186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:05.348372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:05.489365 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1510/3028] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:05.516386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:05.518546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:05.616919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a27cc-da2b-4f8f-ab3e-e2cbec1b0cca service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:05.623065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab131-007c-477d-a9e4-13e74a163602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:05.623704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab131-007c-477d-a9e4-13e74a163602 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:05.623995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab131-007c-477d-a9e4-13e74a163602 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:05.624269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab131-007c-477d-a9e4-13e74a163602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:05.633166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:05.635548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab131-007c-477d-a9e4-13e74a163602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:05.638347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab131-007c-477d-a9e4-13e74a163602 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:05.667127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1519/3029] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:55:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:55:06.658913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:06.662917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:06.797328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:06.873965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7908a1-1486-4f38-99e5-3b9966d0fc73 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:06.888374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81d348-198d-4144-9c75-353871314f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:06.889024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d348-198d-4144-9c75-353871314f53 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:06.891013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d348-198d-4144-9c75-353871314f53 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:06.891323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d348-198d-4144-9c75-353871314f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:06.897462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ef81d348-198d-4144-9c75-353871314f53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:07.090940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81d348-198d-4144-9c75-353871314f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:07.094895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81d348-198d-4144-9c75-353871314f53 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:07.123304 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1511/3030] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:12.695832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:12.696342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:12.696608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:12.753985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] MFA Rules not processed for user `a7130068051c4688a7759ccbfbb134a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:12.755428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:12.920341 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1520/3031] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:12.924611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ef70-330f-4b52-97d0-fd712d912740 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:12.925111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ef70-330f-4b52-97d0-fd712d912740 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:12.925219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ef70-330f-4b52-97d0-fd712d912740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:12.982656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b179ef70-330f-4b52-97d0-fd712d912740 None None] MFA Rules not processed for user `a7130068051c4688a7759ccbfbb134a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:12.992626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b179ef70-330f-4b52-97d0-fd712d912740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:13.040673 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1512/3032] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:13.062744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:13.064341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:13.093980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f22494-9328-45c4-8e91-676c05c8b3d1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:13.110475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:13.111393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:13.113161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:13.113656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:13.139303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:13.143922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:13.176303 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1521/3033] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:19.715272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477c487-1fdc-4e7f-a7ba-68dae3f96ae4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:19.715272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477c487-1fdc-4e7f-a7ba-68dae3f96ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:19.715272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477c487-1fdc-4e7f-a7ba-68dae3f96ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:19.742158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d477c487-1fdc-4e7f-a7ba-68dae3f96ae4 None None] MFA Rules not processed for user `d8b0a160808f4adda7be57cd535f0631`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:19.743786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d477c487-1fdc-4e7f-a7ba-68dae3f96ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:19.843835 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1513/3034] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:23.158957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=40 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:24.643879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:24.658631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8a507-8f4c-468c-9bd0-269abc38a1df service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:24.664511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:24.666126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:24.667473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:24.668016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:24.671608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:24.689582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:24.691942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:24.707307 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1522/3035] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:27.586862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1065a8-7692-4d10-9dc8-3e9141b873ff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:27.587430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1065a8-7692-4d10-9dc8-3e9141b873ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:27.587430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1065a8-7692-4d10-9dc8-3e9141b873ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:27.628052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bd1065a8-7692-4d10-9dc8-3e9141b873ff None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:27.629479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bd1065a8-7692-4d10-9dc8-3e9141b873ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:27.639288 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1514/3036] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:55:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:27.674980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:27.701959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc7570-724a-480f-b16f-7f6e7304c001 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:27.712586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:27.713343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:27.714116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:27.715049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:27.728898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:27.731961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:27.762198 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1523/3037] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:55:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:55:28.539553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e14490-102f-47b7-847b-f061ebaf995a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:28.540118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e14490-102f-47b7-847b-f061ebaf995a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:28.540599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e14490-102f-47b7-847b-f061ebaf995a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:28.568189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e5e14490-102f-47b7-847b-f061ebaf995a None None] MFA Rules not processed for user `687eef270165450a97b4d260b2724b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:28.569802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e5e14490-102f-47b7-847b-f061ebaf995a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:28.688149 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1515/3038] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:37.752920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=12 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:38.676701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:41.135995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:41.139154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:41.177545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e8793-1974-4cdf-9b11-8cbefb1861a1 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:41.194011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:41.197628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:41.200570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:41.201352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] PATH_INFO: `/v3/users/dd431fc2d9834bea8df807edb91771ee` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:41.207441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd431fc2d9834bea8df807edb91771ee)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:41.209725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:41.310478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd431fc2d9834bea8df807edb91771ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:41.348515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd431fc2d9834bea8df807edb91771ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:41.363934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd431fc2d9834bea8df807edb91771ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:41.376787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd431fc2d9834bea8df807edb91771ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:41.377422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd431fc2d9834bea8df807edb91771ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:41.391021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1524/3039] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:55:41 2022] DELETE /identity/v3/users/dd431fc2d9834bea8df807edb91771ee => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:42.221098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e14490-102f-47b7-847b-f061ebaf995a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:42.252982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e14490-102f-47b7-847b-f061ebaf995a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:42.501785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:42.504903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:42.505346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:42.507716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] PATH_INFO: `/v3/projects/89075fefc8ea47c78d4ff7364e12ffde` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:42.512592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89075fefc8ea47c78d4ff7364e12ffde)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:42.519542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:42.607039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89075fefc8ea47c78d4ff7364e12ffde', 'request_id': 'req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.618464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89075fefc8ea47c78d4ff7364e12ffde', 'request_id': 'req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.631626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89075fefc8ea47c78d4ff7364e12ffde', 'request_id': 'req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.647038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] The token cache is being invalidate because project 89075fefc8ea47c78d4ff7364e12ffde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:55:42.647505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.648703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.664026 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1516/3040] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:55:42 2022] DELETE /identity/v3/projects/89075fefc8ea47c78d4ff7364e12ffde => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:42.669761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:42.672352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:42.696130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729be573-e4d0-4f0f-8eee-829764dc767f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:42.790487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:42.791996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:42.792370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:42.793109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] PATH_INFO: `/v3/users/4defc9e17db743d7b79e732a0f695ee6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:42.794642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4defc9e17db743d7b79e732a0f695ee6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:42.797037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:42.841370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:42.842939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:42.882088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4defc9e17db743d7b79e732a0f695ee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.902913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4defc9e17db743d7b79e732a0f695ee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.915596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4defc9e17db743d7b79e732a0f695ee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.920022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:42.931471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4defc9e17db743d7b79e732a0f695ee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.933074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4defc9e17db743d7b79e732a0f695ee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:42.957141 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1525/3041] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:55:42 2022] DELETE /identity/v3/users/4defc9e17db743d7b79e732a0f695ee6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:42.982967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979aa8d-cd4d-407f-b6ce-8f56c4b26db6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:43.083228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:43.095894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:43.099097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:43.099097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:43.099674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:43.109192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:43.248332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:43.251798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:43.308780 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1517/3042] 10.222.0.33 () {60 vars in 1396 bytes} [Sat Sep 10 20:55:42 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:43.603718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:43.607020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:43.660883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:43.663166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:43.738972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cec8b4-9c09-4b73-ad45-89e64616c331 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:43.748616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:43.749514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:43.749997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:43.750381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] PATH_INFO: `/v3/projects/c9ea92bd4cef4840b6e96b68f28099de` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:43.751997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ea92bd4cef4840b6e96b68f28099de)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:43.754698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:43.828612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0816bd2-ef6d-4b7e-863b-c04425dd8db3 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:43.833435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ea92bd4cef4840b6e96b68f28099de', 'request_id': 'req-05db26e7-1618-40c1-9f29-6df4c15d3687'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:43.841819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ea92bd4cef4840b6e96b68f28099de', 'request_id': 'req-05db26e7-1618-40c1-9f29-6df4c15d3687'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:43.859030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ea92bd4cef4840b6e96b68f28099de', 'request_id': 'req-05db26e7-1618-40c1-9f29-6df4c15d3687'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:43.894308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] The token cache is being invalidate because project c9ea92bd4cef4840b6e96b68f28099de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:55:43.894727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:43.895865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-05db26e7-1618-40c1-9f29-6df4c15d3687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:43.899426 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1526/3043] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:55:43 2022] DELETE /identity/v3/projects/c9ea92bd4cef4840b6e96b68f28099de => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:43.907934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d503-50d0-41e6-b9ee-9877ea824678 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:43.909071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d503-50d0-41e6-b9ee-9877ea824678 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:43.909994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d503-50d0-41e6-b9ee-9877ea824678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:43.940537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3687d503-50d0-41e6-b9ee-9877ea824678 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:43.941898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3687d503-50d0-41e6-b9ee-9877ea824678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:43.976209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:43.977404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:43.977884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:43.978326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:43.982351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:44.039814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:44.068631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1527/3044] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:44.079025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25936f3e-b932-4150-9ee8-2e4010c7cf29 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.079640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25936f3e-b932-4150-9ee8-2e4010c7cf29 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.080210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25936f3e-b932-4150-9ee8-2e4010c7cf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.106710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:44.112290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:44.114558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-25936f3e-b932-4150-9ee8-2e4010c7cf29 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:44.116459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25936f3e-b932-4150-9ee8-2e4010c7cf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:44.131996 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1518/3045] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:44.138983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1528/3046] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:44.144876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:44.162158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cef8f3-0e6a-46ec-a930-62278e97343d service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:44.169585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:44.170243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.170526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.170798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.171839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:55:44.172184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:44.173959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:44.191834 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1519/3047] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:55:44.199144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25936f3e-b932-4150-9ee8-2e4010c7cf29 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:44.214585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25936f3e-b932-4150-9ee8-2e4010c7cf29 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:44.220230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:44.220974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.221398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.221682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.222698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:44.225865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:44.247068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:44.271336 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1529/3048] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:55:44 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:55:44.276454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:44.289318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e949c-4f6d-43d1-a5d5-a2e666b335b0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:44.402777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:44.403756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.407136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.407136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.407136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:44.408980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:44.448381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1520/3049] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:55:44 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:55:44.452771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:44.469667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273bca6-4403-4855-9eb8-0e960056ee8a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:44.478147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:44.479829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.480424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.481238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.482287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:44.484482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:44.506143 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1530/3050] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:55:44.507256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:44.520935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e95d67-0592-43a4-a3f5-07c18475f305 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:44.529632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdf924-8326-4539-bec0-8f5ee464dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:44.530803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdf924-8326-4539-bec0-8f5ee464dd70 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.531198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdf924-8326-4539-bec0-8f5ee464dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.531476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdf924-8326-4539-bec0-8f5ee464dd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.532208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdf924-8326-4539-bec0-8f5ee464dd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:44.534499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdf924-8326-4539-bec0-8f5ee464dd70 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:44.555829 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1521/3051] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:55:44.558581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:44.570948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92cf06-8f07-4542-aed6-d675345dbdff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:44.583793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:44.585935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.585935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.585935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] PATH_INFO: `/v3/projects/20eb61ef39584111b731fe161b76ce04/users/4fb714744b77463393388561809eebea/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.612054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20eb61ef39584111b731fe161b76ce04, user_id=4fb714744b77463393388561809eebea, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:44.615463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:44.637918 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1531/3052] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:55:44 2022] PUT /identity/v3/projects/20eb61ef39584111b731fe161b76ce04/users/4fb714744b77463393388561809eebea/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:44.648089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48786493-9563-4233-8c6f-3c1a51f96d48 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:44.648618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48786493-9563-4233-8c6f-3c1a51f96d48 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:44.648904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48786493-9563-4233-8c6f-3c1a51f96d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:44.684678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-48786493-9563-4233-8c6f-3c1a51f96d48 None None] MFA Rules not processed for user `4fb714744b77463393388561809eebea`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:44.685946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-48786493-9563-4233-8c6f-3c1a51f96d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:44.794643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1522/3053] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:44.991461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:44.993807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:45.103682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beab067-dedd-4612-94aa-78bb94c0f768 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:45.109692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:45.110394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:45.110748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:45.110937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:45.120604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:45.122475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:45.133183 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1532/3054] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:45.336716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8831-99f7-4601-b958-5310d60408b7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:45.339051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8831-99f7-4601-b958-5310d60408b7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:45.339536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b8831-99f7-4601-b958-5310d60408b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:45.372919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-dc7b8831-99f7-4601-b958-5310d60408b7 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:45.374740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dc7b8831-99f7-4601-b958-5310d60408b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:45.404956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:45.470528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1523/3055] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:55:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:45.486116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:45.487561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:45.598492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcff93b-99d3-44a6-924f-7908a8afe29b service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:45.606443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c7da1-75fe-4f31-ae42-68bd8f5bd35d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:45.607376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c7da1-75fe-4f31-ae42-68bd8f5bd35d service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:45.607751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c7da1-75fe-4f31-ae42-68bd8f5bd35d service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:45.608124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c7da1-75fe-4f31-ae42-68bd8f5bd35d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:45.620459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c7da1-75fe-4f31-ae42-68bd8f5bd35d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:45.622852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c7da1-75fe-4f31-ae42-68bd8f5bd35d service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:45.634489 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1533/3056] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:55:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:55:45.825153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88d5d9-e157-4b0e-b75a-06db678b2fff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:45.825428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88d5d9-e157-4b0e-b75a-06db678b2fff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:45.825664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88d5d9-e157-4b0e-b75a-06db678b2fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:45.887821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0a88d5d9-e157-4b0e-b75a-06db678b2fff None None] MFA Rules not processed for user `4fb714744b77463393388561809eebea`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:45.889091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0a88d5d9-e157-4b0e-b75a-06db678b2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:45.903010 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1524/3057] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:46.484751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec714c-ed62-4163-84af-cdf23bfb381a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:46.485282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec714c-ed62-4163-84af-cdf23bfb381a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:46.485403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec714c-ed62-4163-84af-cdf23bfb381a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:46.511865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f1ec714c-ed62-4163-84af-cdf23bfb381a None None] MFA Rules not processed for user `eff0636f25bb459cb92e0c733bb51df8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:46.514955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f1ec714c-ed62-4163-84af-cdf23bfb381a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:46.599170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:46.648672 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1534/3058] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:55:47.032312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3264-c50a-4b74-8de7-9d556c128b38 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:47.032851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3264-c50a-4b74-8de7-9d556c128b38 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:47.032851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3264-c50a-4b74-8de7-9d556c128b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:47.101607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fb2e3264-c50a-4b74-8de7-9d556c128b38 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:47.103032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fb2e3264-c50a-4b74-8de7-9d556c128b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:47.114535 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1525/3059] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:47.131025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec714c-ed62-4163-84af-cdf23bfb381a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:47.145607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec714c-ed62-4163-84af-cdf23bfb381a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:47.154931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0c269-3c44-438e-a16e-098abee82c8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:47.155736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0c269-3c44-438e-a16e-098abee82c8f service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:47.156123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0c269-3c44-438e-a16e-098abee82c8f service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:47.156499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0c269-3c44-438e-a16e-098abee82c8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:47.207358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0c269-3c44-438e-a16e-098abee82c8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:47.209094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0c269-3c44-438e-a16e-098abee82c8f service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:47.234388 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1535/3060] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:55:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:55:47.342095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e3264-c50a-4b74-8de7-9d556c128b38 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:47.424209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e3264-c50a-4b74-8de7-9d556c128b38 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:47.431141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:47.433147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:47.433942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:47.434627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] PATH_INFO: `/v3/users/4fb714744b77463393388561809eebea` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:47.437147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb714744b77463393388561809eebea)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:47.440466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:47.515350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb714744b77463393388561809eebea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:47.524957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb714744b77463393388561809eebea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:47.527390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb714744b77463393388561809eebea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:47.538916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb714744b77463393388561809eebea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:47.540230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb714744b77463393388561809eebea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:47.567582 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1526/3061] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:55:47 2022] DELETE /identity/v3/users/4fb714744b77463393388561809eebea => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:55:47.573166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:47.573721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:47.573721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:47.681437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:47.691615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:47.878774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1536/3062] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:47.884445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:47.887366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:47.970271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:48.001968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c186b-1990-4b85-80ef-bce2794b0065 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:48.006542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:48.007307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:48.007637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:48.007912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:48.024745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:48.028251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:48.061012 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1527/3063] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:48.928464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:48.930663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:49.115199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61c2de-e690-4399-bf50-722ef033bfd9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:49.138901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:49.140485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:49.141021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:49.141520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:49.145383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:49.163539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:49.167170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:49.195922 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1537/3064] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:49.366467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:49.390860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7588f-ca26-4552-ab5c-dd5ea2e7f837 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:49.406174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96a3a1-99dd-48be-8240-ea5d7a3edf54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:49.407111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a3a1-99dd-48be-8240-ea5d7a3edf54 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:49.407653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a3a1-99dd-48be-8240-ea5d7a3edf54 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:49.408157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a3a1-99dd-48be-8240-ea5d7a3edf54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:49.413225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-df96a3a1-99dd-48be-8240-ea5d7a3edf54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:49.443729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96a3a1-99dd-48be-8240-ea5d7a3edf54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:49.464483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96a3a1-99dd-48be-8240-ea5d7a3edf54 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:49.489800 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1528/3065] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:55:49.886905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:49.888388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:49.910358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb75b4-dc57-4715-9eb6-bd47a6c76e6e service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:49.915963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:49.918222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:49.918723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:49.919166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] PATH_INFO: `/v3/projects/20eb61ef39584111b731fe161b76ce04` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:49.920589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20eb61ef39584111b731fe161b76ce04)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:49.937084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:50.012006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:50.033735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20eb61ef39584111b731fe161b76ce04', 'request_id': 'req-3d25ad64-0805-4517-9cc0-10670ce46c52'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:50.042864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20eb61ef39584111b731fe161b76ce04', 'request_id': 'req-3d25ad64-0805-4517-9cc0-10670ce46c52'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:50.077529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20eb61ef39584111b731fe161b76ce04', 'request_id': 'req-3d25ad64-0805-4517-9cc0-10670ce46c52'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:50.108881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] The token cache is being invalidate because project 20eb61ef39584111b731fe161b76ce04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:55:50.109562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:50.111136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d25ad64-0805-4517-9cc0-10670ce46c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:55:50.117260 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1538/3066] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:55:49 2022] DELETE /identity/v3/projects/20eb61ef39584111b731fe161b76ce04 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:50.118525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32870eed-eb14-44e2-8ea3-c648d72c1808 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.118811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32870eed-eb14-44e2-8ea3-c648d72c1808 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.119738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32870eed-eb14-44e2-8ea3-c648d72c1808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.157641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-32870eed-eb14-44e2-8ea3-c648d72c1808 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:50.158917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32870eed-eb14-44e2-8ea3-c648d72c1808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:50.328534 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1529/3067] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:50.335732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3fbd3-d20b-4120-bbb6-e7518f3635f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.336331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3fbd3-d20b-4120-bbb6-e7518f3635f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.337164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3fbd3-d20b-4120-bbb6-e7518f3635f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.373424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-64a3fbd3-d20b-4120-bbb6-e7518f3635f0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:50.375201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-64a3fbd3-d20b-4120-bbb6-e7518f3635f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:50.394753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32870eed-eb14-44e2-8ea3-c648d72c1808 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:50.395961 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1539/3068] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:50.406480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32870eed-eb14-44e2-8ea3-c648d72c1808 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:50.412649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:50.414331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.414859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.415318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.416912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:55:50.417431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:50.420336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:50.459267 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1530/3069] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:55:50.464057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3fbd3-d20b-4120-bbb6-e7518f3635f0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:50.484230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3fbd3-d20b-4120-bbb6-e7518f3635f0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:50.500195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:50.501203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.501540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.501996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.503976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:50.511739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:50.570513 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1540/3070] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:55:50 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:55:50.573499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:50.592296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81b30a-4f66-415c-b848-677e21c1e5ac admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:50.732040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:50.733051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.733575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.734058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.734897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:50.737051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:50.769683 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1531/3071] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:55:50 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:55:50.771254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:50.787261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec5c38-7d4d-4176-9e72-9a704d637eff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:50.811847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:50.812954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.812954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.813260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.813947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:50.815739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:50.837112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:50.838413 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1541/3072] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:55:50.851786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b434-99b3-48c2-a0a8-5cda9a6d23af admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:50.865735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc3405-db99-4648-bb7e-4e312527ce29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:50.867044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc3405-db99-4648-bb7e-4e312527ce29 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.867523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc3405-db99-4648-bb7e-4e312527ce29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.868028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc3405-db99-4648-bb7e-4e312527ce29 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.868912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc3405-db99-4648-bb7e-4e312527ce29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:50.871130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc3405-db99-4648-bb7e-4e312527ce29 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:50.888392 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1532/3073] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:55:50.892681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:50.909205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abd43-490b-496d-8e1d-c3e9477334db admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:50.918300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:50.919677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.920193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.920927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] PATH_INFO: `/v3/projects/2d920a0c09bf4fd4a7b60fc7106a598f/users/fb3043f55aad4e91a1c3ceb734d77af3/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:50.955079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d920a0c09bf4fd4a7b60fc7106a598f, user_id=fb3043f55aad4e91a1c3ceb734d77af3, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:50.958594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:50.991745 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1542/3074] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:55:50 2022] PUT /identity/v3/projects/2d920a0c09bf4fd4a7b60fc7106a598f/users/fb3043f55aad4e91a1c3ceb734d77af3/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:55:50.997928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:50.998222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:50.998503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:51.036111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:55:51.037335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:51.081473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:55:51.132556 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1533/3075] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:55:51.198147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:55:51.200055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:55:51.329477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f532-6d8f-45ef-9a49-114c2f0150c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:55:51.338726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:55:51.339137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:55:51.339413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:55:51.339886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:55:51.350886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:55:51.352646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:55:51.365758 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1543/3076] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:56:01.119751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:01.360702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=66,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:02.886301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:02.888015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:02.906144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc32127-6e82-4cef-a2f2-373aa52dee0b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:02.913607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:02.914308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:02.914888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:02.914888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] PATH_INFO: `/v3/users/a7130068051c4688a7759ccbfbb134a9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:02.916410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7130068051c4688a7759ccbfbb134a9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:02.919032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:03.000539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7130068051c4688a7759ccbfbb134a9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.015436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7130068051c4688a7759ccbfbb134a9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.016158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7130068051c4688a7759ccbfbb134a9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.033971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7130068051c4688a7759ccbfbb134a9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.036171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf8c6189-893f-4c64-a11f-1a376c2fce5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7130068051c4688a7759ccbfbb134a9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.113949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1534/3077] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:02 2022] DELETE /identity/v3/users/a7130068051c4688a7759ccbfbb134a9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:03.549938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:03.564450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1deca1-5faa-4f26-aa0f-304eb3847fa9 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:03.664924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:03.666634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:03.667282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:03.667517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] PATH_INFO: `/v3/projects/fcd2501efc564e63834afc0a474d486d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:03.669306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd2501efc564e63834afc0a474d486d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:03.671603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:03.768788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd2501efc564e63834afc0a474d486d', 'request_id': 'req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.782224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd2501efc564e63834afc0a474d486d', 'request_id': 'req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.797983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd2501efc564e63834afc0a474d486d', 'request_id': 'req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.821029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] The token cache is being invalidate because project fcd2501efc564e63834afc0a474d486d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:03.822686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.824237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2932cf1a-cef8-4cb3-9f55-07fc6d5182a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:03.825922 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1544/3078] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:03 2022] DELETE /identity/v3/projects/fcd2501efc564e63834afc0a474d486d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:03.831191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152475-285a-4584-b6fe-ac611fafd9a2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:03.831450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152475-285a-4584-b6fe-ac611fafd9a2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:03.831731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f152475-285a-4584-b6fe-ac611fafd9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:03.870615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2f152475-285a-4584-b6fe-ac611fafd9a2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:03.875141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2f152475-285a-4584-b6fe-ac611fafd9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:03.981256 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1535/3079] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:03.991159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd9488-1323-4307-a77c-af494907a62a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:03.991342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd9488-1323-4307-a77c-af494907a62a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:03.991703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd9488-1323-4307-a77c-af494907a62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.020477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-74fd9488-1323-4307-a77c-af494907a62a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:04.021963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-74fd9488-1323-4307-a77c-af494907a62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:04.036075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1545/3080] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:04.036639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f152475-285a-4584-b6fe-ac611fafd9a2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:04.048118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f152475-285a-4584-b6fe-ac611fafd9a2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:04.054555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:04.055388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.056010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.056452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.057280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:04.057869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:04.059646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:04.078153 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1536/3081] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:04.079291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd9488-1323-4307-a77c-af494907a62a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:04.091974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd9488-1323-4307-a77c-af494907a62a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:04.101865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:04.102705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.104187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.104693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.105387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:04.110468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:04.153359 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1546/3082] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:04 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:04.159487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:04.172318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220dbd1-ae0e-4916-8d4e-7dc6f4ea6b70 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:04.256104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:04.256104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.256666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.257075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.257854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:04.261032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:04.271505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:04.323442 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1537/3083] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:04 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:04.327778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:04.341738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44463a5b-37db-44c3-badb-422ca32a7c59 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:04.347753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:04.348427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.349349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.349632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.350554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:04.352225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:04.365324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1547/3084] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:04.371137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:04.390939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f723cb-9fa5-466e-90ba-90789da74063 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:04.396765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fe4cf-ae83-4c7b-9739-1e0e7a3f4ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:04.397667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fe4cf-ae83-4c7b-9739-1e0e7a3f4ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.398111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fe4cf-ae83-4c7b-9739-1e0e7a3f4ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.398626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fe4cf-ae83-4c7b-9739-1e0e7a3f4ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.399446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fe4cf-ae83-4c7b-9739-1e0e7a3f4ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:04.401599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fe4cf-ae83-4c7b-9739-1e0e7a3f4ad7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:04.421554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:04.423407 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1538/3085] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:04.440319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cb4fc-0c08-4e4c-9975-5c0eb08e355a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:04.448157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1857b-dae8-4739-b4b5-aeea18f5234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:04.448995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1857b-dae8-4739-b4b5-aeea18f5234e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.449424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1857b-dae8-4739-b4b5-aeea18f5234e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.449808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1857b-dae8-4739-b4b5-aeea18f5234e admin admin] PATH_INFO: `/v3/projects/639eb01bc0484295b917ab8d0ceee6ab/users/a681592c2d1043a4afa67838812e7fe9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.469207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1857b-dae8-4739-b4b5-aeea18f5234e admin admin] RBAC: Authorizing `identity:create_grant(project_id=639eb01bc0484295b917ab8d0ceee6ab, user_id=a681592c2d1043a4afa67838812e7fe9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:04.473547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1857b-dae8-4739-b4b5-aeea18f5234e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:04.507427 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1548/3086] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:04 2022] PUT /identity/v3/projects/639eb01bc0484295b917ab8d0ceee6ab/users/a681592c2d1043a4afa67838812e7fe9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:04.511816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.512281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.512563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.546716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] MFA Rules not processed for user `a681592c2d1043a4afa67838812e7fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:04.548209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:04.669873 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1539/3087] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:04.688071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.688598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.688598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.723900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] MFA Rules not processed for user `a681592c2d1043a4afa67838812e7fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:04.725498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:04.736134 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1549/3088] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:04.755357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:04.756823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:04.870693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319710b-3e2d-46a6-9e40-4809a7ff9695 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:04.876594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:04.877216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:04.877495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:04.877766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:04.889790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:04.891663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:04.918664 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1540/3089] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:05.629994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:05.631778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:05.689376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:05.752934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab85f8-7e2f-4cd2-8ae5-0fb458b39224 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:05.760396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:05.761450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:05.762119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:05.763313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] PATH_INFO: `/v3/users/5b0c267722f14e7fa87395f796551ab5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:05.765139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0c267722f14e7fa87395f796551ab5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:05.768341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:05.874100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0c267722f14e7fa87395f796551ab5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:05.897483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0c267722f14e7fa87395f796551ab5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:05.922126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0c267722f14e7fa87395f796551ab5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:05.939268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0c267722f14e7fa87395f796551ab5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:05.940266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-863d223a-f07d-406e-9930-211f572cd7fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0c267722f14e7fa87395f796551ab5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:05.959216 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1550/3090] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:05 2022] DELETE /identity/v3/users/5b0c267722f14e7fa87395f796551ab5 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:06.224810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:06.258153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a0db-f52b-45ed-820a-a2bbd7d9cd13 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:06.399201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:06.416383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:06.417043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:06.417326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:06.417602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] PATH_INFO: `/v3/projects/bbee44bfe21341d295b8c2290f692085` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:06.418819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbee44bfe21341d295b8c2290f692085)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:06.420900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:06.504354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbee44bfe21341d295b8c2290f692085', 'request_id': 'req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:06.520310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbee44bfe21341d295b8c2290f692085', 'request_id': 'req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:06.534175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbee44bfe21341d295b8c2290f692085', 'request_id': 'req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:06.550026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] The token cache is being invalidate because project bbee44bfe21341d295b8c2290f692085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:06.550349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:06.551030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-534b1178-cbaa-4dd1-a4b0-7d3a28b95c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:06.553088 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1541/3091] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:06 2022] DELETE /identity/v3/projects/bbee44bfe21341d295b8c2290f692085 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:06.558905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abbcb5-4366-41ab-8c18-af71a4e09a01 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:06.559315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abbcb5-4366-41ab-8c18-af71a4e09a01 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:06.559661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abbcb5-4366-41ab-8c18-af71a4e09a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:06.581845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b8abbcb5-4366-41ab-8c18-af71a4e09a01 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:06.582886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b8abbcb5-4366-41ab-8c18-af71a4e09a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:06.657099 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1551/3092] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:06.664953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594256ca-f4c8-4078-9f1f-4257c1879b2b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:06.665231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594256ca-f4c8-4078-9f1f-4257c1879b2b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:06.666165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594256ca-f4c8-4078-9f1f-4257c1879b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:06.692864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-594256ca-f4c8-4078-9f1f-4257c1879b2b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:06.694125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-594256ca-f4c8-4078-9f1f-4257c1879b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:06.702813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1542/3093] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:06.707236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abbcb5-4366-41ab-8c18-af71a4e09a01 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:06.720540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abbcb5-4366-41ab-8c18-af71a4e09a01 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:06.726316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:06.727123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:06.727343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:06.727626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:06.728989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:06.729430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:06.731104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:06.752751 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1552/3094] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:06.753297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594256ca-f4c8-4078-9f1f-4257c1879b2b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:06.764139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594256ca-f4c8-4078-9f1f-4257c1879b2b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:06.769913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:06.770747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:06.770926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:06.771389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:06.772065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:06.775295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:06.815666 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1543/3095] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:06 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:06.817128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:06.827926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a28fb0-1aee-4567-88f9-4c115828e8e6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:06.855655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:06.902031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:06.902778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:06.903110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:06.903388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:06.904037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:06.906022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:06.933723 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1553/3096] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:06 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:56:06.935056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:06.951353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f729d8-77fc-406c-a159-6cae959c7136 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:06.959210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:06.959528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:06.959816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:06.960254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:06.960920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:06.963381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:06.979373 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1544/3097] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:06.982567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:07.012064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12174b88-81c2-44a2-84ea-abef5276774d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:07.031205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e20dff-e8ba-4a83-b09a-5eb2178304f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:07.031840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e20dff-e8ba-4a83-b09a-5eb2178304f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:07.032122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e20dff-e8ba-4a83-b09a-5eb2178304f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:07.032400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e20dff-e8ba-4a83-b09a-5eb2178304f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:07.033149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e20dff-e8ba-4a83-b09a-5eb2178304f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:07.034777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e20dff-e8ba-4a83-b09a-5eb2178304f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:07.062441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:07.079665 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1554/3098] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:07.084756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53169347-a1a0-4fab-b1a3-24b11ca55432 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:07.116682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:07.122911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:07.123531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:07.125055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] PATH_INFO: `/v3/projects/c14add960abe42f580aeeade3e6f27d9/users/4c3e0565f75c4f558a8936f547ae9d76/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:07.185263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14add960abe42f580aeeade3e6f27d9, user_id=4c3e0565f75c4f558a8936f547ae9d76, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:07.188473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:07.222092 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1545/3099] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:07 2022] PUT /identity/v3/projects/c14add960abe42f580aeeade3e6f27d9/users/4c3e0565f75c4f558a8936f547ae9d76/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:07.227029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:07.229114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:07.229776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:07.300709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:07.301353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:07.457256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1555/3100] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:07.464702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:07.467017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:07.572763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab39e40-99ca-4007-8ff3-f02f185408be admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:07.578964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70636b18-9b65-4694-ad33-8301f30bbeb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:07.579826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636b18-9b65-4694-ad33-8301f30bbeb4 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:07.580207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636b18-9b65-4694-ad33-8301f30bbeb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:07.581162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70636b18-9b65-4694-ad33-8301f30bbeb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:07.595270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70636b18-9b65-4694-ad33-8301f30bbeb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:07.597640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70636b18-9b65-4694-ad33-8301f30bbeb4 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:07.610864 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1546/3101] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:07.940077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:07.943958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:08.064489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2122d-da35-4591-9d71-83edf7016122 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:08.082121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:08.082910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:08.083201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:08.083609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:08.086341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:08.109588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:08.111488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:08.168228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1556/3102] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:09.400845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:09.401394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:09.401525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:09.479685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] MFA Rules not processed for user `fb3043f55aad4e91a1c3ceb734d77af3`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:09.481416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:09.500230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:09.636815 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1547/3103] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:09.645794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:09.651090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:09.748036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:09.851024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b5c2-fe28-4bd5-86cb-10ec2747e433 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:09.870128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:09.872586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:09.872875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:09.873152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:09.889510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:09.891849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:09.933349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1557/3104] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:15.718356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:15.722295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:15.778587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfde45-8a25-4728-a849-57cd2000ebb8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:15.812931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:15.813782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:15.814110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:15.814690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] PATH_INFO: `/v3/users/f74a2c8cd22c454fa8fdd7c2e1ef9230` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:15.816488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f74a2c8cd22c454fa8fdd7c2e1ef9230)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:15.819775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:15.945010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74a2c8cd22c454fa8fdd7c2e1ef9230'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:15.961625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74a2c8cd22c454fa8fdd7c2e1ef9230'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:15.964556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74a2c8cd22c454fa8fdd7c2e1ef9230'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:15.984954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74a2c8cd22c454fa8fdd7c2e1ef9230'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:15.986022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74a2c8cd22c454fa8fdd7c2e1ef9230'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:16.039355 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1548/3105] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:15 2022] DELETE /identity/v3/users/f74a2c8cd22c454fa8fdd7c2e1ef9230 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:16.513459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:16.532939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172c186-a10e-4669-b0d6-6be05571a601 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:16.667683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:16.672604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:16.672905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:16.673196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] PATH_INFO: `/v3/projects/5bd065954b0e4520a3c1deba0916db85` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:16.674548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd065954b0e4520a3c1deba0916db85)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:16.677752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:16.839823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd065954b0e4520a3c1deba0916db85', 'request_id': 'req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:16.851291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd065954b0e4520a3c1deba0916db85', 'request_id': 'req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:16.866690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd065954b0e4520a3c1deba0916db85', 'request_id': 'req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:16.891517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] The token cache is being invalidate because project 5bd065954b0e4520a3c1deba0916db85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:16.891807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:16.892660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:16.894428 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1558/3106] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:16 2022] DELETE /identity/v3/projects/5bd065954b0e4520a3c1deba0916db85 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:16.899054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:16.901415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:16.922636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced450c-bacf-4f17-a3ba-2fe550af41fb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:17.100109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:17.100797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:17.101089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:17.101373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] PATH_INFO: `/v3/users/41e511ffe08c4d89b852ed8d1f228118` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:17.103698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e511ffe08c4d89b852ed8d1f228118)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:17.105670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:17.167046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:17.208594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e511ffe08c4d89b852ed8d1f228118'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:17.226730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e511ffe08c4d89b852ed8d1f228118'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:17.227554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e511ffe08c4d89b852ed8d1f228118'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:17.239638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e511ffe08c4d89b852ed8d1f228118'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:17.240745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ad9fe254-4f5c-4a77-9aed-53e55665b0d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e511ffe08c4d89b852ed8d1f228118'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:17.267456 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1549/3107] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:16 2022] DELETE /identity/v3/users/41e511ffe08c4d89b852ed8d1f228118 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:17.893329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:17.894819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:17.924619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75542ea3-1ce9-48c2-ab47-c4736d1bb3fe admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.031057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:18.062797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.063458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.064204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.064204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] PATH_INFO: `/v3/projects/f74799f9b954449490f73fc5bffef0a1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.065647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74799f9b954449490f73fc5bffef0a1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.076359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:18.089496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3780a81-b8dc-451c-a73f-273d9c084c6f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.091048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3780a81-b8dc-451c-a73f-273d9c084c6f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.091579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3780a81-b8dc-451c-a73f-273d9c084c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.157118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d3780a81-b8dc-451c-a73f-273d9c084c6f None None] MFA Rules not processed for user `4c3e0565f75c4f558a8936f547ae9d76`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:18.160208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3780a81-b8dc-451c-a73f-273d9c084c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.238585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74799f9b954449490f73fc5bffef0a1', 'request_id': 'req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:18.250341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74799f9b954449490f73fc5bffef0a1', 'request_id': 'req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:18.264527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74799f9b954449490f73fc5bffef0a1', 'request_id': 'req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:18.296398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] The token cache is being invalidate because project f74799f9b954449490f73fc5bffef0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:18.296714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:18.297780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38daaefd-80e1-4b0f-9a1d-52cdb653be5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:18.302763 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1559/3108] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:17 2022] DELETE /identity/v3/projects/f74799f9b954449490f73fc5bffef0a1 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:18.308245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ffde-0238-4d4c-a6b6-b8f67f57ede0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.308764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ffde-0238-4d4c-a6b6-b8f67f57ede0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.309135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200ffde-0238-4d4c-a6b6-b8f67f57ede0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.341315 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1550/3109] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:18.355881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edb421-8c1a-4c9b-abbd-084474ec87ca None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.356179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edb421-8c1a-4c9b-abbd-084474ec87ca None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.356494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edb421-8c1a-4c9b-abbd-084474ec87ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.372350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1200ffde-0238-4d4c-a6b6-b8f67f57ede0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:18.373852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1200ffde-0238-4d4c-a6b6-b8f67f57ede0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.427315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c7edb421-8c1a-4c9b-abbd-084474ec87ca None None] MFA Rules not processed for user `4c3e0565f75c4f558a8936f547ae9d76`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:18.430551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c7edb421-8c1a-4c9b-abbd-084474ec87ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.586736 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1560/3110] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:18.601048 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1551/3111] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:18.679245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.680415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.681295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.738625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:18.742583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.765558 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1561/3112] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:18.769936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edb421-8c1a-4c9b-abbd-084474ec87ca None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.788592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edb421-8c1a-4c9b-abbd-084474ec87ca None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.799988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.800984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.801272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.801570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.802290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:18.802604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.809286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:18.826443 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1552/3113] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:18.829216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.831921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:18.841429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.858486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3164a-e8cc-4fde-a327-14d963f75e52 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.865361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.866625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.866917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.867197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.868048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.870715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:18.903782 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1553/3114] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:18.907837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:18.912939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:18.939626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00770faa-1455-456e-bc7f-375c32e84e64 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.951446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:18.984045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ad744-3553-4265-a0ed-7ba9e3c3a4f1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:18.989685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:18.990334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:18.990617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:18.990895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:18.991531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:18.994329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.019645 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1554/3115] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:18 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:56:19.022601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f8b24-f3a5-46da-9245-db0792e9d659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.023233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f8b24-f3a5-46da-9245-db0792e9d659 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.023516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f8b24-f3a5-46da-9245-db0792e9d659 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.023828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f8b24-f3a5-46da-9245-db0792e9d659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.025146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.036896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7c473-f5b5-420a-8d1f-070b1ed45e22 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:19.047100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.047744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.048031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.052026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.052026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.052026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.066704 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1555/3116] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:19.067649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.099396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67e17d-6611-470e-9a56-d98d8dc75f87 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:19.115892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.117045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.117570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.118401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.119159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.121677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.177211 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1556/3117] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:19.181364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.198239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f8b24-f3a5-46da-9245-db0792e9d659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.207572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f8b24-f3a5-46da-9245-db0792e9d659 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.240422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1562/3118] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:19.254704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73079ec5-80a3-49a1-9922-7c010bcf1280 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:19.283353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.284043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.284323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.284587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] PATH_INFO: `/v3/projects/8099a4e8cf9f4faeb6bd20180ad0370f/users/08080d5828234cca9a09e6b852930780/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.351855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8099a4e8cf9f4faeb6bd20180ad0370f, user_id=08080d5828234cca9a09e6b852930780, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.362522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.398265 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1557/3119] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:19 2022] PUT /identity/v3/projects/8099a4e8cf9f4faeb6bd20180ad0370f/users/08080d5828234cca9a09e6b852930780/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:19.409451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1196f-86c8-43ae-b09a-58522ac275d0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.423457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1196f-86c8-43ae-b09a-58522ac275d0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.423773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1196f-86c8-43ae-b09a-58522ac275d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.530314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-62e1196f-86c8-43ae-b09a-58522ac275d0 None None] MFA Rules not processed for user `08080d5828234cca9a09e6b852930780`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:19.536068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-62e1196f-86c8-43ae-b09a-58522ac275d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:19.684060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1563/3120] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:19.686882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.829755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7239c-7b03-4929-a016-e0296df220de admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:19.842957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:19.843671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:19.844016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:19.844352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:19.845054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:19.847391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:19.906820 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1558/3121] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:19 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:19.908362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1196f-86c8-43ae-b09a-58522ac275d0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:19.923260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1196f-86c8-43ae-b09a-58522ac275d0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.018346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:20.054648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.055303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.056181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.056538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.057379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.061352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.100113 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1564/3122] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:19 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:20.104610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:20.123809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa54f-b647-407d-836c-10b02c778ade admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.146153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.147097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.147609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.148007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.148915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.151689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.170011 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1559/3123] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:56:20.174780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:20.187828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f5841-01c3-4aef-9ed1-aaf9ccae9ff8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.199264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.201111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.201492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.202112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] PATH_INFO: `/v3/projects/7ada60446acf4743aac69a994d78cc98/users/bdfbbdf7d7e34a04809212596a517240/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.230961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ada60446acf4743aac69a994d78cc98, user_id=bdfbbdf7d7e34a04809212596a517240, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.236013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.259620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:20.261478 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1565/3124] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:20 2022] PUT /identity/v3/projects/7ada60446acf4743aac69a994d78cc98/users/bdfbbdf7d7e34a04809212596a517240/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:20.273355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1abc6b-a2a1-40db-821c-6d230dd199d9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.313310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:20.407700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b83c70-65f0-4b1b-8f11-5cb99c1acdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.408332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83c70-65f0-4b1b-8f11-5cb99c1acdce admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.408614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83c70-65f0-4b1b-8f11-5cb99c1acdce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.408890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83c70-65f0-4b1b-8f11-5cb99c1acdce admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.409876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b83c70-65f0-4b1b-8f11-5cb99c1acdce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.411675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b83c70-65f0-4b1b-8f11-5cb99c1acdce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.436098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1560/3125] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:20.438484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:20.473990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d398c-c728-4013-841b-b2a999e2c911 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.488186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.489890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.490806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.491552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] PATH_INFO: `/v3/projects/7ada60446acf4743aac69a994d78cc98/users/bdfbbdf7d7e34a04809212596a517240/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.496110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ada60446acf4743aac69a994d78cc98, user_id=bdfbbdf7d7e34a04809212596a517240, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.502459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.525860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1566/3126] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:20 2022] PUT /identity/v3/projects/7ada60446acf4743aac69a994d78cc98/users/bdfbbdf7d7e34a04809212596a517240/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:20.527514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757331-0bea-4d69-ab6a-01bc20a4c6da None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.532226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757331-0bea-4d69-ab6a-01bc20a4c6da None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.534254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757331-0bea-4d69-ab6a-01bc20a4c6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.596186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-af757331-0bea-4d69-ab6a-01bc20a4c6da None None] MFA Rules not processed for user `bdfbbdf7d7e34a04809212596a517240`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:20.597792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af757331-0bea-4d69-ab6a-01bc20a4c6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:20.777643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1561/3127] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:20.778192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:20.790573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd7df7-7660-4b0a-9b75-437a18697fd7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.797768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.798891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.800686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.801303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.802317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:20.802798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.807139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.824539 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1567/3128] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:20.829151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af757331-0bea-4d69-ab6a-01bc20a4c6da None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:20.865096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af757331-0bea-4d69-ab6a-01bc20a4c6da None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:20.882277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:20.883019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:20.884673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:20.885454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:20.887995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:20.891884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:20.970676 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1562/3129] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:20 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:20.983610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:21.012302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc442e91-16c3-4ce3-9c5e-a02ccadf9e26 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:21.211271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a78cb6-1577-4e97-a635-9e919ce6b2c7 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:21.212114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a78cb6-1577-4e97-a635-9e919ce6b2c7 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:21.212684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a78cb6-1577-4e97-a635-9e919ce6b2c7 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:21.213186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a78cb6-1577-4e97-a635-9e919ce6b2c7 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:21.219979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a78cb6-1577-4e97-a635-9e919ce6b2c7 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:21.220389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a78cb6-1577-4e97-a635-9e919ce6b2c7 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:21.226355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a78cb6-1577-4e97-a635-9e919ce6b2c7 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:21.262653 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1568/3130] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:21.420061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:21.665252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfc8d9-172b-4f3e-b6ef-651f1bd4a62f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:21.682108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:21.683076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:21.684296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:21.686574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:21.708035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:21.709673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:21.732963 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1563/3131] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:21.785042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e87c3e-76fe-40b8-964d-3af83b29b7ba None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:21.785488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e87c3e-76fe-40b8-964d-3af83b29b7ba None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:21.786687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e87c3e-76fe-40b8-964d-3af83b29b7ba None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:21.788351 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1569/3132] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:21.794427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:21.803945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:21.814169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:21.845326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292b429-e7bc-4a2e-9989-445c43d3ace0 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:21.864655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5579a0-d9a7-4678-a6d8-d79b87e6826f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:21.865697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5579a0-d9a7-4678-a6d8-d79b87e6826f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:21.866151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5579a0-d9a7-4678-a6d8-d79b87e6826f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:21.866762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5579a0-d9a7-4678-a6d8-d79b87e6826f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/projects/a7dc1f22c575422ea13061e90e0aeb49` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:21.870868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5579a0-d9a7-4678-a6d8-d79b87e6826f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:get_project(project_id=a7dc1f22c575422ea13061e90e0aeb49)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:21.874305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5579a0-d9a7-4678-a6d8-d79b87e6826f tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:21.891460 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1564/3133] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:21 2022] GET /identity/v3/projects/a7dc1f22c575422ea13061e90e0aeb49 => generated 371 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:21.906723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e87c3e-76fe-40b8-964d-3af83b29b7ba None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:21.909271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25e87c3e-76fe-40b8-964d-3af83b29b7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:22.070026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e87c3e-76fe-40b8-964d-3af83b29b7ba None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:22.100347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:22.101968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.102366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.103492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] PATH_INFO: `/v3/users/fb3043f55aad4e91a1c3ceb734d77af3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.108735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3043f55aad4e91a1c3ceb734d77af3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:22.111509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:22.177451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3587a-dd64-41e7-8a43-e324de780747 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.177754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3587a-dd64-41e7-8a43-e324de780747 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.178060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3587a-dd64-41e7-8a43-e324de780747 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.180068 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1565/3134] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:22.188860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3587a-dd64-41e7-8a43-e324de780747 None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:22.190650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:22.206223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3587a-dd64-41e7-8a43-e324de780747 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:22.218475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3587a-dd64-41e7-8a43-e324de780747 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:22.223945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:22.224756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.225027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.225310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/projects/a7dc1f22c575422ea13061e90e0aeb49` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.227108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:get_project(project_id=a7dc1f22c575422ea13061e90e0aeb49)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:22.244127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:22.251324 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1566/3135] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:22 2022] GET /identity/v3/projects/a7dc1f22c575422ea13061e90e0aeb49 => generated 371 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:22.286222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3043f55aad4e91a1c3ceb734d77af3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:22.333507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:22.335942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3043f55aad4e91a1c3ceb734d77af3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:22.347243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7e0c5-4a42-45d0-9b81-c9afb0a9a20c tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:22.352290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3043f55aad4e91a1c3ceb734d77af3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:22.358059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:22.359468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.359949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.367204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3043f55aad4e91a1c3ceb734d77af3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:22.369864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3043f55aad4e91a1c3ceb734d77af3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:22.370825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.372305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:22.372450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:22.378791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:22.402612 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1567/3136] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:22.404924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:22.409289 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1570/3137] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:21 2022] DELETE /identity/v3/users/fb3043f55aad4e91a1c3ceb734d77af3 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:22.410878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:22.448712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6470ee-f924-406f-ae36-847f812ea991 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:22.621685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e82862-c602-414f-a2f5-e894780101dc tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:22.622509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e82862-c602-414f-a2f5-e894780101dc tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.622830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e82862-c602-414f-a2f5-e894780101dc tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.623125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e82862-c602-414f-a2f5-e894780101dc tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.623783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e82862-c602-414f-a2f5-e894780101dc tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:22.626141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e82862-c602-414f-a2f5-e894780101dc tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:22.684013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:22.694661 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1568/3138] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:22 2022] POST /identity/v3/users => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:22.706606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd56e-4032-4d17-b16a-d0622bcd885f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:22.723133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:22.723919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.724375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.724861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.725760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:22.726278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:22.728768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:22.747206 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1571/3139] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:22.768564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e10528-b24b-4eff-9c88-e1644f3da4dd None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.768928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e10528-b24b-4eff-9c88-e1644f3da4dd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.769445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e10528-b24b-4eff-9c88-e1644f3da4dd None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.771014 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1569/3140] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:22.777060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:22.779150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:22.858228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e10528-b24b-4eff-9c88-e1644f3da4dd None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:22.860652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f6e10528-b24b-4eff-9c88-e1644f3da4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:22.940051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:22.950969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5c53a-d365-4c42-b3e0-d9d40da97513 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:22.983105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e10528-b24b-4eff-9c88-e1644f3da4dd None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:22.988420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:22.990101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.990703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.991205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] PATH_INFO: `/v3/projects/2d920a0c09bf4fd4a7b60fc7106a598f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:22.991957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6b637-3580-44ca-875f-bee224cb3879 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:22.992818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6b637-3580-44ca-875f-bee224cb3879 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:22.993682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6b637-3580-44ca-875f-bee224cb3879 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:22.994712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d920a0c09bf4fd4a7b60fc7106a598f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:22.996661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:23.000174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6b637-3580-44ca-875f-bee224cb3879 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/projects/a7dc1f22c575422ea13061e90e0aeb49` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.002917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6b637-3580-44ca-875f-bee224cb3879 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:get_project(project_id=a7dc1f22c575422ea13061e90e0aeb49)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:23.005767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6b637-3580-44ca-875f-bee224cb3879 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:23.013704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1572/3141] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:22 2022] GET /identity/v3/projects/a7dc1f22c575422ea13061e90e0aeb49 => generated 371 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:23.107381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d920a0c09bf4fd4a7b60fc7106a598f', 'request_id': 'req-4173d4d2-a376-4b33-bbed-cd048f9054f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:23.120318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d920a0c09bf4fd4a7b60fc7106a598f', 'request_id': 'req-4173d4d2-a376-4b33-bbed-cd048f9054f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:23.150944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d920a0c09bf4fd4a7b60fc7106a598f', 'request_id': 'req-4173d4d2-a376-4b33-bbed-cd048f9054f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:23.161376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:23.194551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] The token cache is being invalidate because project 2d920a0c09bf4fd4a7b60fc7106a598f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:23.194852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:23.196779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4173d4d2-a376-4b33-bbed-cd048f9054f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:23.200196 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1570/3142] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:22 2022] DELETE /identity/v3/projects/2d920a0c09bf4fd4a7b60fc7106a598f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:23.205101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6259bb-f300-48b0-8f4c-070694621898 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:23.205484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6259bb-f300-48b0-8f4c-070694621898 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:23.205977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6259bb-f300-48b0-8f4c-070694621898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.256288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2b6259bb-f300-48b0-8f4c-070694621898 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:23.258762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2b6259bb-f300-48b0-8f4c-070694621898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:23.370477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:23.370781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:23.371031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.372813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1571/3143] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:23.383070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:23.385453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:23.429190 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1573/3144] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:23.437530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157df5-7fe1-43e1-ae9a-e13b04d89972 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:23.438937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157df5-7fe1-43e1-ae9a-e13b04d89972 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:23.439319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157df5-7fe1-43e1-ae9a-e13b04d89972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.485746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9f157df5-7fe1-43e1-ae9a-e13b04d89972 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:23.489741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9f157df5-7fe1-43e1-ae9a-e13b04d89972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:23.517262 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1574/3145] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:23.518322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f157df5-7fe1-43e1-ae9a-e13b04d89972 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:23.528371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:23.540627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:23.542862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:23.543913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f157df5-7fe1-43e1-ae9a-e13b04d89972 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:23.552371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:23.553278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:23.553799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:23.554249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.555066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:23.555814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:23.590194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:23.608728 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1575/3146] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:23.613887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:23.644535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291cf42-c971-4fd3-9dbf-77fa51451c74 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:23.674827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:23.679749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:23.679969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:23.680242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.682061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:23.684182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:23.740795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5223b8-7367-4a5f-ba1d-33a49cf2e26a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:23.766317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:23.767158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:23.767605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:23.768020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/projects/a7dc1f22c575422ea13061e90e0aeb49` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.771346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:get_project(project_id=a7dc1f22c575422ea13061e90e0aeb49)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:23.772026 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1576/3147] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:23 2022] POST /identity/v3/projects => generated 397 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:23.774719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:23.776915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:23.786671 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1572/3148] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:23 2022] GET /identity/v3/projects/a7dc1f22c575422ea13061e90e0aeb49 => generated 371 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:23.891566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:23.908105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b5db4-bc17-43a5-b76e-e7a16664cada tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:23.946626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bde0e-7c22-49d1-ab06-16caa6dbb46d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:23.953166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:23.953982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:23.954544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:23.954963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:23.955785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:23.959038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:23.991767 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1577/3149] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:23 2022] POST /identity/v3/users => generated 299 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:24.002474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:24.026757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d086f-ec7d-4cea-9692-5a5fa0adcecb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:24.034788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:24.035607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:24.036024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:24.036351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:24.037298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:24.038709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:24.040726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:24.041188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:24.041641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/users/2aedf4145553484089d22f87ae3e5cd9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:24.041728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:24.060265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1578/3150] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:24.063580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:24.088080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:delete_user(user_id=2aedf4145553484089d22f87ae3e5cd9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:24.090138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:24.096456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3c741-43f3-4205-9d71-40d0348a8b36 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:24.136772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:24.137820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:24.138383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:24.139180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:24.139578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:24.142996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:24.145629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:24.172093 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1579/3151] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:24.177047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:24.198327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5729b8f-0606-49eb-97d3-80e0e11db4ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:24.201870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aedf4145553484089d22f87ae3e5cd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.215814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ae4ea-6d13-4437-8b60-f7914b1bf53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:24.216530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae4ea-6d13-4437-8b60-f7914b1bf53e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:24.216803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae4ea-6d13-4437-8b60-f7914b1bf53e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:24.217082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae4ea-6d13-4437-8b60-f7914b1bf53e admin admin] PATH_INFO: `/v3/projects/d1d08fe3ab2a4e8ebe566c13c72d3b5f/users/d6f9c9d09c354be3b9fa0595514745bd/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:24.219728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aedf4145553484089d22f87ae3e5cd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.222171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aedf4145553484089d22f87ae3e5cd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.239525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aedf4145553484089d22f87ae3e5cd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.242524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aedf4145553484089d22f87ae3e5cd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.266332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ae4ea-6d13-4437-8b60-f7914b1bf53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d08fe3ab2a4e8ebe566c13c72d3b5f, user_id=d6f9c9d09c354be3b9fa0595514745bd, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:24.269733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ae4ea-6d13-4437-8b60-f7914b1bf53e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:24.287749 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1573/3152] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:23 2022] DELETE /identity/v3/users/2aedf4145553484089d22f87ae3e5cd9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:24.290934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:24.294473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:24.304103 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1580/3153] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:24 2022] PUT /identity/v3/projects/d1d08fe3ab2a4e8ebe566c13c72d3b5f/users/d6f9c9d09c354be3b9fa0595514745bd/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:24.314386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:24.314676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:24.315048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:24.383270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] MFA Rules not processed for user `d6f9c9d09c354be3b9fa0595514745bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:24.384904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:24.459227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4d441-f7b4-4a5b-a792-eb39f2bf1642 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:24.515642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:24.520951 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1581/3154] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:24.535649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:24.537651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:24.636476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdfbbdf7d7e34a04809212596a517240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ada60446acf4743aac69a994d78cc98', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:24.639102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:24.639682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:24.639988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] PATH_INFO: `/v3/projects/a7dc1f22c575422ea13061e90e0aeb49` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:24.642879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a7dc1f22c575422ea13061e90e0aeb49)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:24.651737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:24.718375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734512b-8da0-4c4c-a6d7-05c4e06364b7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:24.726032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:24.729428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:24.732623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:24.733338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:24.752118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:24.754175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:24.776353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:24.778853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7dc1f22c575422ea13061e90e0aeb49', 'request_id': 'req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.790687 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1582/3155] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:24 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:24.791664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:24.800295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:24.821402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7dc1f22c575422ea13061e90e0aeb49', 'request_id': 'req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.836827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7dc1f22c575422ea13061e90e0aeb49', 'request_id': 'req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.858720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] The token cache is being invalidate because project a7dc1f22c575422ea13061e90e0aeb49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:24.859823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.861074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:24.864676 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1574/3156] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:24 2022] DELETE /identity/v3/projects/a7dc1f22c575422ea13061e90e0aeb49 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:24.872248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:24.883605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:24.949557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a65ef1-79ad-4d69-859a-7799c528b5b3 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:25.095563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84aa1b-bb4a-4382-8f2a-1ee92702e804 tempest-QuotasAdminTestV257-1397415925 tempest-QuotasAdminTestV257-1397415925-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:25.102797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:25.103441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.103718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.104000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] PATH_INFO: `/v3/users/08080d5828234cca9a09e6b852930780` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.106704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=08080d5828234cca9a09e6b852930780)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:25.108641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:25.166202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:25.166934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.167385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.173197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] PATH_INFO: `/v3/users/870c4c66a15b4546bd981cfb5932b214` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.176973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870c4c66a15b4546bd981cfb5932b214)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:25.179112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:25.251774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08080d5828234cca9a09e6b852930780'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.264810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08080d5828234cca9a09e6b852930780'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.266342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08080d5828234cca9a09e6b852930780'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.281808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08080d5828234cca9a09e6b852930780'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.283718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bc2d632-3a0b-42fe-8f6e-a09e6732ba5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08080d5828234cca9a09e6b852930780'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.289190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:25.319111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870c4c66a15b4546bd981cfb5932b214'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.342439 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1575/3157] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:24 2022] DELETE /identity/v3/users/08080d5828234cca9a09e6b852930780 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:25.349327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.349625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.349931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.351526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870c4c66a15b4546bd981cfb5932b214'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.361586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870c4c66a15b4546bd981cfb5932b214'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.371808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870c4c66a15b4546bd981cfb5932b214'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.372630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-feee82b2-f05b-4047-8226-e03b6159abf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870c4c66a15b4546bd981cfb5932b214'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.386208 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1583/3158] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:24 2022] DELETE /identity/v3/users/870c4c66a15b4546bd981cfb5932b214 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:25.391636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.392002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.392357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.397771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:25.399118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:25.425906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:25.428748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:25.517812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:25.532880 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1584/3159] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:25.551236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:25.552759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:25.561543 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1576/3160] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:25.569142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:25.570550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:25.652011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659bdb2-4a4e-42e7-9a51-365a83015247 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:25.654895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:25.655516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.656240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.656240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.682329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:25.682835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:25.696653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e6422-5de0-4154-a492-a494e39335b6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:25.708598 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1585/3161] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:56:25.709638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:25.711664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452666f2-bacc-4f65-8da9-ab3f8ec81a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:25.712481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452666f2-bacc-4f65-8da9-ab3f8ec81a7c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.712765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452666f2-bacc-4f65-8da9-ab3f8ec81a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.712864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:25.713034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452666f2-bacc-4f65-8da9-ab3f8ec81a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.728280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452666f2-bacc-4f65-8da9-ab3f8ec81a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:25.728606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cc203-1b19-49af-a86b-589311828096 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:25.732932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452666f2-bacc-4f65-8da9-ab3f8ec81a7c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:25.734858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:25.736336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.736336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.736336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] PATH_INFO: `/v3/users/d6f9c9d09c354be3b9fa0595514745bd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.737210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f9c9d09c354be3b9fa0595514745bd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:25.739675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:25.767113 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1577/3162] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:25.833780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f9c9d09c354be3b9fa0595514745bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.853639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:25.873328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f9c9d09c354be3b9fa0595514745bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.881305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f9c9d09c354be3b9fa0595514745bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.891538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f9c9d09c354be3b9fa0595514745bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.892226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f9c9d09c354be3b9fa0595514745bd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:25.904138 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1586/3163] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:25 2022] DELETE /identity/v3/users/d6f9c9d09c354be3b9fa0595514745bd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:25.906407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:25.906756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:25.907357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:25.937059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:25.938422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:26.066421 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1578/3164] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:26.120802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:26.122458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:26.139335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5a12b-1995-40bb-9f61-29944a67dd9c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:26.144725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:26.145379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:26.145665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:26.146000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] PATH_INFO: `/v3/projects/d969d135196e409584d6647d5186c9c9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:26.147172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d969d135196e409584d6647d5186c9c9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:26.149237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:26.162207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:26.163918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:26.221557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd969d135196e409584d6647d5186c9c9', 'request_id': 'req-e163e4a4-c99e-4907-95f5-2406c5cc25f9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.236438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd969d135196e409584d6647d5186c9c9', 'request_id': 'req-e163e4a4-c99e-4907-95f5-2406c5cc25f9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.249254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd969d135196e409584d6647d5186c9c9', 'request_id': 'req-e163e4a4-c99e-4907-95f5-2406c5cc25f9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.303233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] The token cache is being invalidate because project d969d135196e409584d6647d5186c9c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:26.304072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.304908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.307937 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1587/3165] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:26 2022] DELETE /identity/v3/projects/d969d135196e409584d6647d5186c9c9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:26.312029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:26.315901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:26.356269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c9239-1b0b-45db-8f4f-6b89da44c03c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:26.418947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163e4a4-c99e-4907-95f5-2406c5cc25f9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:26.424818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:26.425458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:26.425745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:26.426084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] PATH_INFO: `/v3/users/ff6039a197324406a774cd55dc06b9be` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:26.427215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6039a197324406a774cd55dc06b9be)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:26.428158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:26.428844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:26.429141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:26.429236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:26.429550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:26.431856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:26.445596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:26.447946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:26.452508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:26.460689 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1579/3166] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:56:26.462496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:26.464384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:26.478741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fd85e-0656-4b25-a8ef-f3031baa31da service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:26.484231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:26.485020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:26.485238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6039a197324406a774cd55dc06b9be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.485599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:26.486057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] PATH_INFO: `/v3/projects/8099a4e8cf9f4faeb6bd20180ad0370f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:26.487635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=8099a4e8cf9f4faeb6bd20180ad0370f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:26.489707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:26.490113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:26.500417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6039a197324406a774cd55dc06b9be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.509205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6039a197324406a774cd55dc06b9be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.527072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6039a197324406a774cd55dc06b9be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.527831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6039a197324406a774cd55dc06b9be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.538731 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1588/3167] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:26 2022] DELETE /identity/v3/users/ff6039a197324406a774cd55dc06b9be => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:26.576446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8099a4e8cf9f4faeb6bd20180ad0370f', 'request_id': 'req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.589552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8099a4e8cf9f4faeb6bd20180ad0370f', 'request_id': 'req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.607631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8099a4e8cf9f4faeb6bd20180ad0370f', 'request_id': 'req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.621074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] The token cache is being invalidate because project 8099a4e8cf9f4faeb6bd20180ad0370f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:26.621459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.622258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.626424 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1580/3168] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:26 2022] DELETE /identity/v3/projects/8099a4e8cf9f4faeb6bd20180ad0370f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:26.630774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:26.632861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:26.717328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635b66-73f1-4851-8134-dbb7380508c4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:26.725425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:26.726900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:26.726900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:26.727451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] PATH_INFO: `/v3/users/bdfbbdf7d7e34a04809212596a517240` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:26.730121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfbbdf7d7e34a04809212596a517240)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:26.731550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:26.808932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfbbdf7d7e34a04809212596a517240'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.825174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfbbdf7d7e34a04809212596a517240'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.835161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfbbdf7d7e34a04809212596a517240'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.850762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfbbdf7d7e34a04809212596a517240'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.854260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfbbdf7d7e34a04809212596a517240'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:26.892067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1589/3169] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:26 2022] DELETE /identity/v3/users/bdfbbdf7d7e34a04809212596a517240 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:27.044780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:27.046283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.188225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06b3d8-3bdb-48d0-9468-88e183bc9eff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:27.267423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:27.268129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.268420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.268700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] PATH_INFO: `/v3/projects/13d6a80cfe5e49eda7f76cde966f9799` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.269871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d6a80cfe5e49eda7f76cde966f9799)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:27.271862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:27.299240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:27.300671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.324492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c127ab7-1469-4025-8cf3-7e1e147185a9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:27.329183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:27.348239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d6a80cfe5e49eda7f76cde966f9799', 'request_id': 'req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.364224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d6a80cfe5e49eda7f76cde966f9799', 'request_id': 'req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.392844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d6a80cfe5e49eda7f76cde966f9799', 'request_id': 'req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.407588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] The token cache is being invalidate because project 13d6a80cfe5e49eda7f76cde966f9799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:27.407927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.408617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c25fb0d4-8d64-4320-b38e-c7dc3cc3aa9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.411274 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1581/3170] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:27 2022] DELETE /identity/v3/projects/13d6a80cfe5e49eda7f76cde966f9799 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:27.416479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.416942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.417302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.426769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:27.449216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:27.449350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.449573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.450262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] PATH_INFO: `/v3/projects/d1d08fe3ab2a4e8ebe566c13c72d3b5f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.451047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d08fe3ab2a4e8ebe566c13c72d3b5f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:27.457636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:27.490376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:27.491615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.509976 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1582/3171] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:27.511437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:27.513442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.523516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d08fe3ab2a4e8ebe566c13c72d3b5f', 'request_id': 'req-ae807625-533c-454f-afdf-d3411a0b8028'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.527727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe040c7-ccd9-49b5-9611-8d503d98f9bc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:27.530272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d08fe3ab2a4e8ebe566c13c72d3b5f', 'request_id': 'req-ae807625-533c-454f-afdf-d3411a0b8028'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.537354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d08fe3ab2a4e8ebe566c13c72d3b5f', 'request_id': 'req-ae807625-533c-454f-afdf-d3411a0b8028'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.553078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] The token cache is being invalidate because project d1d08fe3ab2a4e8ebe566c13c72d3b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:27.553752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.554151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae807625-533c-454f-afdf-d3411a0b8028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.561339 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1590/3172] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:27 2022] DELETE /identity/v3/projects/d1d08fe3ab2a4e8ebe566c13c72d3b5f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:27.561339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30d85c-f518-43c3-a0c6-9bcba0ef5c34 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.561339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30d85c-f518-43c3-a0c6-9bcba0ef5c34 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.561339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30d85c-f518-43c3-a0c6-9bcba0ef5c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.587051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4d30d85c-f518-43c3-a0c6-9bcba0ef5c34 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:27.588515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4d30d85c-f518-43c3-a0c6-9bcba0ef5c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.592980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:27.593719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.594118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.594470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] PATH_INFO: `/v3/projects/7ada60446acf4743aac69a994d78cc98` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.595667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ada60446acf4743aac69a994d78cc98)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:27.597568 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1591/3173] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:27.601068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:27.601166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac69b-67cd-42bb-8ddd-a02d619207ee None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.601166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac69b-67cd-42bb-8ddd-a02d619207ee None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.601166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addac69b-67cd-42bb-8ddd-a02d619207ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.625574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-addac69b-67cd-42bb-8ddd-a02d619207ee None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:27.626854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-addac69b-67cd-42bb-8ddd-a02d619207ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.636415 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1592/3174] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:27.637383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac69b-67cd-42bb-8ddd-a02d619207ee None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:27.648566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addac69b-67cd-42bb-8ddd-a02d619207ee None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:27.654459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf02fe-d4c7-4ef9-95db-b05e852f027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:27.655090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf02fe-d4c7-4ef9-95db-b05e852f027b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.655371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf02fe-d4c7-4ef9-95db-b05e852f027b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.655799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf02fe-d4c7-4ef9-95db-b05e852f027b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.656889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf02fe-d4c7-4ef9-95db-b05e852f027b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:27.657475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf02fe-d4c7-4ef9-95db-b05e852f027b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:27.658920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf02fe-d4c7-4ef9-95db-b05e852f027b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:27.670982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ada60446acf4743aac69a994d78cc98', 'request_id': 'req-34dfcc91-f120-4d77-b7fd-5c923fd088c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.683531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ada60446acf4743aac69a994d78cc98', 'request_id': 'req-34dfcc91-f120-4d77-b7fd-5c923fd088c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.684573 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1593/3175] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:27.686146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e015a-cfce-44a6-b973-4b1517963911 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.686442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e015a-cfce-44a6-b973-4b1517963911 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.686723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e015a-cfce-44a6-b973-4b1517963911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.693997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ada60446acf4743aac69a994d78cc98', 'request_id': 'req-34dfcc91-f120-4d77-b7fd-5c923fd088c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.706840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] The token cache is being invalidate because project 7ada60446acf4743aac69a994d78cc98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:27.706840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.708171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:27.708633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e20e015a-cfce-44a6-b973-4b1517963911 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:27.709863 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1583/3176] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:27 2022] DELETE /identity/v3/projects/7ada60446acf4743aac69a994d78cc98 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:27.711284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:27.712948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.714244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e20e015a-cfce-44a6-b973-4b1517963911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.751266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:27.791849 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1594/3177] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:27.798557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d8ba7-f866-498b-bcf4-0965d23d5015 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.799184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d8ba7-f866-498b-bcf4-0965d23d5015 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.799590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d8ba7-f866-498b-bcf4-0965d23d5015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.802490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfcc91-f120-4d77-b7fd-5c923fd088c4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:27.810823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0f477-ba95-4870-bb1c-089a816754ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:27.811863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0f477-ba95-4870-bb1c-089a816754ac admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.812204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0f477-ba95-4870-bb1c-089a816754ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.812487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0f477-ba95-4870-bb1c-089a816754ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.813129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0f477-ba95-4870-bb1c-089a816754ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:27.818055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0f477-ba95-4870-bb1c-089a816754ac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:27.826043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-842d8ba7-f866-498b-bcf4-0965d23d5015 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:27.827504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-842d8ba7-f866-498b-bcf4-0965d23d5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.839433 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1595/3178] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:27.840743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d8ba7-f866-498b-bcf4-0965d23d5015 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:27.854410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d8ba7-f866-498b-bcf4-0965d23d5015 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:27.861856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:27.862716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.862999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.863598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.864255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:27.865180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:27.866899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:27.878409 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1584/3179] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:27.878807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc804-22be-47eb-8407-c14d2b545b36 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:27.879465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc804-22be-47eb-8407-c14d2b545b36 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:27.880058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc804-22be-47eb-8407-c14d2b545b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:27.886221 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1596/3180] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:27.887475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:27.901298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460583-e21c-44df-86a1-efab8d4611d3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:27.908849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9bddc804-22be-47eb-8407-c14d2b545b36 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:27.910676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bddc804-22be-47eb-8407-c14d2b545b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:27.947617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:28.008692 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1585/3181] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:28.009911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc804-22be-47eb-8407-c14d2b545b36 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.023704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc804-22be-47eb-8407-c14d2b545b36 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.032545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.033436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.033995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.034449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.050430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.050430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.051045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.051045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.051045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.051045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.052525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.052525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.115821 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1597/3182] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:27 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:28.115821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.116138 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1586/3183] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:28.117598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.133657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f90355-3fb5-494d-b570-f3fe7a651cd2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.236382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.238093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.238475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.239010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.240385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:28.240718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.243244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.258429 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1598/3184] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:28.259572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.282505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccef20-87f0-484f-a43d-0e53f0d0efe3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.287093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b5cbd-871a-4a77-9741-68a811787229 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.302926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.304270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.304354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.304723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.305553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.307404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.309635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.310301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.311053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.311342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.312042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.314323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.331050 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1587/3185] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:28.331050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.354540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e64d1-6129-4041-ae78-b3e543dcda55 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.359929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.360824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.361205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.361961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.362715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.364766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.371066 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1599/3186] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:28.372001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.398652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc719760-f86c-48a5-9eca-95d9be6962a9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.404654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.405607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.405607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.405805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.406477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.408470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.424508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1588/3187] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/projects => generated 409 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:28.437853 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1600/3188] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:28.438034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.438139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.439696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824b1e-87c7-4c52-80f4-3559eb43cc47 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.441915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:28.513869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.514503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.514790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.515067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.516011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.517989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.529175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06aab1-bd46-4149-a413-f8b854a32dbe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.534651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.535502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.535795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.536109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.536799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.538626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.546196 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1601/3189] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/users => generated 305 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:28.550075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.551434 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1589/3190] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:28.552488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.564492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91354-ba91-4f62-8c5e-367f4e7bc79d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.565390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff4cc3-f470-4478-9b3a-7379bac75874 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.570085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.571090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.571521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.571865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.572062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.572651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.573218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.573505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.573784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] PATH_INFO: `/v3/projects/0d9e9604a15e4ba1ba5ea54bf57070be/users/ea4e245cc6f044298aac866aabe3ee33/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.574556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.585616 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1590/3191] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:28.586303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.591435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d9e9604a15e4ba1ba5ea54bf57070be, user_id=ea4e245cc6f044298aac866aabe3ee33, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.594333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.596533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:28.597117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dba0f5-0bdd-4d66-adad-3ac24d1fc92c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.603607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe474e27-b8b2-4ec8-9910-a92473b16cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.604347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe474e27-b8b2-4ec8-9910-a92473b16cca admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.604637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe474e27-b8b2-4ec8-9910-a92473b16cca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.604911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe474e27-b8b2-4ec8-9910-a92473b16cca admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.605554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe474e27-b8b2-4ec8-9910-a92473b16cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.607231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe474e27-b8b2-4ec8-9910-a92473b16cca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.613895 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1602/3192] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:28 2022] PUT /identity/v3/projects/0d9e9604a15e4ba1ba5ea54bf57070be/users/ea4e245cc6f044298aac866aabe3ee33/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:28.615440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.619866 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1591/3193] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:28.631619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a66c-2b39-48e4-98db-9c25b70e6a13 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.632059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a66c-2b39-48e4-98db-9c25b70e6a13 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.633903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a66c-2b39-48e4-98db-9c25b70e6a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.675473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bb41a66c-2b39-48e4-98db-9c25b70e6a13 None None] MFA Rules not processed for user `ea4e245cc6f044298aac866aabe3ee33`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:28.676813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb41a66c-2b39-48e4-98db-9c25b70e6a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:28.708893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90fbb7-62ab-4186-996a-447c25ebad58 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.714342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.714962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.715239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.715513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.716152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.717745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.731525 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1603/3194] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:28.731525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.742232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1d229-30dd-48dc-88df-74b866fb2bf0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.750537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.751315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.754202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.754494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] PATH_INFO: `/v3/projects/02dadff0c0cd4e94ab17400e30e43ba6/users/2ec3657e512a404db2270dee3d9d1c65/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.776445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02dadff0c0cd4e94ab17400e30e43ba6, user_id=2ec3657e512a404db2270dee3d9d1c65, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.779361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.795542 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1604/3195] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:28 2022] PUT /identity/v3/projects/02dadff0c0cd4e94ab17400e30e43ba6/users/2ec3657e512a404db2270dee3d9d1c65/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:28.796620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.799405 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1592/3196] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:28.803415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b148aa4-1763-465c-a8c1-fa1576b045b1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.803763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b148aa4-1763-465c-a8c1-fa1576b045b1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.804042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b148aa4-1763-465c-a8c1-fa1576b045b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.833432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2b148aa4-1763-465c-a8c1-fa1576b045b1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:28.833432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b148aa4-1763-465c-a8c1-fa1576b045b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:28.894281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935a8ae-b25a-438a-a33c-2e52fb59b697 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.900640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.901502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.901820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.902774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] PATH_INFO: `/v3/projects/814c2ef6babc42b3ae28b6cd61008615/users/4d2c974d0faa4f1da6237d3fa7b60856/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.913175 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1593/3197] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:28.917652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b148aa4-1763-465c-a8c1-fa1576b045b1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.928882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814c2ef6babc42b3ae28b6cd61008615, user_id=4d2c974d0faa4f1da6237d3fa7b60856, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.931056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b148aa4-1763-465c-a8c1-fa1576b045b1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:28.932438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.936475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60e1d5-2744-444e-9369-c6ec8b614be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:28.937086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60e1d5-2744-444e-9369-c6ec8b614be7 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.937427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60e1d5-2744-444e-9369-c6ec8b614be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.937641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60e1d5-2744-444e-9369-c6ec8b614be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:28.938303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60e1d5-2744-444e-9369-c6ec8b614be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:28.940324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60e1d5-2744-444e-9369-c6ec8b614be7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:28.946293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:28.949452 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1605/3198] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:28 2022] PUT /identity/v3/projects/814c2ef6babc42b3ae28b6cd61008615/users/4d2c974d0faa4f1da6237d3fa7b60856/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:28.955340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:28.957584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:28.976930 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1594/3199] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:28.979570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaae204-f366-498c-a3c7-51c72dbef70f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:28.980876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaae204-f366-498c-a3c7-51c72dbef70f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:28.981233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaae204-f366-498c-a3c7-51c72dbef70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.014927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-efaae204-f366-498c-a3c7-51c72dbef70f None None] MFA Rules not processed for user `4d2c974d0faa4f1da6237d3fa7b60856`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:29.016518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-efaae204-f366-498c-a3c7-51c72dbef70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:29.035450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:29.040313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963dfc3-3a02-401f-9c6a-557d8ccdf6e5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.112434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.113328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.113752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.114218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.119007 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1595/3200] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:29.119007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaae204-f366-498c-a3c7-51c72dbef70f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.197486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.199007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.199211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaae204-f366-498c-a3c7-51c72dbef70f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.204516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.205126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.205410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.205751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.206430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.208527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.218780 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1606/3201] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:29.220220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.235449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b37c4a-ade4-400a-bfc7-3d1422b807cd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.236225 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1596/3202] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:29 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:29.239766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.244743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.245371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.245658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.246067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.246702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.248650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.252995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c37e6-0c4e-4443-9a17-86c26b5052aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.258420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.259125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.259454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.259728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.260384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.262017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.277869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1597/3203] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:29.290287 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1607/3204] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:29 2022] POST /identity/v3/projects => generated 409 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:29.292917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.294645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.316791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc802ce-78c9-40e3-8be8-bc1d79495edf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.425708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:29.431081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548fb42-9973-4ac3-8ad1-f5171f6a0aaf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.438403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.438591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.441464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.441464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.441464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] PATH_INFO: `/v3/projects/7cc76ec8de9c475b9433a1248d255e5a/users/c968baea1b3c43b8967eaa13180b9dd8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.442783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.444033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.444602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.445593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.459651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.474706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc76ec8de9c475b9433a1248d255e5a, user_id=c968baea1b3c43b8967eaa13180b9dd8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.480380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.505187 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1598/3205] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:29 2022] PUT /identity/v3/projects/7cc76ec8de9c475b9433a1248d255e5a/users/c968baea1b3c43b8967eaa13180b9dd8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:29.513188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.522177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1608/3206] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:29 2022] POST /identity/v3/users => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:56:29.530035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.554283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90cb80-963f-4e1e-a261-96272063d9f6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.593316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:29.649123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.650185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.650790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.651163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.651990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.653813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.670968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1609/3207] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:29.674841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.699710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35233d74-119b-4a54-827d-d1e3ab3cd311 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.710978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2661f7-c3ad-4daf-b566-763c1684b898 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.711342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.712622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.712903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.713253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] PATH_INFO: `/v3/projects/d0372a6e4e214c4a9c6ca72325ecfc01/users/0e40bf4f21574030a9f689cea876288e/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.718460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.719389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.719869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.720416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.721355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.723489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.739579 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1599/3208] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:29.740763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.748495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0372a6e4e214c4a9c6ca72325ecfc01, user_id=0e40bf4f21574030a9f689cea876288e, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.753253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.755502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c32f86-8239-471a-b1d9-1dd86e9676f4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:29.761358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a31f9-8d8a-4ce5-8504-baf0bec11966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:29.762683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a31f9-8d8a-4ce5-8504-baf0bec11966 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.763149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a31f9-8d8a-4ce5-8504-baf0bec11966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.763670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a31f9-8d8a-4ce5-8504-baf0bec11966 admin admin] PATH_INFO: `/v3/projects/7cc76ec8de9c475b9433a1248d255e5a/users/c968baea1b3c43b8967eaa13180b9dd8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.766782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a31f9-8d8a-4ce5-8504-baf0bec11966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc76ec8de9c475b9433a1248d255e5a, user_id=c968baea1b3c43b8967eaa13180b9dd8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:29.769752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a31f9-8d8a-4ce5-8504-baf0bec11966 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:29.781991 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1610/3209] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:29 2022] PUT /identity/v3/projects/d0372a6e4e214c4a9c6ca72325ecfc01/users/0e40bf4f21574030a9f689cea876288e/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:29.789025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:29.789543 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1600/3210] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:29 2022] PUT /identity/v3/projects/7cc76ec8de9c475b9433a1248d255e5a/users/c968baea1b3c43b8967eaa13180b9dd8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:29.807697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b819e84-2a16-44f7-9b95-f6d9f69ec43d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:29.809095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b819e84-2a16-44f7-9b95-f6d9f69ec43d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:29.809991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b819e84-2a16-44f7-9b95-f6d9f69ec43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:29.864897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0b819e84-2a16-44f7-9b95-f6d9f69ec43d None None] MFA Rules not processed for user `c968baea1b3c43b8967eaa13180b9dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:29.867492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0b819e84-2a16-44f7-9b95-f6d9f69ec43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:29.910767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f969-7c4c-4d69-b0f7-7ce5e5fba14d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.113594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.119240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.119583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.120069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.120806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.122691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.129189 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1601/3211] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:30.136584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b819e84-2a16-44f7-9b95-f6d9f69ec43d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.147423 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1611/3212] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:30.148491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.151015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b819e84-2a16-44f7-9b95-f6d9f69ec43d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.156326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.156947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.157301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.157586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.158650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:30.158962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.160583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.175295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100fbb0-775b-4dd3-aa37-885a0bc8b82c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.177817 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1602/3213] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:30.179219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.190580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01c53-56e4-40e1-98a7-945d332c3663 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.192268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cba7b-2047-4ae7-a51a-5691f631ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.193187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cba7b-2047-4ae7-a51a-5691f631ee17 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.193625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cba7b-2047-4ae7-a51a-5691f631ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.193984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cba7b-2047-4ae7-a51a-5691f631ee17 admin admin] PATH_INFO: `/v3/projects/d0372a6e4e214c4a9c6ca72325ecfc01/users/0e40bf4f21574030a9f689cea876288e/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.195773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.196345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cba7b-2047-4ae7-a51a-5691f631ee17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0372a6e4e214c4a9c6ca72325ecfc01, user_id=0e40bf4f21574030a9f689cea876288e, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.196526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.196886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.197234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.197983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.199603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cba7b-2047-4ae7-a51a-5691f631ee17 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.200459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.238939 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1612/3214] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:30 2022] PUT /identity/v3/projects/d0372a6e4e214c4a9c6ca72325ecfc01/users/0e40bf4f21574030a9f689cea876288e/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:30.251513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d5aa7-bc50-4279-a818-66ec3635ba75 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.252058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d5aa7-bc50-4279-a818-66ec3635ba75 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.252471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d5aa7-bc50-4279-a818-66ec3635ba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.253147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1603/3215] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:30 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:30.257501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.272898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:30.338380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045dd2-898b-494e-9ba1-06e93e7a0a9d tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.341108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0e3d5aa7-bc50-4279-a818-66ec3635ba75 None None] MFA Rules not processed for user `0e40bf4f21574030a9f689cea876288e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:30.344457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0e3d5aa7-bc50-4279-a818-66ec3635ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:30.344838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.345606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.346535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.347183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.348572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:30.349092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.351079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.377141 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1604/3216] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:30.378308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.380190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:30.382732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:30.549200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de651b73-d6d7-4405-b1f3-80b3a0ea7886 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.560216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db57494-26da-4e71-9d40-32232c2ddc17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.562838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db57494-26da-4e71-9d40-32232c2ddc17 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.563412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db57494-26da-4e71-9d40-32232c2ddc17 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.563712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db57494-26da-4e71-9d40-32232c2ddc17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.588006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db57494-26da-4e71-9d40-32232c2ddc17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.590578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db57494-26da-4e71-9d40-32232c2ddc17 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.595275 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1613/3217] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:30.605130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d5aa7-bc50-4279-a818-66ec3635ba75 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.627907 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1605/3218] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:30.659674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b216-3e76-4a1b-b6cf-e5c0a17a8963 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.660072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b216-3e76-4a1b-b6cf-e5c0a17a8963 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.663403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b216-3e76-4a1b-b6cf-e5c0a17a8963 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.666293 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1606/3219] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:30.669873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.670289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.670672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.696094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:30.697629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:30.764007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d5aa7-bc50-4279-a818-66ec3635ba75 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.782491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.783368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.784602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.784892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] PATH_INFO: `/v3/users/4d2c974d0faa4f1da6237d3fa7b60856` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.789182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2c974d0faa4f1da6237d3fa7b60856)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.795080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.796284 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1607/3220] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:30.798637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:30.828401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.836683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:30.844166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04758b40-a7ef-441b-9ab8-2ec0afabbbc4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:30.855181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:30.856132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:30.856716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:30.857140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/projects/ecb5991c55d944b7b1f57970d2a94bad` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:30.860127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:get_project(project_id=ecb5991c55d944b7b1f57970d2a94bad)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:30.863924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:30.866148 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1608/3221] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:30 2022] GET /identity/v3/projects/ecb5991c55d944b7b1f57970d2a94bad => generated 369 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:30.867073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:30.870452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:30.968423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2c974d0faa4f1da6237d3fa7b60856'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:31.034017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2c974d0faa4f1da6237d3fa7b60856'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:31.066832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2c974d0faa4f1da6237d3fa7b60856'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:31.067861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582aa5-5333-4548-aad1-fb3296a34bbe tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:31.083876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:31.084751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.085042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.085209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2c974d0faa4f1da6237d3fa7b60856'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:31.085310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.085984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c48bef56-ffbb-4f0e-90fc-0b266fd9a0e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2c974d0faa4f1da6237d3fa7b60856'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:31.096607 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1614/3222] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:30 2022] DELETE /identity/v3/users/4d2c974d0faa4f1da6237d3fa7b60856 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:31.101053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8f403-525c-4501-84ff-5950b9e343fc None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.105498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8f403-525c-4501-84ff-5950b9e343fc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.105780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8f403-525c-4501-84ff-5950b9e343fc None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.107395 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1615/3223] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:31.108731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:31.108815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.109106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.109292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.110401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:31.130627 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1609/3224] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:56:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:56:31.137730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:31.139323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:31.159466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:31.166030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:31.215329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:31.229201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:31.262074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc6fbd-ca1d-40a3-941e-1a581beac838 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:31.332913 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1616/3225] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:31.350908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:31.353522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:31.403650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:31.406224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.406641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.407079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/projects/ecb5991c55d944b7b1f57970d2a94bad` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.409813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:get_project(project_id=ecb5991c55d944b7b1f57970d2a94bad)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:31.413270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:31.419717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1610/3226] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:31 2022] GET /identity/v3/projects/ecb5991c55d944b7b1f57970d2a94bad => generated 369 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:31.496507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9107a6b-ecf4-4796-bcf2-5649144ee014 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:31.505315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:31.505965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.507413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.507774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.522209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:31.525127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:31.530036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:31.534643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:31.547679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c49ac-fa9f-45d9-899e-d5d851bc7f8b tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:31.560163 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1617/3227] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:31.565766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:31.566596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.567120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.567613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.568826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:31.569265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:31.571967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:31.605812 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1611/3228] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:31.606826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:31.626303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3951da-3c42-4738-87dc-58642fc86881 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:31.631355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a9a05-af4d-4160-850e-947ff54eb4f4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:31.631699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9a05-af4d-4160-850e-947ff54eb4f4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.631990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9a05-af4d-4160-850e-947ff54eb4f4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.632268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9a05-af4d-4160-850e-947ff54eb4f4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.632912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a9a05-af4d-4160-850e-947ff54eb4f4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:31.640683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a9a05-af4d-4160-850e-947ff54eb4f4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:31.674172 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1618/3229] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:31 2022] POST /identity/v3/users => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:56:31.677630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:31.691122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad320451-1a04-4671-80d0-3b109384fb2e tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:31.703691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:31.704466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.704821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.705172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.705957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:31.707315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:31.709030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:31.743565 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1612/3230] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:31.776609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f000fc-29a8-4981-a57e-742512a9596b None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:31.776810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f000fc-29a8-4981-a57e-742512a9596b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:31.777113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f000fc-29a8-4981-a57e-742512a9596b None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:31.778763 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1619/3231] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:31.799122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:31.801205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:31.935130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:31.978323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:31.996566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39beef0-8bd9-4b17-96d7-4df42bbc1459 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:32.013758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ef2d7-a223-454b-bc4c-69d41087d3b4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:32.014619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ef2d7-a223-454b-bc4c-69d41087d3b4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:32.015025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ef2d7-a223-454b-bc4c-69d41087d3b4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:32.015401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ef2d7-a223-454b-bc4c-69d41087d3b4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/projects/ecb5991c55d944b7b1f57970d2a94bad` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:32.019445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ef2d7-a223-454b-bc4c-69d41087d3b4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:get_project(project_id=ecb5991c55d944b7b1f57970d2a94bad)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:32.023231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ef2d7-a223-454b-bc4c-69d41087d3b4 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:32.025050 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1613/3232] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:31 2022] GET /identity/v3/projects/ecb5991c55d944b7b1f57970d2a94bad => generated 369 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:32.558641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f000fc-29a8-4981-a57e-742512a9596b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:32.561173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-84f000fc-29a8-4981-a57e-742512a9596b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:32.649730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f000fc-29a8-4981-a57e-742512a9596b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:32.690928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:32.691637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:32.691925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:32.692231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] PATH_INFO: `/v3/projects/814c2ef6babc42b3ae28b6cd61008615` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:32.694094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=814c2ef6babc42b3ae28b6cd61008615)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:32.697575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:32.732762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a407f3-eb7f-4525-9607-192d67c7e97e None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:32.733276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a407f3-eb7f-4525-9607-192d67c7e97e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:32.733276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a407f3-eb7f-4525-9607-192d67c7e97e None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:32.741900 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1614/3233] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:32.750442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a407f3-eb7f-4525-9607-192d67c7e97e None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:32.788064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a407f3-eb7f-4525-9607-192d67c7e97e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:32.820245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a407f3-eb7f-4525-9607-192d67c7e97e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:32.837219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814c2ef6babc42b3ae28b6cd61008615', 'request_id': 'req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:32.851769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814c2ef6babc42b3ae28b6cd61008615', 'request_id': 'req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:32.873328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814c2ef6babc42b3ae28b6cd61008615', 'request_id': 'req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:32.901752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] The token cache is being invalidate because project 814c2ef6babc42b3ae28b6cd61008615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:32.902058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:32.903122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:32.909514 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1620/3234] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:32 2022] DELETE /identity/v3/projects/814c2ef6babc42b3ae28b6cd61008615 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:32.911831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:32.914641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:33.022241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:33.022893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:33.023177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:33.023458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/projects/ecb5991c55d944b7b1f57970d2a94bad` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:33.031366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:get_project(project_id=ecb5991c55d944b7b1f57970d2a94bad)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:33.033527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:33.043824 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1615/3235] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:32 2022] GET /identity/v3/projects/ecb5991c55d944b7b1f57970d2a94bad => generated 369 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:33.053779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:33.092933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:33.096151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:33.133116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db97aa-873a-4e22-b75d-0f5ecbd28ed2 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:33.148300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40675aa4-c56c-4bf9-87b5-3c28b4a14c0c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:33.153924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:33.156807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:33.157129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:33.157411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/users/bcba5f391e5845c88802f9b67012c019` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:33.173286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:33.174974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:33.176036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:33.176483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] PATH_INFO: `/v3/users/0e40bf4f21574030a9f689cea876288e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:33.182636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e40bf4f21574030a9f689cea876288e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:33.187809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:33.224004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:delete_user(user_id=bcba5f391e5845c88802f9b67012c019)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:33.230402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:33.345610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e40bf4f21574030a9f689cea876288e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.365240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e40bf4f21574030a9f689cea876288e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.375866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e40bf4f21574030a9f689cea876288e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.388708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcba5f391e5845c88802f9b67012c019'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.402528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e40bf4f21574030a9f689cea876288e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.406711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e40bf4f21574030a9f689cea876288e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.407545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcba5f391e5845c88802f9b67012c019'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.408503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcba5f391e5845c88802f9b67012c019'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.421074 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1621/3236] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:32 2022] DELETE /identity/v3/users/0e40bf4f21574030a9f689cea876288e => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:33.429255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcba5f391e5845c88802f9b67012c019'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.430288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcba5f391e5845c88802f9b67012c019'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:33.461864 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1616/3237] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:33 2022] DELETE /identity/v3/users/bcba5f391e5845c88802f9b67012c019 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:33.463584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:33.465264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:33.557772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131dbb3-e882-4f4e-a27a-11c9ab594ea7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:33.747814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c968baea1b3c43b8967eaa13180b9dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7cc76ec8de9c475b9433a1248d255e5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:33.748506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:33.754403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:33.757065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] PATH_INFO: `/v3/projects/ecb5991c55d944b7b1f57970d2a94bad` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:33.763973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ecb5991c55d944b7b1f57970d2a94bad)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:33.771365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:33.820569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:33.988172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb5991c55d944b7b1f57970d2a94bad', 'request_id': 'req-88f10cf3-1fac-4292-a94a-aed3c31caabd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.010826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb5991c55d944b7b1f57970d2a94bad', 'request_id': 'req-88f10cf3-1fac-4292-a94a-aed3c31caabd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.040470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb5991c55d944b7b1f57970d2a94bad', 'request_id': 'req-88f10cf3-1fac-4292-a94a-aed3c31caabd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.106048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] The token cache is being invalidate because project ecb5991c55d944b7b1f57970d2a94bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:34.106352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.109287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-88f10cf3-1fac-4292-a94a-aed3c31caabd tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.118330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1622/3238] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:33 2022] DELETE /identity/v3/projects/ecb5991c55d944b7b1f57970d2a94bad => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:34.120286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:34.122685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:34.328215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:34.367582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927c7a6-010b-4fed-96c5-68de665759f3 tempest-QuotasAdminTestV236-1898887567 tempest-QuotasAdminTestV236-1898887567-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:34.547773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:34.548427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:34.548707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:34.548982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] PATH_INFO: `/v3/users/ea4e245cc6f044298aac866aabe3ee33` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:34.550301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4e245cc6f044298aac866aabe3ee33)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:34.552416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:34.649618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4e245cc6f044298aac866aabe3ee33'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.658934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4e245cc6f044298aac866aabe3ee33'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.659668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4e245cc6f044298aac866aabe3ee33'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.670882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4e245cc6f044298aac866aabe3ee33'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.671730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4e245cc6f044298aac866aabe3ee33'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:34.699614 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1617/3239] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:34 2022] DELETE /identity/v3/users/ea4e245cc6f044298aac866aabe3ee33 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:34.703863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:34.704351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:34.704808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:34.736372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:34.740173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:34.886702 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1623/3240] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:34.923051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:34.925109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:34.947982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5853e4-6948-4063-a076-a825bcd2c6eb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:34.956606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:34.959007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:35.002177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:35.005689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:35.006504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:35.007743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] PATH_INFO: `/v3/projects/d0372a6e4e214c4a9c6ca72325ecfc01` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:35.012718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0372a6e4e214c4a9c6ca72325ecfc01)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:35.021135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:35.080439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab5b5a-6f12-4521-a3a4-0f6f24111b6b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:35.090467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e18dd-a958-4b38-b300-c4d4a011f283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:35.091400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e18dd-a958-4b38-b300-c4d4a011f283 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:35.092123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e18dd-a958-4b38-b300-c4d4a011f283 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:35.092494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e18dd-a958-4b38-b300-c4d4a011f283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:35.111703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e18dd-a958-4b38-b300-c4d4a011f283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:35.114344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e18dd-a958-4b38-b300-c4d4a011f283 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:35.129096 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1624/3241] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:35.150910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0372a6e4e214c4a9c6ca72325ecfc01', 'request_id': 'req-cba8c44c-dc13-490f-b859-ec8b16220217'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:35.162554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0372a6e4e214c4a9c6ca72325ecfc01', 'request_id': 'req-cba8c44c-dc13-490f-b859-ec8b16220217'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:35.179621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0372a6e4e214c4a9c6ca72325ecfc01', 'request_id': 'req-cba8c44c-dc13-490f-b859-ec8b16220217'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:35.193299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:35.202223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] The token cache is being invalidate because project d0372a6e4e214c4a9c6ca72325ecfc01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:35.204020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:35.204976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cba8c44c-dc13-490f-b859-ec8b16220217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:35.211839 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1618/3242] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:34 2022] DELETE /identity/v3/projects/d0372a6e4e214c4a9c6ca72325ecfc01 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:35.214330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a1b03-5b69-40c1-93be-95faf8bd94f7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:35.214836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a1b03-5b69-40c1-93be-95faf8bd94f7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:35.215888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a1b03-5b69-40c1-93be-95faf8bd94f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:35.268623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ec2a1b03-5b69-40c1-93be-95faf8bd94f7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:35.270741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ec2a1b03-5b69-40c1-93be-95faf8bd94f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:35.372311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:35.447839 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1625/3243] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:35.455930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b79269-f331-4d29-82bd-7f9112754a23 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:35.456218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b79269-f331-4d29-82bd-7f9112754a23 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:35.456501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b79269-f331-4d29-82bd-7f9112754a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:35.491729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-94b79269-f331-4d29-82bd-7f9112754a23 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:35.493558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-94b79269-f331-4d29-82bd-7f9112754a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:35.522862 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1619/3244] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:35.527834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a1b03-5b69-40c1-93be-95faf8bd94f7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:35.549301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a1b03-5b69-40c1-93be-95faf8bd94f7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:35.562632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:35.566691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:35.566982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:35.567264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:35.577068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:35.577392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:35.579087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:35.664590 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1626/3245] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:35.666458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b79269-f331-4d29-82bd-7f9112754a23 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:35.680802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b79269-f331-4d29-82bd-7f9112754a23 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:35.694115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:35.694750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:35.695328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:35.698483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:35.701382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:35.703374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:35.761752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:35.763562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:35.768432 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1620/3246] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:35 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:35.770967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:35.828142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6ae2-fd27-48b4-9e45-4e6614c6702c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:35.924959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db991091-2c14-4616-bf58-b37e4890705d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:35.937242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:35.937963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:35.938254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:35.940841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:35.944280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:35.949966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:36.032843 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1621/3247] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:35 2022] POST /identity/v3/users => generated 296 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:36.037133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:36.040637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:36.041383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:36.041801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:36.042109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] PATH_INFO: `/v3/projects/0d9e9604a15e4ba1ba5ea54bf57070be` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:36.043919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d9e9604a15e4ba1ba5ea54bf57070be)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:36.060405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:36.062639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63104ce-8de7-4a3f-a1e1-679c0977dd95 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:36.069758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:36.071661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:36.072391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:36.072814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:36.073813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:36.076144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:36.114187 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1622/3248] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:36.119036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:36.134421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1150ae4-27d4-4d05-bfd2-aebd39eb868c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:36.134877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d9e9604a15e4ba1ba5ea54bf57070be', 'request_id': 'req-5875f334-5c25-4005-ab47-273e6b7cb766'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.142385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d9e9604a15e4ba1ba5ea54bf57070be', 'request_id': 'req-5875f334-5c25-4005-ab47-273e6b7cb766'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.153753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d9e9604a15e4ba1ba5ea54bf57070be', 'request_id': 'req-5875f334-5c25-4005-ab47-273e6b7cb766'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.179630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] The token cache is being invalidate because project 0d9e9604a15e4ba1ba5ea54bf57070be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:36.181662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.183119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.188793 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1627/3249] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:35 2022] DELETE /identity/v3/projects/0d9e9604a15e4ba1ba5ea54bf57070be => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:36.189965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:36.191665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:36.265382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:36.266045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:36.266335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:36.266604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:36.267265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:36.268874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:36.290946 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1623/3250] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:36.294047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:36.302925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:36.329377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14de666-7ce6-40cc-9c95-47b86007801b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:36.346626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eedbda-d614-492c-a051-5d2ee6fdcde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:36.347595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eedbda-d614-492c-a051-5d2ee6fdcde0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:36.347890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eedbda-d614-492c-a051-5d2ee6fdcde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:36.348173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eedbda-d614-492c-a051-5d2ee6fdcde0 admin admin] PATH_INFO: `/v3/projects/a9327061a3ba468f83e978315b6ed753/users/87df880447e941f0b336470ac8aba19f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:36.385098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eedbda-d614-492c-a051-5d2ee6fdcde0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9327061a3ba468f83e978315b6ed753, user_id=87df880447e941f0b336470ac8aba19f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:36.392984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eedbda-d614-492c-a051-5d2ee6fdcde0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:36.421285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1624/3251] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:36 2022] PUT /identity/v3/projects/a9327061a3ba468f83e978315b6ed753/users/87df880447e941f0b336470ac8aba19f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:36.422196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:36.435697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c20d7e-b831-4977-ba68-e52f8bf0c342 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:36.435697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c20d7e-b831-4977-ba68-e52f8bf0c342 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:36.435697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c20d7e-b831-4977-ba68-e52f8bf0c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:36.450676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875f334-5c25-4005-ab47-273e6b7cb766 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:36.510778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-02c20d7e-b831-4977-ba68-e52f8bf0c342 None None] MFA Rules not processed for user `87df880447e941f0b336470ac8aba19f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:36.512134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-02c20d7e-b831-4977-ba68-e52f8bf0c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:36.531317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:36.597382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:36.598133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:36.598791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:36.598791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] PATH_INFO: `/v3/users/c968baea1b3c43b8967eaa13180b9dd8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:36.599909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c968baea1b3c43b8967eaa13180b9dd8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:36.603349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:36.673283 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1625/3252] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:36.677070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c20d7e-b831-4977-ba68-e52f8bf0c342 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:36.696949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c20d7e-b831-4977-ba68-e52f8bf0c342 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:36.704144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c968baea1b3c43b8967eaa13180b9dd8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.709483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:36.710225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:36.710631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:36.710916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:36.712771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:36.717129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:36.729907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c968baea1b3c43b8967eaa13180b9dd8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.749880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c968baea1b3c43b8967eaa13180b9dd8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.761538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c968baea1b3c43b8967eaa13180b9dd8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.762138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c968baea1b3c43b8967eaa13180b9dd8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:36.775394 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1628/3253] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:36 2022] DELETE /identity/v3/users/c968baea1b3c43b8967eaa13180b9dd8 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:36.783337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:36.786758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:36.788298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1626/3254] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:36 2022] POST /identity/v3/projects => generated 393 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:37.032715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c10fc-2477-4875-a239-773cacf2d364 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:37.055377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:37.057852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:37.058138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:37.058422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:37.064158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:37.069990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:37.156460 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1629/3255] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:36 2022] POST /identity/v3/users => generated 303 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:37.161598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:37.306346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecf153-d8a7-4d59-b2b8-8d1331acb460 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:37.328755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:37.329672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:37.330050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:37.330399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:37.331137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:37.332819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:37.362901 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1627/3256] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:37.367309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:37.386585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba568d-b1eb-471b-a495-e2e9bbe2c1e5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:37.393233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:37.394249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:37.394705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:37.395503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] PATH_INFO: `/v3/projects/275fe06a58ac4e0994c87079255a7faa/users/860c77c7bfb14a69a9a1ea46446621c7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:37.412969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:37.433412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=275fe06a58ac4e0994c87079255a7faa, user_id=860c77c7bfb14a69a9a1ea46446621c7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:37.439775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:37.482330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1630/3257] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:37 2022] PUT /identity/v3/projects/275fe06a58ac4e0994c87079255a7faa/users/860c77c7bfb14a69a9a1ea46446621c7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:37.493295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:37.520213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bfadf-3c2a-4a0a-90fe-cf6df1484c7a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:37.623706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:37.629975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:37.805719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:37.812055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:37.812340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:37.812591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:37.815079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:37.820269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:37.850181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1683f8-3b56-49dd-a6a6-15439e21f38b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:37.858576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:37.859274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:37.859561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:37.859841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] PATH_INFO: `/v3/projects/7cc76ec8de9c475b9433a1248d255e5a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:37.866518 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1628/3258] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:37.870277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc76ec8de9c475b9433a1248d255e5a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:37.872316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:37.876656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:37.906086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c6195-6dcc-44bb-bcf7-8e651894562f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:37.921640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5013e-5e9d-44f5-9487-3e0a19c50190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:37.922646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5013e-5e9d-44f5-9487-3e0a19c50190 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:37.923111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5013e-5e9d-44f5-9487-3e0a19c50190 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:37.923554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5013e-5e9d-44f5-9487-3e0a19c50190 admin admin] PATH_INFO: `/v3/projects/275fe06a58ac4e0994c87079255a7faa/users/860c77c7bfb14a69a9a1ea46446621c7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:37.935386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5013e-5e9d-44f5-9487-3e0a19c50190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275fe06a58ac4e0994c87079255a7faa, user_id=860c77c7bfb14a69a9a1ea46446621c7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:37.944357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5013e-5e9d-44f5-9487-3e0a19c50190 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:37.986720 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1629/3259] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:37 2022] PUT /identity/v3/projects/275fe06a58ac4e0994c87079255a7faa/users/860c77c7bfb14a69a9a1ea46446621c7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:38.000231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91e00-2665-4252-b290-b7bce8933d7c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:38.001716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91e00-2665-4252-b290-b7bce8933d7c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:38.002206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91e00-2665-4252-b290-b7bce8933d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:38.022830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc76ec8de9c475b9433a1248d255e5a', 'request_id': 'req-fb2e96f6-5be0-4416-b236-f6185dcc3a42'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:38.033225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc76ec8de9c475b9433a1248d255e5a', 'request_id': 'req-fb2e96f6-5be0-4416-b236-f6185dcc3a42'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:38.051011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc76ec8de9c475b9433a1248d255e5a', 'request_id': 'req-fb2e96f6-5be0-4416-b236-f6185dcc3a42'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:38.081187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:38.091250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] The token cache is being invalidate because project 7cc76ec8de9c475b9433a1248d255e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:38.091786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:38.095797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e96f6-5be0-4416-b236-f6185dcc3a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:38.100237 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1631/3260] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:37 2022] DELETE /identity/v3/projects/7cc76ec8de9c475b9433a1248d255e5a => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:38.106574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-51c91e00-2665-4252-b290-b7bce8933d7c None None] MFA Rules not processed for user `860c77c7bfb14a69a9a1ea46446621c7`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:38.108449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-51c91e00-2665-4252-b290-b7bce8933d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:38.108804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaba5c4-b4e1-43ba-baee-78d48ba977fc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:38.109295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaba5c4-b4e1-43ba-baee-78d48ba977fc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:38.109750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaba5c4-b4e1-43ba-baee-78d48ba977fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:38.181382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fbaba5c4-b4e1-43ba-baee-78d48ba977fc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:38.188140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fbaba5c4-b4e1-43ba-baee-78d48ba977fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:38.372566 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1632/3261] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:38.389408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9171e5-53dd-40c0-a6b8-ba9288158d0d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:38.389729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9171e5-53dd-40c0-a6b8-ba9288158d0d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:38.390046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9171e5-53dd-40c0-a6b8-ba9288158d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:38.390828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1630/3262] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:38.410098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:38.410708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:38.412171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:38.448382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0d9171e5-53dd-40c0-a6b8-ba9288158d0d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:38.454707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0d9171e5-53dd-40c0-a6b8-ba9288158d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:38.475872 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1633/3263] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:38.480654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] MFA Rules not processed for user `87df880447e941f0b336470ac8aba19f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:38.481534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9171e5-53dd-40c0-a6b8-ba9288158d0d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:38.482150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:38.494983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9171e5-53dd-40c0-a6b8-ba9288158d0d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:38.535640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:38.537763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:38.538678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:38.539734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:38.579580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:38.581473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:38.583292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:38.593937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:38.610489 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1634/3264] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:38.611453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:38.647206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e57b9-0f3a-4870-8b72-523e6745f245 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:38.675292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1631/3265] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:38.684201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:38.684838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:38.688270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:38.690178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:38.690178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:38.690178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:38.691962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:38.694674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:38.770116 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1635/3266] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:38 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:38.773156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:38.960634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52768a-3241-4efd-8dec-95fc06e73cab None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:38.999784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f359db8-e532-4e37-8296-e04966884512 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:39.005774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:39.006770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:39.007063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:39.007340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:39.008563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:39.011008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:39.057451 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1636/3267] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:38 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:39.058986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:39.076573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd044b-8ae8-4cc4-bf5b-5d27d220e269 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:39.082538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:39.083928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:39.084262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:39.084539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:39.085190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:39.090312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:39.116257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:39.132448 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1637/3268] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:39.135381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:39.176992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4291c3b-8548-4cf7-9638-3ef8ef8caf5c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:39.201862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:39.201862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:39.201862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:39.205397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:39.209926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:39.209926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:39.245071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8c57d-0a42-4230-a022-3d994ceb1d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:39.246135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8c57d-0a42-4230-a022-3d994ceb1d60 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:39.246426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8c57d-0a42-4230-a022-3d994ceb1d60 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:39.248583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8c57d-0a42-4230-a022-3d994ceb1d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:39.250619 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1638/3269] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:39.255489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-94f8c57d-0a42-4230-a022-3d994ceb1d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:39.258178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:39.296781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7aec-40b0-4952-ae7d-1b03f216c257 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:39.316759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:39.322698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:39.323025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:39.323511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15/users/a72b2314d1d2485fb47f53ed60195110/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:39.384634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57854db577054fa3acf6b2ff3704ab15, user_id=a72b2314d1d2485fb47f53ed60195110, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:39.395240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:39.408094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8c57d-0a42-4230-a022-3d994ceb1d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:39.410476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8c57d-0a42-4230-a022-3d994ceb1d60 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:39.423321 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1632/3270] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:39.456587 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1639/3271] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:39 2022] PUT /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15/users/a72b2314d1d2485fb47f53ed60195110/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:39.457461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b0c78-bf7e-4d0e-9f5d-64a48f06355d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:39.457888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b0c78-bf7e-4d0e-9f5d-64a48f06355d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:39.458305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b0c78-bf7e-4d0e-9f5d-64a48f06355d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:39.514756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ea2b0c78-bf7e-4d0e-9f5d-64a48f06355d None None] MFA Rules not processed for user `a72b2314d1d2485fb47f53ed60195110`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:39.521965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ea2b0c78-bf7e-4d0e-9f5d-64a48f06355d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:39.655564 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1633/3272] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:39.661542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:40.044390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0e838-8ae9-4dbb-8fd0-381b37374e5b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:40.078011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:40.078969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:40.079256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:40.079877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:40.080179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:40.082630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:40.136787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1640/3273] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:39 2022] POST /identity/v3/projects => generated 379 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:40.141397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b0c78-bf7e-4d0e-9f5d-64a48f06355d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:40.164281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b0c78-bf7e-4d0e-9f5d-64a48f06355d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:40.336247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:40.336887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:40.337371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:40.337652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:40.338325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:40.348947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:40.392480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:40.444331 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1634/3274] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:40 2022] POST /identity/v3/users => generated 296 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:40.455710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:40.467851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:40.495303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0af15-3889-4f84-9ca0-278bccb0e16f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:40.511214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:40.512408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:40.512838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:40.513542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:40.514277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:40.521374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:40.536383 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1641/3275] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:40.541912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:40.561798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeafe4-69c8-42c6-a15f-b2e182ea19ad admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:40.589386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:40.589636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:40.589951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:40.590665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] PATH_INFO: `/v3/projects/032e998177cf4696b70d6c89a5ddd3a8/users/2698bd7906e2463e85c60ce79f33c240/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:40.630848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032e998177cf4696b70d6c89a5ddd3a8, user_id=2698bd7906e2463e85c60ce79f33c240, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:40.644442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:40.692464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1635/3276] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:40 2022] PUT /identity/v3/projects/032e998177cf4696b70d6c89a5ddd3a8/users/2698bd7906e2463e85c60ce79f33c240/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:40.696481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:40.715894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d0128-e442-4a3a-9f2f-1a794b29d087 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:40.961142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270284b-f8f6-4818-bf36-f5c23675a83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:40.961852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270284b-f8f6-4818-bf36-f5c23675a83c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:40.962158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270284b-f8f6-4818-bf36-f5c23675a83c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:40.962433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270284b-f8f6-4818-bf36-f5c23675a83c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:40.963095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270284b-f8f6-4818-bf36-f5c23675a83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:40.983830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270284b-f8f6-4818-bf36-f5c23675a83c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:41.036099 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1642/3277] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:41.036658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:41.052784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae64b8-68ae-440b-859c-d2c9ef5e4f64 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:41.071599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:41.079152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:41.079152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:41.082422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] PATH_INFO: `/v3/projects/032e998177cf4696b70d6c89a5ddd3a8/users/2698bd7906e2463e85c60ce79f33c240/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:41.088358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032e998177cf4696b70d6c89a5ddd3a8, user_id=2698bd7906e2463e85c60ce79f33c240, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:41.094890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:41.119550 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1636/3278] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:41 2022] PUT /identity/v3/projects/032e998177cf4696b70d6c89a5ddd3a8/users/2698bd7906e2463e85c60ce79f33c240/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:41.130059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa3d5-396d-4496-a313-b1bacac58f5b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:41.130613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa3d5-396d-4496-a313-b1bacac58f5b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:41.130898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa3d5-396d-4496-a313-b1bacac58f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:41.209375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d6aa3d5-396d-4496-a313-b1bacac58f5b None None] MFA Rules not processed for user `2698bd7906e2463e85c60ce79f33c240`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:41.211473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d6aa3d5-396d-4496-a313-b1bacac58f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:41.395220 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1643/3279] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:41.410189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:41.411933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:41.622426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837b1c9-74df-423a-be32-6abfdb4c3318 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:41.632016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:41.633563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:41.634200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:41.634853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:41.651603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:41.653579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:41.676214 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1637/3280] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:41.710491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac5b0-5016-422d-a183-f4854646e173 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:41.710725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac5b0-5016-422d-a183-f4854646e173 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:41.710866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ac5b0-5016-422d-a183-f4854646e173 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:41.713239 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1644/3281] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:41.725394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:41.741612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:41.753672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546b23-9731-426c-badd-e5c67ce90cef service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:41.762062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:41.762668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:41.762962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:41.763243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:41.764945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:41.771621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:41.786454 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1638/3282] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:41 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:41.824270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b593-8520-42b9-a0ca-eaf1902f0858 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:41.824910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b593-8520-42b9-a0ca-eaf1902f0858 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:41.825206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b593-8520-42b9-a0ca-eaf1902f0858 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:41.828774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1645/3283] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:41.834137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:41.852339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:41.864296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26d5f5-196a-4e42-a949-e7bc1ed0d145 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:41.870823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:41.871552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:41.871840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:41.874638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:41.876734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:41.879489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:41.893150 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1639/3284] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:41 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:42.029245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909b6f-8664-4ffc-a899-f8274d89fd24 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:42.030868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909b6f-8664-4ffc-a899-f8274d89fd24 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:42.031391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909b6f-8664-4ffc-a899-f8274d89fd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:42.065767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fc909b6f-8664-4ffc-a899-f8274d89fd24 None None] MFA Rules not processed for user `a72b2314d1d2485fb47f53ed60195110`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:42.069627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fc909b6f-8664-4ffc-a899-f8274d89fd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:42.208217 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1646/3285] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:42.274539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:42.323531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4882532-f85f-4f11-a6ec-1ab9ed75b78d tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:42.330569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:42.331436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:42.332503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:42.332932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:42.346813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:42.352132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:42.368581 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1640/3286] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:42.406609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:42.407452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:42.407975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:42.468587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] MFA Rules not processed for user `2ec3657e512a404db2270dee3d9d1c65`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:42.472913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:42.493132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:42.647367 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1647/3287] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:42.745493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:42.777415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f49a9-8ba3-42a0-b2bf-a3138800c01c service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:42.796136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d12cef-b376-4fce-b7f7-b8dc93eaa65d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:42.798067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12cef-b376-4fce-b7f7-b8dc93eaa65d service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:42.799229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12cef-b376-4fce-b7f7-b8dc93eaa65d service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:42.799660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12cef-b376-4fce-b7f7-b8dc93eaa65d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:42.817202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d12cef-b376-4fce-b7f7-b8dc93eaa65d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:42.821131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d12cef-b376-4fce-b7f7-b8dc93eaa65d service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:42.849624 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1641/3288] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:42.922096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:42.924563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:43.079444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8cc72-8f0c-4027-afc8-e30f5601a41f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:43.092312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffd56c-4280-42bc-95e9-1affaf48b4c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:43.097966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936ee2-79d1-422b-999b-4aa4f969d008 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:43.098339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936ee2-79d1-422b-999b-4aa4f969d008 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:43.098685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936ee2-79d1-422b-999b-4aa4f969d008 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:43.100114 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1642/3289] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:56:43.100724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd56c-4280-42bc-95e9-1affaf48b4c5 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:43.103151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd56c-4280-42bc-95e9-1affaf48b4c5 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:43.103520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd56c-4280-42bc-95e9-1affaf48b4c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:43.105541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-84ffd56c-4280-42bc-95e9-1affaf48b4c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:43.113375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b936ee2-79d1-422b-999b-4aa4f969d008 None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:43.137792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b936ee2-79d1-422b-999b-4aa4f969d008 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:43.147534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffd56c-4280-42bc-95e9-1affaf48b4c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:43.150553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffd56c-4280-42bc-95e9-1affaf48b4c5 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:43.163652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b936ee2-79d1-422b-999b-4aa4f969d008 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:43.170903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:43.173200 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1648/3290] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:43.173631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:43.174187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:43.175038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:43.176942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:43.179315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:43.183136 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1643/3291] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:43 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:56:43.925553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e777ea-8e5e-488b-bb46-d37a15adbe23 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:43.926594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e777ea-8e5e-488b-bb46-d37a15adbe23 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:43.927080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e777ea-8e5e-488b-bb46-d37a15adbe23 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:43.929062 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1649/3292] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:43.938786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:43.961212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:43.988643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234164e8-1d76-4b00-a952-b9d842d7d6db tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:44.002533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:44.003491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:44.003948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:44.004543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:44.007009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:44.015320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:44.035258 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1644/3293] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:43 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:44.400060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24753d3-1a0b-41d0-8b58-747a879a96c0 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:44.400471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24753d3-1a0b-41d0-8b58-747a879a96c0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:44.400829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24753d3-1a0b-41d0-8b58-747a879a96c0 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:44.402480 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1650/3294] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 20:56:44.408077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:44.433779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:44.449004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b73781-024b-4cb5-84b5-6a02ef23e786 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:44.454459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185ffab-65c7-4d4f-b3a2-4810804933ba tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:44.455201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ffab-65c7-4d4f-b3a2-4810804933ba tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:44.455559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ffab-65c7-4d4f-b3a2-4810804933ba tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:44.455907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185ffab-65c7-4d4f-b3a2-4810804933ba tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:44.457244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185ffab-65c7-4d4f-b3a2-4810804933ba tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:44.459452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185ffab-65c7-4d4f-b3a2-4810804933ba tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:44.462193 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1645/3295] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:44 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:44.626766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:44.627052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:44.627538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:44.650412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] MFA Rules not processed for user `a72b2314d1d2485fb47f53ed60195110`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:44.651660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:44.666063 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1651/3296] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:46.066017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512a9ec-be82-4b66-bbd2-d1f68c73f9d8 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:46.066736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512a9ec-be82-4b66-bbd2-d1f68c73f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:46.066839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512a9ec-be82-4b66-bbd2-d1f68c73f9d8 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:46.068464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1646/3297] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:46.083950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:46.102119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:46.125651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e165a4-9258-4b97-bd6b-2abb0c5d7862 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:46.131340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:46.132228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:46.132662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:46.133872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:46.136239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:46.139576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:46.142278 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1652/3298] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:46 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:46.272111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02dbdb-1f53-4db7-b0c5-144ebb60bdc0 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:46.274020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02dbdb-1f53-4db7-b0c5-144ebb60bdc0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:46.274610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02dbdb-1f53-4db7-b0c5-144ebb60bdc0 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:46.277621 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1647/3299] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:46.283575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:46.309192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:46.349375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90710-e7cc-40fb-aae1-f6a2f8001f9a tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:46.358144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:46.361779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:46.362973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:46.363752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:46.368044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:46.375865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:46.379413 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1653/3300] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:46 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:46.433118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc182b-19e1-46a6-bab3-585b0fff9dd7 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:46.433403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc182b-19e1-46a6-bab3-585b0fff9dd7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:46.433677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc182b-19e1-46a6-bab3-585b0fff9dd7 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:46.435036 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1648/3301] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:46.440477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:46.459736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:46.500795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cf513-b580-456d-9bd2-9a289066f98f tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:46.518060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc8fa0-ef48-4db9-98a6-f4104cfe1359 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:46.518724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc8fa0-ef48-4db9-98a6-f4104cfe1359 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:46.520575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc8fa0-ef48-4db9-98a6-f4104cfe1359 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:46.520575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc8fa0-ef48-4db9-98a6-f4104cfe1359 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:46.521937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc8fa0-ef48-4db9-98a6-f4104cfe1359 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:46.528422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc8fa0-ef48-4db9-98a6-f4104cfe1359 tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:46.538667 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1654/3302] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:46 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:46.746067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:46.746739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:46.747357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:46.812948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] MFA Rules not processed for user `a72b2314d1d2485fb47f53ed60195110`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:46.815364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:46.828623 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1649/3303] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:47.905619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf444417-81b4-4efb-b8c5-b6fb02ec1bb5 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:47.906291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf444417-81b4-4efb-b8c5-b6fb02ec1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:47.906291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf444417-81b4-4efb-b8c5-b6fb02ec1bb5 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:47.907625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1655/3304] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 20:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 20:56:47.913595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 20:56:47.927786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:47.958855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382857d-7bab-44e0-acad-92f4744e8903 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:47.976643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f7c53-c4ef-423b-95a1-92b2fefc29ac tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2698bd7906e2463e85c60ce79f33c240', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '032e998177cf4696b70d6c89a5ddd3a8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:47.977969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f7c53-c4ef-423b-95a1-92b2fefc29ac tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:47.978483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f7c53-c4ef-423b-95a1-92b2fefc29ac tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:47.978961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f7c53-c4ef-423b-95a1-92b2fefc29ac tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:47.980463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f7c53-c4ef-423b-95a1-92b2fefc29ac tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorizing `identity:get_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:47.986202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f7c53-c4ef-423b-95a1-92b2fefc29ac tempest-QuotasAdminNegativeTest-99224850 tempest-QuotasAdminNegativeTest-99224850-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:47.989560 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1650/3305] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 20:56:47 2022] GET /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 383 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:48.298427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf444417-81b4-4efb-b8c5-b6fb02ec1bb5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:48.400366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf444417-81b4-4efb-b8c5-b6fb02ec1bb5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:48.406434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:48.407262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:48.407712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:48.408198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] PATH_INFO: `/v3/users/a72b2314d1d2485fb47f53ed60195110` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:48.409594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72b2314d1d2485fb47f53ed60195110)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:48.411729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:48.419783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:48.488701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72b2314d1d2485fb47f53ed60195110'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:48.506857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72b2314d1d2485fb47f53ed60195110'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:48.517431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72b2314d1d2485fb47f53ed60195110'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:48.529160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72b2314d1d2485fb47f53ed60195110'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:48.530118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72b2314d1d2485fb47f53ed60195110'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:48.540705 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1656/3306] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:48 2022] DELETE /identity/v3/users/a72b2314d1d2485fb47f53ed60195110 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:48.544609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:48.544892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:48.545328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:48.577604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:48.578950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:48.590649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:48.783137 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1651/3307] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:48.803226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:48.807101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:48.969237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b20522-2356-4283-8cfe-0b7c870c8799 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:48.995337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17564b9c-630a-418e-86eb-7cd872f5c7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:48.998517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17564b9c-630a-418e-86eb-7cd872f5c7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:48.998517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17564b9c-630a-418e-86eb-7cd872f5c7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:48.998727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17564b9c-630a-418e-86eb-7cd872f5c7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:49.032239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17564b9c-630a-418e-86eb-7cd872f5c7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:49.034979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17564b9c-630a-418e-86eb-7cd872f5c7a1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:49.050151 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1657/3308] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:49.811136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:49.813817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:49.843143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2841d-ad57-40e2-a2c0-c4824506f8ec None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:49.864038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:49.865145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:49.865145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:49.865300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] PATH_INFO: `/v3/users/a681592c2d1043a4afa67838812e7fe9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:49.869413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a681592c2d1043a4afa67838812e7fe9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:49.871728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:50.046112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a681592c2d1043a4afa67838812e7fe9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.071134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a681592c2d1043a4afa67838812e7fe9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.073926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a681592c2d1043a4afa67838812e7fe9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.091116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a681592c2d1043a4afa67838812e7fe9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.092430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a681592c2d1043a4afa67838812e7fe9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.146909 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1652/3309] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:49 2022] DELETE /identity/v3/users/a681592c2d1043a4afa67838812e7fe9 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:50.152047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:50.152973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:50.153472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:50.189725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:50.191294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:50.338964 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1658/3310] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:50.346871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:50.349416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:50.375992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:50.377784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:50.436929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d3843-8c65-4049-994b-dd43d1564441 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:50.458908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:50.459935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:50.460417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:50.461577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] PATH_INFO: `/v3/projects/57854db577054fa3acf6b2ff3704ab15` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:50.462146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94aa40-c321-4873-b2d4-933324366172 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:50.466563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57854db577054fa3acf6b2ff3704ab15)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:50.467088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:50.469041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:50.473447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:50.474173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:50.474460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:50.487877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:50.491480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:50.524749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:50.533222 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1653/3311] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:50.573739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57854db577054fa3acf6b2ff3704ab15', 'request_id': 'req-a9270c8a-d284-42a0-9f50-d7a2a0f88668'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.581956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57854db577054fa3acf6b2ff3704ab15', 'request_id': 'req-a9270c8a-d284-42a0-9f50-d7a2a0f88668'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.591773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57854db577054fa3acf6b2ff3704ab15', 'request_id': 'req-a9270c8a-d284-42a0-9f50-d7a2a0f88668'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.615952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] The token cache is being invalidate because project 57854db577054fa3acf6b2ff3704ab15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:50.617722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.619188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.623463 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1659/3312] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:50 2022] DELETE /identity/v3/projects/57854db577054fa3acf6b2ff3704ab15 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:50.634985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:50.654387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df317d92-7f8a-4feb-bfbd-6802b518bd68 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:50.719834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:50.794055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:50.794968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:50.794968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:50.795243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] PATH_INFO: `/v3/users/2698bd7906e2463e85c60ce79f33c240` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:50.802040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2698bd7906e2463e85c60ce79f33c240)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:50.804077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:50.931055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2698bd7906e2463e85c60ce79f33c240'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.949420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2698bd7906e2463e85c60ce79f33c240'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.950331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2698bd7906e2463e85c60ce79f33c240'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.983619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2698bd7906e2463e85c60ce79f33c240'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:50.984633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2698bd7906e2463e85c60ce79f33c240'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:51.029868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1654/3313] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:50 2022] DELETE /identity/v3/users/2698bd7906e2463e85c60ce79f33c240 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:51.648025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:51.650973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:51.675592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9270c8a-d284-42a0-9f50-d7a2a0f88668 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:51.822640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:51.824361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:51.824643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:51.824921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] PATH_INFO: `/v3/projects/639eb01bc0484295b917ab8d0ceee6ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:51.826702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=639eb01bc0484295b917ab8d0ceee6ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:51.828347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:51.956857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639eb01bc0484295b917ab8d0ceee6ab', 'request_id': 'req-90ca996f-7540-46c5-b9a2-22f625c61cbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:51.971807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639eb01bc0484295b917ab8d0ceee6ab', 'request_id': 'req-90ca996f-7540-46c5-b9a2-22f625c61cbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:51.991612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639eb01bc0484295b917ab8d0ceee6ab', 'request_id': 'req-90ca996f-7540-46c5-b9a2-22f625c61cbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.014075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:52.016868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:52.041204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] The token cache is being invalidate because project 639eb01bc0484295b917ab8d0ceee6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:52.041509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.042489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-90ca996f-7540-46c5-b9a2-22f625c61cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.052619 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1660/3314] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:51 2022] DELETE /identity/v3/projects/639eb01bc0484295b917ab8d0ceee6ab => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:52.056412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3185f-8db1-4198-9e26-a7722e7a2a65 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.057013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3185f-8db1-4198-9e26-a7722e7a2a65 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.057787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3185f-8db1-4198-9e26-a7722e7a2a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.079175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1070f-9f56-4312-b40f-b090f806107b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:52.109650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c3f3185f-8db1-4198-9e26-a7722e7a2a65 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:52.110945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c3f3185f-8db1-4198-9e26-a7722e7a2a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:52.247388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:52.248414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.248895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.249978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] PATH_INFO: `/v3/projects/032e998177cf4696b70d6c89a5ddd3a8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.251548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=032e998177cf4696b70d6c89a5ddd3a8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:52.255604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:52.282921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:52.322887 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1661/3315] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:52.336269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236707c-ffd9-4d27-b9bb-eb32dd24f9d5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.336687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236707c-ffd9-4d27-b9bb-eb32dd24f9d5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.337558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236707c-ffd9-4d27-b9bb-eb32dd24f9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.364997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4236707c-ffd9-4d27-b9bb-eb32dd24f9d5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:52.366403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4236707c-ffd9-4d27-b9bb-eb32dd24f9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:52.387577 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1662/3316] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:52.402719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236707c-ffd9-4d27-b9bb-eb32dd24f9d5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:52.418318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236707c-ffd9-4d27-b9bb-eb32dd24f9d5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:52.433098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032e998177cf4696b70d6c89a5ddd3a8', 'request_id': 'req-24a26032-f20a-48a9-b63f-c739791e4679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.436452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:52.451218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032e998177cf4696b70d6c89a5ddd3a8', 'request_id': 'req-24a26032-f20a-48a9-b63f-c739791e4679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.473889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032e998177cf4696b70d6c89a5ddd3a8', 'request_id': 'req-24a26032-f20a-48a9-b63f-c739791e4679'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.500178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] The token cache is being invalidate because project 032e998177cf4696b70d6c89a5ddd3a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:52.500539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.501322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-24a26032-f20a-48a9-b63f-c739791e4679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.504001 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1655/3317] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:52 2022] DELETE /identity/v3/projects/032e998177cf4696b70d6c89a5ddd3a8 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:56:52.518908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ffc8e-4fb8-4dba-8d6c-5dbbc3ecc803 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.519467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ffc8e-4fb8-4dba-8d6c-5dbbc3ecc803 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.520848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ffc8e-4fb8-4dba-8d6c-5dbbc3ecc803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.567109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-dd1ffc8e-4fb8-4dba-8d6c-5dbbc3ecc803 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:52.569185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dd1ffc8e-4fb8-4dba-8d6c-5dbbc3ecc803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:52.573406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:52.574245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.574524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.574795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.575550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:52.575947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:52.579387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:52.584200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1656/3318] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:52.610243 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1663/3319] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:52.611441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.611737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.612657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:52.613276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.620850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:52.652727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716c67-0e70-46a7-923d-c884a5def6e0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:52.655699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:52.658882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:52.668458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:52.669106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.669421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.669703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.671076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:52.686093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:52.697225 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1657/3320] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:52.698240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:52.700244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:52.717032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cfb5a-6949-4109-a244-8cff4db3299a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:52.726642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:52.727709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:52.728038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:52.728320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] PATH_INFO: `/v3/users/eff0636f25bb459cb92e0c733bb51df8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:52.731658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff0636f25bb459cb92e0c733bb51df8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:52.742998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:52.782733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1664/3321] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:52 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:52.782733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:52.829320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff0636f25bb459cb92e0c733bb51df8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.848156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff0636f25bb459cb92e0c733bb51df8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.849116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff0636f25bb459cb92e0c733bb51df8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.870666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff0636f25bb459cb92e0c733bb51df8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.873603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff0636f25bb459cb92e0c733bb51df8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:52.931286 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1658/3322] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:52 2022] DELETE /identity/v3/users/eff0636f25bb459cb92e0c733bb51df8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:52.931286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:52.937588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffb462-1e38-4141-ab94-c3138cb5968c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:52.938310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:53.131771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:53.160550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.161304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.162619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.164279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.166254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:53.166571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.170206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.181174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637803b-17a5-4a53-8834-2d4994ac0bde admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.188054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.188372 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1665/3323] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:53.189172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.189763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.190109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.190827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.193262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.193380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.195225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:53.210752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb112a56-2ef3-4ead-be52-1ce141836248 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.220866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.221392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.221680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.222404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.224041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.226118 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1659/3324] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:52 2022] POST /identity/v3/users => generated 287 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:53.230092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.232310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.252680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbec82b-7c48-44bf-82a6-926dd961af65 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.260846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.262222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.262222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.262222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.263411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.265508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.281249 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1660/3325] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:53.282439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.300652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddc28-6239-45a9-aa2f-f17cb65213d1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.308836 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1666/3326] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:53 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:53.312983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.313808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.317181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.317489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.317898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.332574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.338105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.361903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1661/3327] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:53.363664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.380729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdeab1-848b-417f-b111-0dc1e2a62b8d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.388993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:INSERT=5,keystone:DELETE=13 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:53.464689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f7fd2-992c-4e47-9155-193702f4e1bd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.477232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.477888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.478177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.478454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.479102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.481040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.503032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.504135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.504736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.505366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] PATH_INFO: `/v3/projects/c2e6f2d4d90d489d88762a6a0a97ed81/users/c0c4e26a7d1d423392c05c3a1b25c8c0/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.513164 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1667/3328] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:53 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:53.519369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.536427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e6f2d4d90d489d88762a6a0a97ed81, user_id=c0c4e26a7d1d423392c05c3a1b25c8c0, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.540691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.541096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce69a9-7fd8-4354-b8c4-e7aee64a1990 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.549073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fa8aa-a3c1-49cc-84d9-4d32c70619ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.550160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa8aa-a3c1-49cc-84d9-4d32c70619ca admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.550621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa8aa-a3c1-49cc-84d9-4d32c70619ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.551097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fa8aa-a3c1-49cc-84d9-4d32c70619ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.551967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fa8aa-a3c1-49cc-84d9-4d32c70619ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.561047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fa8aa-a3c1-49cc-84d9-4d32c70619ca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.569082 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1662/3329] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:53 2022] PUT /identity/v3/projects/c2e6f2d4d90d489d88762a6a0a97ed81/users/c0c4e26a7d1d423392c05c3a1b25c8c0/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:53.572745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.579657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:53.588706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1668/3330] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:53.590417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db4799-a585-4195-a473-548506efb9d9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.591042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db4799-a585-4195-a473-548506efb9d9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.591877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db4799-a585-4195-a473-548506efb9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.646449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-09db4799-a585-4195-a473-548506efb9d9 None None] MFA Rules not processed for user `c0c4e26a7d1d423392c05c3a1b25c8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:53.648451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-09db4799-a585-4195-a473-548506efb9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:53.694476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3bd16-2fea-4b1f-afbb-e2d8edf93950 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.702462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.703201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.703564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.703884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] PATH_INFO: `/v3/projects/f2836de68f8546259ebb0649f7567c4f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.705280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2836de68f8546259ebb0649f7567c4f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.710057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:53.788080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2836de68f8546259ebb0649f7567c4f', 'request_id': 'req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:53.797074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2836de68f8546259ebb0649f7567c4f', 'request_id': 'req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:53.801464 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1669/3331] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:53.802382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db4799-a585-4195-a473-548506efb9d9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:53.828878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db4799-a585-4195-a473-548506efb9d9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:53.832757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2836de68f8546259ebb0649f7567c4f', 'request_id': 'req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:53.853424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] The token cache is being invalidate because project f2836de68f8546259ebb0649f7567c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:53.855446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:53.856368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87cb9a31-4932-4d8e-99a5-04dd7fbf1474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:53.858134 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1663/3332] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:53 2022] DELETE /identity/v3/projects/f2836de68f8546259ebb0649f7567c4f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:53.862703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.863062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.863412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.892103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] MFA Rules not processed for user `c0c4e26a7d1d423392c05c3a1b25c8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:53.893485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:53.912402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:53.923668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:53.978612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:53.979255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:53.979542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:53.979824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:53.980483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:53.982997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:54.006939 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1670/3333] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:54.010830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:54.015566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:54.044471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6982b5b-0822-4084-8856-d2a05b25f381 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:54.053474 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1664/3334] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:54.055939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:54.057900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:54.060958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:54.061861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:54.062548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:54.063022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] PATH_INFO: `/v3/users/5fa4298166524e10a0cf869cdec9df34` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:54.065118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa4298166524e10a0cf869cdec9df34)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:54.068508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:54.098888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa48142-0993-4358-87c0-8f45dd7062e0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:54.110514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c878ea-6d9b-43a8-b038-f3d0b0944a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:54.112739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c878ea-6d9b-43a8-b038-f3d0b0944a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:54.113424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c878ea-6d9b-43a8-b038-f3d0b0944a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:54.114237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c878ea-6d9b-43a8-b038-f3d0b0944a6a admin admin] PATH_INFO: `/v3/projects/bb3b6560487b4adc8e7e655a6fbf1513/users/c7a7eb073a03410c8183ae02fab0e74e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:54.165620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c878ea-6d9b-43a8-b038-f3d0b0944a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3b6560487b4adc8e7e655a6fbf1513, user_id=c7a7eb073a03410c8183ae02fab0e74e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:54.172909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c878ea-6d9b-43a8-b038-f3d0b0944a6a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:54.188558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa4298166524e10a0cf869cdec9df34'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.210366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1665/3335] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:54 2022] PUT /identity/v3/projects/bb3b6560487b4adc8e7e655a6fbf1513/users/c7a7eb073a03410c8183ae02fab0e74e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:54.221568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:54.222266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:54.223223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:54.231640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa4298166524e10a0cf869cdec9df34'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.240648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa4298166524e10a0cf869cdec9df34'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.261801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa4298166524e10a0cf869cdec9df34'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.262660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa4298166524e10a0cf869cdec9df34'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.282639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1671/3336] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:54 2022] DELETE /identity/v3/users/5fa4298166524e10a0cf869cdec9df34 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:54.287086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] MFA Rules not processed for user `c7a7eb073a03410c8183ae02fab0e74e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:54.288344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:54.290122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:54.292936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:54.449965 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1666/3337] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:54.453318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76832-c484-4eec-96b7-b1664d6b909e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:54.460450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:54.461479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:54.462054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:54.462513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:54.465873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:54.555244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:54.589489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:54.594941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:54.634064 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1672/3338] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:54.634487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:54.635935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:54.733283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:54.734945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:54.752545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115a818-7303-4da4-be94-8dbeaf2600df service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:54.762308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4fb30-cccf-4a94-868d-c4f3f81d0b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:54.762955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4fb30-cccf-4a94-868d-c4f3f81d0b97 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:54.763206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4fb30-cccf-4a94-868d-c4f3f81d0b97 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:54.763482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4fb30-cccf-4a94-868d-c4f3f81d0b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:54.772012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4fb30-cccf-4a94-868d-c4f3f81d0b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:54.774394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4fb30-cccf-4a94-868d-c4f3f81d0b97 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:54.798671 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1673/3339] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:56:54.800026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d95250-10d0-4c48-a290-9a248047d871 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:54.807596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:54.808259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:54.808548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:54.809033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] PATH_INFO: `/v3/projects/8d8536a456e94f5b842c0f9be5caf0ff` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:54.819242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d8536a456e94f5b842c0f9be5caf0ff)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:54.828026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:54.892143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:54.916528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d8536a456e94f5b842c0f9be5caf0ff', 'request_id': 'req-712a2fef-4694-40ff-b271-32d61dd33632'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.927796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d8536a456e94f5b842c0f9be5caf0ff', 'request_id': 'req-712a2fef-4694-40ff-b271-32d61dd33632'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.965031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d8536a456e94f5b842c0f9be5caf0ff', 'request_id': 'req-712a2fef-4694-40ff-b271-32d61dd33632'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.986470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] The token cache is being invalidate because project 8d8536a456e94f5b842c0f9be5caf0ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:54.986844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.988898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-712a2fef-4694-40ff-b271-32d61dd33632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:54.991302 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1667/3340] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:54 2022] DELETE /identity/v3/projects/8d8536a456e94f5b842c0f9be5caf0ff => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:54.997404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27f422-4038-41f3-847f-ab7874756948 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:54.997932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27f422-4038-41f3-847f-ab7874756948 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:54.998008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27f422-4038-41f3-847f-ab7874756948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.026147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ee27f422-4038-41f3-847f-ab7874756948 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:55.028058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ee27f422-4038-41f3-847f-ab7874756948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:55.166443 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1674/3341] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:55.168693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c9174-f434-4502-be24-124ea5d1099a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.168983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c9174-f434-4502-be24-124ea5d1099a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.169282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c9174-f434-4502-be24-124ea5d1099a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.195443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7f6c9174-f434-4502-be24-124ea5d1099a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:55.197767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f6c9174-f434-4502-be24-124ea5d1099a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:55.213217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1668/3342] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:55.219209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27f422-4038-41f3-847f-ab7874756948 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:55.239741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27f422-4038-41f3-847f-ab7874756948 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:55.247572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:55.248866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.249374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.250748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.251942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:55.252467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:55.255081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:55.280626 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1675/3343] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:55.281603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c9174-f434-4502-be24-124ea5d1099a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:55.296536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c9174-f434-4502-be24-124ea5d1099a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:55.302908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:55.303666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.303959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.304238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.305441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:55.307777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:55.349691 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1669/3344] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:55 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:55.355865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:55.369524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207edc75-8d8c-45c5-b321-5056c4283856 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:55.478281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:55.479453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.479945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.480480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.481402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:55.486212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:55.519816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1676/3345] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:55 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:55.525133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:55.544898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720926-2b69-4dee-8cb7-95a31c950001 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:55.564111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:55.564884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.565183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.565774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.568306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:55.573143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:55.613488 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1670/3346] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:55.613793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:55.627753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c15659-f2c4-43b6-a23a-3f6c1e194d28 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:55.634711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:55.645629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.645943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.646224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.646903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:55.649118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:55.670719 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1677/3347] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:55.672058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:55.683670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50e7e6-7f07-443c-9323-804f0717c240 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:55.689255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c197c05-23db-44b9-91ff-7844fdd7db7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:55.689942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c197c05-23db-44b9-91ff-7844fdd7db7b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.690283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c197c05-23db-44b9-91ff-7844fdd7db7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.690700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c197c05-23db-44b9-91ff-7844fdd7db7b admin admin] PATH_INFO: `/v3/projects/9648416fc26049d6bbc02ac1aabe9d2d/users/0a3e9b9e9386455cbab59056e9e32dd9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.722411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c197c05-23db-44b9-91ff-7844fdd7db7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9648416fc26049d6bbc02ac1aabe9d2d, user_id=0a3e9b9e9386455cbab59056e9e32dd9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:55.726254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c197c05-23db-44b9-91ff-7844fdd7db7b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:55.758976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:55.760085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:55.782356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018df36-127f-4ba3-a235-9ea4dc49e3b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:55.810899 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1671/3348] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:55 2022] PUT /identity/v3/projects/9648416fc26049d6bbc02ac1aabe9d2d/users/0a3e9b9e9386455cbab59056e9e32dd9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:55.819366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d2a2-1952-4919-af2e-c8518fcae398 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.819652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d2a2-1952-4919-af2e-c8518fcae398 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.820664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494d2a2-1952-4919-af2e-c8518fcae398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.825597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:55.870518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:55.871458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:55.872371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:55.872875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] PATH_INFO: `/v3/users/c7a7eb073a03410c8183ae02fab0e74e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:55.883816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a7eb073a03410c8183ae02fab0e74e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:55.887639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:55.893165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7494d2a2-1952-4919-af2e-c8518fcae398 None None] MFA Rules not processed for user `0a3e9b9e9386455cbab59056e9e32dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:55.895452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7494d2a2-1952-4919-af2e-c8518fcae398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:55.983476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a7eb073a03410c8183ae02fab0e74e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:56.004286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a7eb073a03410c8183ae02fab0e74e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:56.015490 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1672/3349] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:56.016922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:56.018923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:56.019210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:56.022660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a7eb073a03410c8183ae02fab0e74e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:56.032640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a7eb073a03410c8183ae02fab0e74e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:56.033327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2595ef83-941d-4ae8-a625-b6cf8b8fea4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a7eb073a03410c8183ae02fab0e74e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:56.041631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1678/3350] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:55 2022] DELETE /identity/v3/users/c7a7eb073a03410c8183ae02fab0e74e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:56.045996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6095e-81eb-421f-a958-09517883d877 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:56.046295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6095e-81eb-421f-a958-09517883d877 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:56.046413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:56.046589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6095e-81eb-421f-a958-09517883d877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:56.047560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:56.099463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-50a6095e-81eb-421f-a958-09517883d877 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:56.100770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-50a6095e-81eb-421f-a958-09517883d877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:56.103760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:56.123887 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1673/3351] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:56:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:56.124615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:56.126238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:56.220437 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1679/3352] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:56.221485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6095e-81eb-421f-a958-09517883d877 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:56.223686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-50a6095e-81eb-421f-a958-09517883d877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:56.230807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c21c56-793f-4ed6-9403-a2f1c7268c29 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:56.238462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:56.239092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:56.239408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:56.239854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:56.243215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:56.288057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:56.308529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6095e-81eb-421f-a958-09517883d877 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:56.310335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:56.311980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:56.313840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:56.314459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:56.314740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:56.315015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:56.321375 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1674/3353] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:56.323499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:56.325196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:56.326754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:56.328091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:56.347992 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1680/3354] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:56:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:56:56.400838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79950a7-573f-4edb-9582-dcc58c07f7ea service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:56.404602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b9764-1257-4c04-8d81-1c21179f9bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:56.405170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b9764-1257-4c04-8d81-1c21179f9bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:56.405453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b9764-1257-4c04-8d81-1c21179f9bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:56.405728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b9764-1257-4c04-8d81-1c21179f9bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:56.416398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b9764-1257-4c04-8d81-1c21179f9bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:56.417894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b9764-1257-4c04-8d81-1c21179f9bf2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:56.429158 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1675/3355] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:57.329615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:57.331761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:57.366251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91339b82-68df-440a-bfad-71fb3ef60bdb service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:57.371659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:57.372300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:57.372579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:57.372854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] PATH_INFO: `/v3/projects/bb3b6560487b4adc8e7e655a6fbf1513` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:57.374053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3b6560487b4adc8e7e655a6fbf1513)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:57.375976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:57.436213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3b6560487b4adc8e7e655a6fbf1513', 'request_id': 'req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:57.444454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3b6560487b4adc8e7e655a6fbf1513', 'request_id': 'req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:57.456374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3b6560487b4adc8e7e655a6fbf1513', 'request_id': 'req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:57.475686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] The token cache is being invalidate because project bb3b6560487b4adc8e7e655a6fbf1513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:56:57.476483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:57.477861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5f0a3117-b110-420a-87ca-3c1e08f6cb2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:56:57.488489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ff598-b29b-4ffc-b49e-a29670ed765a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:57.489480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ff598-b29b-4ffc-b49e-a29670ed765a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:57.490031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ff598-b29b-4ffc-b49e-a29670ed765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:57.491422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1681/3356] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:56:57 2022] DELETE /identity/v3/projects/bb3b6560487b4adc8e7e655a6fbf1513 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:57.504567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da8777-ca09-4120-bae5-3a69a892daa8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:57.504876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da8777-ca09-4120-bae5-3a69a892daa8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:57.505187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da8777-ca09-4120-bae5-3a69a892daa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:57.534417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e25ff598-b29b-4ffc-b49e-a29670ed765a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:57.535870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e25ff598-b29b-4ffc-b49e-a29670ed765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:57.536535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-18da8777-ca09-4120-bae5-3a69a892daa8 None None] MFA Rules not processed for user `687eef270165450a97b4d260b2724b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:57.538641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-18da8777-ca09-4120-bae5-3a69a892daa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:57.572955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:57.629401 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1676/3357] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:57.631671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e1e0-e352-4ca9-8abd-eade23530814 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:57.631959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e1e0-e352-4ca9-8abd-eade23530814 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:57.632240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e1e0-e352-4ca9-8abd-eade23530814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:57.638756 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1682/3358] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:56:57.671239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4ff7e1e0-e352-4ca9-8abd-eade23530814 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:57.672805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4ff7e1e0-e352-4ca9-8abd-eade23530814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:57.682730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1677/3359] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:57.695565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da8777-ca09-4120-bae5-3a69a892daa8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:57.710730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da8777-ca09-4120-bae5-3a69a892daa8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:57.735688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:57.736322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:57.736610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:57.737049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:57.737721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:56:57.738064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:57.742711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:57.767360 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1683/3360] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:56:57.771468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e1e0-e352-4ca9-8abd-eade23530814 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:57.782606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e1e0-e352-4ca9-8abd-eade23530814 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:57.789388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:57.790186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:57.798373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:57.798628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:57.799419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:57.801480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:57.858210 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1678/3361] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:56:57 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:57.858599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:57.870761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1695d-fba6-4e74-a45e-0ea9ed728abe admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:57.941117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:56:57.950922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:57.952623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:57.953404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:57.953925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:57.957851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:57.959093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:57.997964 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1684/3362] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:56:57 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:56:57.999266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:58.010504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc392e-2fb9-432d-8442-3aca4c1daa7c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:58.015664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:58.016273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:58.016557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:58.016841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:58.017542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:58.019182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:58.033591 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1679/3363] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:58.034834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:58.046900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a57eb7-6e10-41f3-b6a2-2428919643a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:58.053020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ab5af-fa63-4e6e-bbbb-3d26ca48b392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:58.053631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ab5af-fa63-4e6e-bbbb-3d26ca48b392 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:58.054465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ab5af-fa63-4e6e-bbbb-3d26ca48b392 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:58.054465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ab5af-fa63-4e6e-bbbb-3d26ca48b392 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:58.054957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ab5af-fa63-4e6e-bbbb-3d26ca48b392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:58.057251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ab5af-fa63-4e6e-bbbb-3d26ca48b392 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:58.072998 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1685/3364] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:56:58.073488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:58.083592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e6cf-a4d9-405f-bafa-0f6545c2d903 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:58.089216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353efef-f1ad-42fe-9a53-a232da9bf30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:58.089976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353efef-f1ad-42fe-9a53-a232da9bf30c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:58.090264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353efef-f1ad-42fe-9a53-a232da9bf30c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:58.090541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353efef-f1ad-42fe-9a53-a232da9bf30c admin admin] PATH_INFO: `/v3/projects/f57a1c44d4684b2fa210973f062adbce/users/bec33de4067d4b1bb6e4ad866ddf2d01/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:58.107750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353efef-f1ad-42fe-9a53-a232da9bf30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57a1c44d4684b2fa210973f062adbce, user_id=bec33de4067d4b1bb6e4ad866ddf2d01, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:58.110531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353efef-f1ad-42fe-9a53-a232da9bf30c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:58.128876 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1680/3365] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:56:58 2022] PUT /identity/v3/projects/f57a1c44d4684b2fa210973f062adbce/users/bec33de4067d4b1bb6e4ad866ddf2d01/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:56:58.140876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:58.141866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:58.142286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:58.173282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] MFA Rules not processed for user `bec33de4067d4b1bb6e4ad866ddf2d01`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:58.174754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:58.282360 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1686/3366] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:58.293187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:58.293473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:58.293748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:58.315031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] MFA Rules not processed for user `bec33de4067d4b1bb6e4ad866ddf2d01`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:56:58.316247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:58.328361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1681/3367] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:56:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:56:58.453005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:58.454503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:56:58.559051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000b92c-55cb-41e1-8a65-6d30b57d9911 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:56:58.567637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee62a8-28f9-4310-8bd3-ff511d1c3992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:56:58.568407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee62a8-28f9-4310-8bd3-ff511d1c3992 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:56:58.568693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee62a8-28f9-4310-8bd3-ff511d1c3992 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:56:58.569105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee62a8-28f9-4310-8bd3-ff511d1c3992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:56:58.580602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee62a8-28f9-4310-8bd3-ff511d1c3992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:56:58.582752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee62a8-28f9-4310-8bd3-ff511d1c3992 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:56:58.595798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1687/3368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:56:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:56:59.932752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:56:59.934464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:00.050393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d7938-16f8-4a6e-b1f3-a5fb454afbbc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:00.059756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:00.060535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:00.060913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:00.061287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] PATH_INFO: `/v3/users/0a3e9b9e9386455cbab59056e9e32dd9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:00.062607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3e9b9e9386455cbab59056e9e32dd9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:00.064755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:00.130693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3e9b9e9386455cbab59056e9e32dd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:00.142374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3e9b9e9386455cbab59056e9e32dd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:00.143992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3e9b9e9386455cbab59056e9e32dd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:00.160429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3e9b9e9386455cbab59056e9e32dd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:00.161864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3e9b9e9386455cbab59056e9e32dd9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:00.162514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:00.192957 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1682/3369] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:56:59 2022] DELETE /identity/v3/users/0a3e9b9e9386455cbab59056e9e32dd9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:00.197747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:00.198727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:00.199011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:00.221286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:00.223101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:00.344731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:00.392027 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1688/3370] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:00.404841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:00.406404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:00.558481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf5783-9c63-4e16-9695-9cee47dc36ac admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:00.577203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f074fec-95c0-4191-8a5a-1ad5b5d3c428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:00.577759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f074fec-95c0-4191-8a5a-1ad5b5d3c428 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:00.578062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f074fec-95c0-4191-8a5a-1ad5b5d3c428 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:00.578334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f074fec-95c0-4191-8a5a-1ad5b5d3c428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:00.602402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:00.612695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:00.622698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f074fec-95c0-4191-8a5a-1ad5b5d3c428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:00.624287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f074fec-95c0-4191-8a5a-1ad5b5d3c428 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:00.645146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1683/3371] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:00.731424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e8035-d191-4c7e-b91b-f659f438ba98 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:00.738252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:00.739455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:00.739938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:00.740447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:00.742713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:00.813285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:00.815093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:00.838475 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1689/3372] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:00.878980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7039d2-b42f-4854-ac2b-cd1451837360 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:00.879375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7039d2-b42f-4854-ac2b-cd1451837360 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:00.879794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7039d2-b42f-4854-ac2b-cd1451837360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:00.901170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ac7039d2-b42f-4854-ac2b-cd1451837360 None None] MFA Rules not processed for user `87df880447e941f0b336470ac8aba19f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:00.902516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ac7039d2-b42f-4854-ac2b-cd1451837360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:00.998228 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1684/3373] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:01.798781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:01.800803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:01.824927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611e13-0bf0-423e-827c-5eced76ae249 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:01.839983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:01.842846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:01.843535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:01.844349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] PATH_INFO: `/v3/projects/9648416fc26049d6bbc02ac1aabe9d2d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:01.851417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9648416fc26049d6bbc02ac1aabe9d2d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:01.876540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:02.012410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9648416fc26049d6bbc02ac1aabe9d2d', 'request_id': 'req-173853e0-9f9d-4b0f-b48f-0a187a7d9911'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:02.020131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9648416fc26049d6bbc02ac1aabe9d2d', 'request_id': 'req-173853e0-9f9d-4b0f-b48f-0a187a7d9911'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:02.028957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9648416fc26049d6bbc02ac1aabe9d2d', 'request_id': 'req-173853e0-9f9d-4b0f-b48f-0a187a7d9911'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:02.060817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] The token cache is being invalidate because project 9648416fc26049d6bbc02ac1aabe9d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:57:02.061462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:02.064360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-173853e0-9f9d-4b0f-b48f-0a187a7d9911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:02.068882 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1690/3374] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:57:01 2022] DELETE /identity/v3/projects/9648416fc26049d6bbc02ac1aabe9d2d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:02.074755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f3b0-8eae-4bab-b16a-8ffcdc4dfd51 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.075871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f3b0-8eae-4bab-b16a-8ffcdc4dfd51 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.076293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f3b0-8eae-4bab-b16a-8ffcdc4dfd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.114498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9641f3b0-8eae-4bab-b16a-8ffcdc4dfd51 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:02.115728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9641f3b0-8eae-4bab-b16a-8ffcdc4dfd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:02.249573 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1685/3375] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:02.258695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a99981-688b-46bd-b9ac-798322f5118c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.259291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a99981-688b-46bd-b9ac-798322f5118c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.260019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a99981-688b-46bd-b9ac-798322f5118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.281910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:02.306299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c6a99981-688b-46bd-b9ac-798322f5118c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:02.308025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6a99981-688b-46bd-b9ac-798322f5118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:02.320966 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1691/3376] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:57:02.325107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641f3b0-8eae-4bab-b16a-8ffcdc4dfd51 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:02.336315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641f3b0-8eae-4bab-b16a-8ffcdc4dfd51 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:02.342304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:02.343284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.343781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.344355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.345356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:57:02.345910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:02.348010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:02.373617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a99981-688b-46bd-b9ac-798322f5118c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:02.379271 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1686/3377] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:57:02.387774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a99981-688b-46bd-b9ac-798322f5118c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:02.397263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:02.399576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.399576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.399576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.400688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:02.405044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:02.481227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1692/3378] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:57:02 2022] POST /identity/v3/projects => generated 413 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:02.484377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:02.511193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e999c-c7ec-4a03-8150-07ffe01a94c1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:02.531283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:02.605190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:02.605190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.605479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.605701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.607252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:02.610649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:02.658139 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1687/3379] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:57:02 2022] POST /identity/v3/users => generated 307 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:02.659515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:02.674776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b327ec-84d6-46af-a58c-7d7feb6e47aa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:02.687750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:02.688384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.688667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.688947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.689605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:02.691625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:02.709287 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1693/3380] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:02.712623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:02.729199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98343fa4-4690-40fe-b3a0-5e688b5f15eb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:02.735219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cf2ee-940f-4f43-9b38-c2251bae0486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:02.735737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cf2ee-940f-4f43-9b38-c2251bae0486 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.736019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cf2ee-940f-4f43-9b38-c2251bae0486 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.737024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cf2ee-940f-4f43-9b38-c2251bae0486 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.737176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610cf2ee-940f-4f43-9b38-c2251bae0486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:02.739670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610cf2ee-940f-4f43-9b38-c2251bae0486 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:02.759825 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1688/3381] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:02.761102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:02.773864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf355089-697b-4bfb-8ba6-d39c95c2e551 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:02.779237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:02.780155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.780460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.780916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] PATH_INFO: `/v3/projects/f1f9b47aca3845d6a3fc34662d34ccf1/users/609617a03387478b9b72cbe52d25dbf0/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.849297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f9b47aca3845d6a3fc34662d34ccf1, user_id=609617a03387478b9b72cbe52d25dbf0, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:02.857216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:02.886806 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1694/3382] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:57:02 2022] PUT /identity/v3/projects/f1f9b47aca3845d6a3fc34662d34ccf1/users/609617a03387478b9b72cbe52d25dbf0/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:02.888201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:02.913993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:02.913993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:02.946064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:02.946981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:03.129351 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1689/3383] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:03.186620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:03.188619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:03.317480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140ce8e-e719-4d26-a66e-af01b09b49ee admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:03.322160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:03.323169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:03.323518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:03.323847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:03.361908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:03.363341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:03.378299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1695/3384] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:57:10.152192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:10.157185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:10.182671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdd0bb-f951-4f4a-b427-eb4cba33982a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:10.190003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:10.190883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:10.191319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:10.191747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] PATH_INFO: `/v3/users/2ec3657e512a404db2270dee3d9d1c65` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:10.193238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec3657e512a404db2270dee3d9d1c65)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:10.196516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:10.271578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec3657e512a404db2270dee3d9d1c65'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:10.283137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec3657e512a404db2270dee3d9d1c65'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:10.284133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec3657e512a404db2270dee3d9d1c65'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:10.301132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec3657e512a404db2270dee3d9d1c65'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:10.301777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f7d0d47-e15f-47ae-99c5-f8438f3a35cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec3657e512a404db2270dee3d9d1c65'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:10.339062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1690/3385] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:57:10 2022] DELETE /identity/v3/users/2ec3657e512a404db2270dee3d9d1c65 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:10.669789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:10.688955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cdfbf-d056-41bf-aed0-511bfbf733cf service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:10.977041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:10.977987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:10.978420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:10.978845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] PATH_INFO: `/v3/projects/02dadff0c0cd4e94ab17400e30e43ba6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:10.986790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02dadff0c0cd4e94ab17400e30e43ba6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:10.989208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:11.157398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:11.168971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02dadff0c0cd4e94ab17400e30e43ba6', 'request_id': 'req-031f067b-3f10-4bdb-a114-b89fb030b9f4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:11.191507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02dadff0c0cd4e94ab17400e30e43ba6', 'request_id': 'req-031f067b-3f10-4bdb-a114-b89fb030b9f4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:11.202565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02dadff0c0cd4e94ab17400e30e43ba6', 'request_id': 'req-031f067b-3f10-4bdb-a114-b89fb030b9f4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:11.238220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] The token cache is being invalidate because project 02dadff0c0cd4e94ab17400e30e43ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:57:11.238220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:11.250665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-031f067b-3f10-4bdb-a114-b89fb030b9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:11.253106 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1696/3386] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:57:10 2022] DELETE /identity/v3/projects/02dadff0c0cd4e94ab17400e30e43ba6 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:11.260511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ded3c-29c9-41dc-86f4-60c070bc98b3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:11.261033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ded3c-29c9-41dc-86f4-60c070bc98b3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:11.261108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ded3c-29c9-41dc-86f4-60c070bc98b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:11.294230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:11.310631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c13ded3c-29c9-41dc-86f4-60c070bc98b3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:11.312405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c13ded3c-29c9-41dc-86f4-60c070bc98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:11.476870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1691/3387] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:11.488193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1e4d3-8992-46af-af61-be8befd9f308 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:11.488668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1e4d3-8992-46af-af61-be8befd9f308 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:11.488952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1e4d3-8992-46af-af61-be8befd9f308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:11.523940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5cf1e4d3-8992-46af-af61-be8befd9f308 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:11.528096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5cf1e4d3-8992-46af-af61-be8befd9f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:11.555177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1697/3388] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:11.559019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ded3c-29c9-41dc-86f4-60c070bc98b3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:11.574464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ded3c-29c9-41dc-86f4-60c070bc98b3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:11.594291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:11.595137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:11.595659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:11.596027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:11.596933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:57:11.597872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:11.599647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:11.622369 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1692/3389] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 20:57:11.626758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1e4d3-8992-46af-af61-be8befd9f308 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:11.639703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1e4d3-8992-46af-af61-be8befd9f308 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:11.650586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:11.651230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:11.653668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:11.654027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:11.654711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:11.656711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:11.730031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1698/3390] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:57:11 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:11.734686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:11.751162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f903b1d-4118-4fcb-83b4-24b1fd441ed3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:11.855298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:11.855990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:11.856373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:11.856750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:11.857495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:11.859630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:11.907092 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1693/3391] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:57:11 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:11.908089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:11.924604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef23525-f71f-4cb0-846f-e51b4627f8c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:11.933329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:11.934013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:11.934300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:11.934583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:11.935279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:11.937161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:11.952023 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1699/3392] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:11.956291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:11.998211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bafce-49aa-4381-80ac-9de886790e53 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:12.018678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b7f88-2961-4aae-8684-6705e784eda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:12.019502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b7f88-2961-4aae-8684-6705e784eda2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:12.019938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b7f88-2961-4aae-8684-6705e784eda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:12.020357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b7f88-2961-4aae-8684-6705e784eda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:12.021176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b7f88-2961-4aae-8684-6705e784eda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:12.023039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b7f88-2961-4aae-8684-6705e784eda2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:12.048711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:12.056109 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1694/3393] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:12.064044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e11ea-388b-4314-9267-b99d3dd35e70 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:12.090624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:12.090624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:12.090624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:12.090624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] PATH_INFO: `/v3/projects/38b49720bd644bed9a3504ff26f01768/users/91d41f09b29b4f20816318805722b2e4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:12.119378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b49720bd644bed9a3504ff26f01768, user_id=91d41f09b29b4f20816318805722b2e4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:12.122315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:12.154855 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1700/3394] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:57:12 2022] PUT /identity/v3/projects/38b49720bd644bed9a3504ff26f01768/users/91d41f09b29b4f20816318805722b2e4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:57:12.161319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0c40-e2e0-4129-b9b4-8bbade9fb2cd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:12.161868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0c40-e2e0-4129-b9b4-8bbade9fb2cd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:12.162899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0c40-e2e0-4129-b9b4-8bbade9fb2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:12.206074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-50ff0c40-e2e0-4129-b9b4-8bbade9fb2cd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:12.207572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-50ff0c40-e2e0-4129-b9b4-8bbade9fb2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:12.325648 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1695/3395] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:12.351617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:12.351617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:12.440580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10cad8-a1a0-4752-ad56-ed2434a35cd9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:12.444504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df987b-4d48-4da9-a082-0ef744e707af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:12.445924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df987b-4d48-4da9-a082-0ef744e707af service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:12.446195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df987b-4d48-4da9-a082-0ef744e707af service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:12.446478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df987b-4d48-4da9-a082-0ef744e707af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:12.470741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df987b-4d48-4da9-a082-0ef744e707af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:12.472476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df987b-4d48-4da9-a082-0ef744e707af service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:12.485665 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1701/3396] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:12.566725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:12.567534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:12.568050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:12.618340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:12.629768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] MFA Rules not processed for user `609617a03387478b9b72cbe52d25dbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:12.631413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:12.735177 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1696/3397] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:12.737228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a4f0f-028b-4257-ac85-66556a7cfa8b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:12.737683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a4f0f-028b-4257-ac85-66556a7cfa8b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:12.737950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a4f0f-028b-4257-ac85-66556a7cfa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:12.760488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-599a4f0f-028b-4257-ac85-66556a7cfa8b None None] MFA Rules not processed for user `609617a03387478b9b72cbe52d25dbf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:12.762486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-599a4f0f-028b-4257-ac85-66556a7cfa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:12.771507 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1702/3398] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:12.800493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:12.802926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:12.893153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d96a50-c326-4cdc-8d2b-dce6b705f35e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:12.900243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94753eaa-be1c-4f2a-95c9-4437894d503c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:12.900911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94753eaa-be1c-4f2a-95c9-4437894d503c service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:12.901198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94753eaa-be1c-4f2a-95c9-4437894d503c service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:12.901626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94753eaa-be1c-4f2a-95c9-4437894d503c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:12.910380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94753eaa-be1c-4f2a-95c9-4437894d503c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:12.912049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94753eaa-be1c-4f2a-95c9-4437894d503c service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:12.924921 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1697/3399] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:12 2022] GET /identity/v3/auth/tokens => generated 3290 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:21.097758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:21.098356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:21.098356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:21.137640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] MFA Rules not processed for user `4c3e0565f75c4f558a8936f547ae9d76`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:21.138818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:21.274732 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1703/3400] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:21.748630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a12e3-058f-4f09-aedd-998211fbfd1d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:21.749242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a12e3-058f-4f09-aedd-998211fbfd1d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:21.749338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a12e3-058f-4f09-aedd-998211fbfd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:21.827509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2a5a12e3-058f-4f09-aedd-998211fbfd1d None None] MFA Rules not processed for user `91d41f09b29b4f20816318805722b2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:21.829321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a5a12e3-058f-4f09-aedd-998211fbfd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:22.020313 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1698/3401] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:22.646747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:22.650277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:22.675868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d43f6-d59e-460a-9ab0-e60bb8d84a7f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:22.685299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:22.686255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:22.686741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:22.687242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:22.711514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:22.711514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:22.711514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:22.714039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1704/3402] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:26.706517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513e19b-e815-4c90-8b15-dfe26e95e160 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:26.707424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513e19b-e815-4c90-8b15-dfe26e95e160 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:26.707923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513e19b-e815-4c90-8b15-dfe26e95e160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:26.746683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c513e19b-e815-4c90-8b15-dfe26e95e160 None None] MFA Rules not processed for user `91d41f09b29b4f20816318805722b2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:26.747923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c513e19b-e815-4c90-8b15-dfe26e95e160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:26.755832 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1699/3403] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:36.724810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:36.943561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:36.945163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:36.959131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c7e8-840f-4675-b687-936964942983 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:36.964565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:36.965200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:36.965482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:36.965758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] PATH_INFO: `/v3/users/87df880447e941f0b336470ac8aba19f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:36.967045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87df880447e941f0b336470ac8aba19f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:36.968972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:37.026744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87df880447e941f0b336470ac8aba19f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.039826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87df880447e941f0b336470ac8aba19f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.046702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87df880447e941f0b336470ac8aba19f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.055706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87df880447e941f0b336470ac8aba19f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.056399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87df880447e941f0b336470ac8aba19f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.062801 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1705/3404] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:57:36 2022] DELETE /identity/v3/users/87df880447e941f0b336470ac8aba19f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:37.067190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:37.067482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:37.067761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:37.088310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:37.089512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.160270 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1700/3405] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:37.170906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:37.172544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.247095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e3980-74ce-48fd-a1c9-d2a91d44a140 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:37.251484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f1b02-647b-4c79-af19-65dae815749a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:37.252227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f1b02-647b-4c79-af19-65dae815749a service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:37.252583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f1b02-647b-4c79-af19-65dae815749a service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:37.253197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f1b02-647b-4c79-af19-65dae815749a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:37.264427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f1b02-647b-4c79-af19-65dae815749a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:37.266012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f1b02-647b-4c79-af19-65dae815749a service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:37.277538 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1706/3406] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:37.332453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:37.334074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.347485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b9f3-6069-4412-bf27-c6ab48e1388e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:37.354150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:37.354800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:37.355075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:37.355352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] PATH_INFO: `/v3/users/c0c4e26a7d1d423392c05c3a1b25c8c0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:37.356514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c4e26a7d1d423392c05c3a1b25c8c0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:37.358499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:37.420338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c4e26a7d1d423392c05c3a1b25c8c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.429534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c4e26a7d1d423392c05c3a1b25c8c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.430507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c4e26a7d1d423392c05c3a1b25c8c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.440882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c4e26a7d1d423392c05c3a1b25c8c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.441672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c4e26a7d1d423392c05c3a1b25c8c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.462393 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1701/3407] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:57:37 2022] DELETE /identity/v3/users/c0c4e26a7d1d423392c05c3a1b25c8c0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:37.467131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:37.467508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:37.467941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:37.493960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:37.494500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:37.495567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.496395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.600269 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1707/3408] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:37.602162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:37.603658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.607479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfad473-13c1-41c0-91cc-8cade2274706 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:37.612697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:37.613339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:37.613620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:37.614347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] PATH_INFO: `/v3/projects/a9327061a3ba468f83e978315b6ed753` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:37.615292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9327061a3ba468f83e978315b6ed753)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:37.617946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:37.625892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:37.641053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:37.708019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9327061a3ba468f83e978315b6ed753', 'request_id': 'req-705c495a-2b2b-408e-8eb3-f6627694fc39'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.712409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdf095-b911-451b-bc97-24edbe7e359a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:37.716025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9327061a3ba468f83e978315b6ed753', 'request_id': 'req-705c495a-2b2b-408e-8eb3-f6627694fc39'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.729204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9327061a3ba468f83e978315b6ed753', 'request_id': 'req-705c495a-2b2b-408e-8eb3-f6627694fc39'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.744318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] The token cache is being invalidate because project a9327061a3ba468f83e978315b6ed753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:57:37.744641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.745344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:37.747477 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1702/3409] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:57:37 2022] DELETE /identity/v3/projects/a9327061a3ba468f83e978315b6ed753 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:57:37.751572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:37.753559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.818816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:37.824024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:37.824367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:37.824655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:37.827004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:37.880111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c495a-2b2b-408e-8eb3-f6627694fc39 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:37.894629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:37.895285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:37.895571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:37.895845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] PATH_INFO: `/v3/users/860c77c7bfb14a69a9a1ea46446621c7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:37.897400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=860c77c7bfb14a69a9a1ea46446621c7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:37.902399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:37.954256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:37.956616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:37.969199 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1708/3410] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:38.043147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860c77c7bfb14a69a9a1ea46446621c7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.062670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860c77c7bfb14a69a9a1ea46446621c7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.063511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860c77c7bfb14a69a9a1ea46446621c7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.074380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860c77c7bfb14a69a9a1ea46446621c7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.075030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860c77c7bfb14a69a9a1ea46446621c7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.117101 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1703/3411] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:57:37 2022] DELETE /identity/v3/users/860c77c7bfb14a69a9a1ea46446621c7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:57:38.671164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:38.697760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e408c4f-f596-4ce7-b3d8-c88669285e7f service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:38.698248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:38.700527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:38.764039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:38.840774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:38.841465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:38.842024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:38.842375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] PATH_INFO: `/v3/projects/275fe06a58ac4e0994c87079255a7faa` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:38.844473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=275fe06a58ac4e0994c87079255a7faa)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:38.846740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cacfd-ea42-47a3-af91-a4a0b8fe8139 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:38.846863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:38.852357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:38.853037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:38.853358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:38.853670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] PATH_INFO: `/v3/projects/c2e6f2d4d90d489d88762a6a0a97ed81` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:38.855160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e6f2d4d90d489d88762a6a0a97ed81)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:38.857159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:38.886587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:38.924449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e6f2d4d90d489d88762a6a0a97ed81', 'request_id': 'req-59760361-dabe-414e-bdef-7969130f926b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.935245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e6f2d4d90d489d88762a6a0a97ed81', 'request_id': 'req-59760361-dabe-414e-bdef-7969130f926b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.937543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275fe06a58ac4e0994c87079255a7faa', 'request_id': 'req-9cecd8ae-8fef-4d7a-935b-b401f30420fd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.949367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275fe06a58ac4e0994c87079255a7faa', 'request_id': 'req-9cecd8ae-8fef-4d7a-935b-b401f30420fd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.952403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e6f2d4d90d489d88762a6a0a97ed81', 'request_id': 'req-59760361-dabe-414e-bdef-7969130f926b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.958768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275fe06a58ac4e0994c87079255a7faa', 'request_id': 'req-9cecd8ae-8fef-4d7a-935b-b401f30420fd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.979159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] The token cache is being invalidate because project c2e6f2d4d90d489d88762a6a0a97ed81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:57:38.980269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.981915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59760361-dabe-414e-bdef-7969130f926b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.987309 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1704/3412] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:57:38 2022] DELETE /identity/v3/projects/c2e6f2d4d90d489d88762a6a0a97ed81 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:38.989569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] The token cache is being invalidate because project 275fe06a58ac4e0994c87079255a7faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:57:38.990060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.990854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cecd8ae-8fef-4d7a-935b-b401f30420fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:57:38.993182 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1709/3413] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:57:38 2022] DELETE /identity/v3/projects/275fe06a58ac4e0994c87079255a7faa => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:38.993657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68069-f331-4565-99be-21086cbe3017 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:38.994054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68069-f331-4565-99be-21086cbe3017 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:38.994405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68069-f331-4565-99be-21086cbe3017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:38.999728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a97a4-bc31-4eb4-ad70-35f413f89f8f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.000019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a97a4-bc31-4eb4-ad70-35f413f89f8f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.000294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a97a4-bc31-4eb4-ad70-35f413f89f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.036471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e79a97a4-bc31-4eb4-ad70-35f413f89f8f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:39.037904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e79a97a4-bc31-4eb4-ad70-35f413f89f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:39.042315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-88a68069-f331-4565-99be-21086cbe3017 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:39.043713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88a68069-f331-4565-99be-21086cbe3017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:39.137867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1710/3414] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:57:39.138759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1705/3415] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:39.145700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abfa48-69b8-41f4-8732-c92aceb59709 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.146016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b3d7-f75c-4a38-8d9e-49d866f703d7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.146125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abfa48-69b8-41f4-8732-c92aceb59709 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.146295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abfa48-69b8-41f4-8732-c92aceb59709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.146396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b3d7-f75c-4a38-8d9e-49d866f703d7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.146567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b3d7-f75c-4a38-8d9e-49d866f703d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.184725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-72abfa48-69b8-41f4-8732-c92aceb59709 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:39.186031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-72abfa48-69b8-41f4-8732-c92aceb59709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:39.196501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8ec6b3d7-f75c-4a38-8d9e-49d866f703d7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:39.198718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8ec6b3d7-f75c-4a38-8d9e-49d866f703d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:39.204222 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1711/3416] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:39.208638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abfa48-69b8-41f4-8732-c92aceb59709 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.214124 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1706/3417] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:39.214769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6b3d7-f75c-4a38-8d9e-49d866f703d7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.219370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abfa48-69b8-41f4-8732-c92aceb59709 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.226087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.227216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.227500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.228129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.228260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6b3d7-f75c-4a38-8d9e-49d866f703d7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.228889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:57:39.229165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.230797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.234009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.234614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.234895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.235168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.236513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:57:39.236833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.238811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.247395 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1712/3418] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:57:39.249067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.259152 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1707/3419] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:57:39.263236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.270165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8965e5b-eee1-413b-976d-6dfa40109a91 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.276550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35f747-43ff-4b50-a9dc-3d77bccb3fa5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.277456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.278158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.278445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.279050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.279694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.281641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.284588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.285467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.286124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.286498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.287370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.291342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.316683 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1713/3420] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:39.321702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.332277 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1708/3421] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:39.338952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.357037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:39.434979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd695d4-48df-4842-b04d-16d9a5a29c05 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.441795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.442594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.443020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.443375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.444015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.446769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5a208-a01b-4c97-9d4e-0b1c04ff285e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.447704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.451988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.453176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.453467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.453763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.454438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.456855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.467117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:39.480006 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1714/3422] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:39.484346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.489299 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1709/3423] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:39.495013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.509174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a78e9-c3f3-42a3-84c1-7d22774e294c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.510810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e862c-9bb1-4924-a133-e774724f4dce admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.514568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.515282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.515667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.516094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.516772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.518512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.528550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.529278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.529629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.529937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.530586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.533150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.540897 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1710/3424] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:39.541776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.559873 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1715/3425] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:39.560819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.571703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349d770-76f1-4a76-b7a0-04d5d75c156a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.574621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764b1fd-1496-4095-9f30-83648b451d81 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.581410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.582718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.583216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.583334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.584076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.584316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.584432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.584619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.585048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.585269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.587284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.592152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.600333 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1716/3426] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:39.605904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.619219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1711/3427] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:39.621047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.623901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf70840-5183-41bf-9de4-50df71d616b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.633030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030084e-4f1b-4536-a2ad-c0aaf4b5e0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.633772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030084e-4f1b-4536-a2ad-c0aaf4b5e0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.634092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030084e-4f1b-4536-a2ad-c0aaf4b5e0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.634374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030084e-4f1b-4536-a2ad-c0aaf4b5e0a4 admin admin] PATH_INFO: `/v3/projects/d30ef9b7525e4784be26268f46121e62/users/f38321284ec449e599d645625ed11139/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.645144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8211ea-e456-400a-84e2-e7996da53ee1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.654863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030084e-4f1b-4536-a2ad-c0aaf4b5e0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30ef9b7525e4784be26268f46121e62, user_id=f38321284ec449e599d645625ed11139, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.657304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030084e-4f1b-4536-a2ad-c0aaf4b5e0a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.659192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8feea0-e812-4a19-a255-e628726db4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.659859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feea0-e812-4a19-a255-e628726db4d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.660139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feea0-e812-4a19-a255-e628726db4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.660413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feea0-e812-4a19-a255-e628726db4d0 admin admin] PATH_INFO: `/v3/projects/d640711fd8c64403adc4d7e7235a8fad/users/2bed1a15bbff41c6928afd3c057e7dfc/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.693470 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1717/3428] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:57:39 2022] PUT /identity/v3/projects/d30ef9b7525e4784be26268f46121e62/users/f38321284ec449e599d645625ed11139/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:39.699167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8feea0-e812-4a19-a255-e628726db4d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d640711fd8c64403adc4d7e7235a8fad, user_id=2bed1a15bbff41c6928afd3c057e7dfc, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.702299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8feea0-e812-4a19-a255-e628726db4d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.707855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61247-891f-41b8-92a8-1809e47e0306 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.708290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61247-891f-41b8-92a8-1809e47e0306 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.708603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61247-891f-41b8-92a8-1809e47e0306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.725123 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1712/3429] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:57:39 2022] PUT /identity/v3/projects/d640711fd8c64403adc4d7e7235a8fad/users/2bed1a15bbff41c6928afd3c057e7dfc/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:39.731085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.731565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.731982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.751886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-12d61247-891f-41b8-92a8-1809e47e0306 None None] MFA Rules not processed for user `f38321284ec449e599d645625ed11139`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:39.754223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-12d61247-891f-41b8-92a8-1809e47e0306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:39.758165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:39.760446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:39.844410 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1713/3430] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:39.857547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.859328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:39.871683 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1718/3431] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:57:39.873223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d61247-891f-41b8-92a8-1809e47e0306 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:39.891653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d61247-891f-41b8-92a8-1809e47e0306 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:39.906094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:39.907062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:39.907467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:39.907850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:39.908588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:39.912069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:39.989980 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1719/3432] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:39.993152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:40.059113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec085877-d245-41fe-bfe2-7a083a7f909d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:40.079168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:40.186943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:40.187426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.187711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.187978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.238658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce554844-3595-458c-9f84-71440abf59e3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:40.244520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:40.247468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.247923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.248370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.249473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:40.252291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:40.290979 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1720/3433] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:57:39 2022] POST /identity/v3/users => generated 298 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:57:40.293009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:40.308664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:40.311084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:40.313120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2fd0e-a006-457a-aa17-ac59cef83a70 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:40.326568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:40.327195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.327476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.327753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.328574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:40.332596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:40.334174 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1714/3434] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:40.339504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:40.348078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1721/3435] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:57:40.351710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:40.362805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebabe79-07c4-4d92-b42d-9244a09f0d66 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:40.370121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:40.370822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.371098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.371375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] PATH_INFO: `/v3/projects/80bf9de322d94dfcaaf62eb62aeba095/users/9d79404e087f4e3e87f52ac0941a33c9/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.399521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bf9de322d94dfcaaf62eb62aeba095, user_id=9d79404e087f4e3e87f52ac0941a33c9, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:40.404008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:40.428346 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1715/3436] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:57:40 2022] PUT /identity/v3/projects/80bf9de322d94dfcaaf62eb62aeba095/users/9d79404e087f4e3e87f52ac0941a33c9/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:40.430284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:40.442984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf424-9007-41ed-b799-6522ec6f6d3e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:40.514920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be8215-9c59-4438-a585-2791052a5701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:40.515512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be8215-9c59-4438-a585-2791052a5701 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.515803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be8215-9c59-4438-a585-2791052a5701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.516082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be8215-9c59-4438-a585-2791052a5701 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.516739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be8215-9c59-4438-a585-2791052a5701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:40.518400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be8215-9c59-4438-a585-2791052a5701 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:40.530811 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1722/3437] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:57:40.534557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:40.548041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cae2f-50a7-44a8-9b04-1a3681006e7e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:40.556271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32a8b3-998a-4db1-a16e-6ae48fc3115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:40.557661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32a8b3-998a-4db1-a16e-6ae48fc3115f admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.558875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32a8b3-998a-4db1-a16e-6ae48fc3115f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.559381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32a8b3-998a-4db1-a16e-6ae48fc3115f admin admin] PATH_INFO: `/v3/projects/80bf9de322d94dfcaaf62eb62aeba095/users/9d79404e087f4e3e87f52ac0941a33c9/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.565264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32a8b3-998a-4db1-a16e-6ae48fc3115f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bf9de322d94dfcaaf62eb62aeba095, user_id=9d79404e087f4e3e87f52ac0941a33c9, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:40.571469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32a8b3-998a-4db1-a16e-6ae48fc3115f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:40.591033 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1716/3438] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:57:40 2022] PUT /identity/v3/projects/80bf9de322d94dfcaaf62eb62aeba095/users/9d79404e087f4e3e87f52ac0941a33c9/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:57:40.603207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00d511-7b47-4c0d-92c7-eeab57bd9d34 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.603620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00d511-7b47-4c0d-92c7-eeab57bd9d34 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.603918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00d511-7b47-4c0d-92c7-eeab57bd9d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.668388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ce00d511-7b47-4c0d-92c7-eeab57bd9d34 None None] MFA Rules not processed for user `9d79404e087f4e3e87f52ac0941a33c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:40.669703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ce00d511-7b47-4c0d-92c7-eeab57bd9d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:40.694281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.694647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.695050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.735231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:40.736601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:40.857922 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1723/3439] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:40.871521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:40.871904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:40.872278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:40.883306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1717/3440] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:57:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:57:40.913171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:40.915572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:40.919274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] MFA Rules not processed for user `f38321284ec449e599d645625ed11139`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:40.921230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:40.962103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:41.015858 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1724/3441] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:57:41.024726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4883d-2332-4523-bba1-6af3831fcaec None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:41.030938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f75494-6732-4e39-928c-5043af82bacb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:41.031757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f75494-6732-4e39-928c-5043af82bacb service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:41.032468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f75494-6732-4e39-928c-5043af82bacb service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:41.032468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f75494-6732-4e39-928c-5043af82bacb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:41.046246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f75494-6732-4e39-928c-5043af82bacb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:41.048114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f75494-6732-4e39-928c-5043af82bacb service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:41.064888 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1718/3442] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:57:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 20:57:41.453852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:41.455683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:41.516364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:57:41.616322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ac996-00ff-4460-86cd-04f0cc0fd516 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:41.624612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:41.626397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:41.627014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:41.628166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:41.647899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:41.649979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:41.668361 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1725/3443] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:57:42.858053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ba2cb-3ea3-486a-b055-29ef9b8b7626 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:42.858569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ba2cb-3ea3-486a-b055-29ef9b8b7626 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:42.858649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ba2cb-3ea3-486a-b055-29ef9b8b7626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:42.891887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-284ba2cb-3ea3-486a-b055-29ef9b8b7626 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:42.893248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-284ba2cb-3ea3-486a-b055-29ef9b8b7626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:42.910308 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1719/3444] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:57:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:42.925721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:42.953957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f0c7a-871b-45e1-a7b6-bfbfa878a1fa service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:42.983650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d171fa-5549-4c91-96c3-c0fba9d56f93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:42.983650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d171fa-5549-4c91-96c3-c0fba9d56f93 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:42.983650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d171fa-5549-4c91-96c3-c0fba9d56f93 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:42.984048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d171fa-5549-4c91-96c3-c0fba9d56f93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:43.015692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d171fa-5549-4c91-96c3-c0fba9d56f93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:43.017310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d171fa-5549-4c91-96c3-c0fba9d56f93 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:43.073289 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1726/3445] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:57:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:57:44.036998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e42ec-3d0e-471e-bb3f-d7a15e25afc4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:44.036998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e42ec-3d0e-471e-bb3f-d7a15e25afc4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:44.036998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e42ec-3d0e-471e-bb3f-d7a15e25afc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:44.104552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9c6e42ec-3d0e-471e-bb3f-d7a15e25afc4 None None] MFA Rules not processed for user `d8b0a160808f4adda7be57cd535f0631`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:44.104552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9c6e42ec-3d0e-471e-bb3f-d7a15e25afc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:44.300198 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1720/3446] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:51.034182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:51.034644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:51.034969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:51.091430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:51.093508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:51.202658 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1727/3447] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:57:51.208098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22a4e-2416-4a40-ab7b-9568ff185a48 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:51.208602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22a4e-2416-4a40-ab7b-9568ff185a48 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:51.208716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22a4e-2416-4a40-ab7b-9568ff185a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:51.244262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-17a22a4e-2416-4a40-ab7b-9568ff185a48 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:57:51.245570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-17a22a4e-2416-4a40-ab7b-9568ff185a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:51.262292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1721/3448] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:57:58.170937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:57:58.174012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:57:58.223527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1862e3-ddf8-4eb8-8e0c-f4b1b45800eb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:57:58.240370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf7243-035e-4042-ab1a-c4cb44a467e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:57:58.241558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf7243-035e-4042-ab1a-c4cb44a467e0 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:57:58.242017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf7243-035e-4042-ab1a-c4cb44a467e0 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:57:58.242659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf7243-035e-4042-ab1a-c4cb44a467e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:57:58.263297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf7243-035e-4042-ab1a-c4cb44a467e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:57:58.265803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf7243-035e-4042-ab1a-c4cb44a467e0 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:57:58.282443 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1728/3449] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:57:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:00.591282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a22a4e-2416-4a40-ab7b-9568ff185a48 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:00.605761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a22a4e-2416-4a40-ab7b-9568ff185a48 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:00.611319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:00.611939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:00.612216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:00.612488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:00.622792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:00.624412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:00.637741 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1722/3450] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:01.167630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15080b-75ca-4f2f-badb-41a334575e74 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:01.168142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15080b-75ca-4f2f-badb-41a334575e74 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:01.168221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15080b-75ca-4f2f-badb-41a334575e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:01.188167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dc15080b-75ca-4f2f-badb-41a334575e74 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:01.189622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dc15080b-75ca-4f2f-badb-41a334575e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:01.205203 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1729/3451] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:01.213622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:01.227687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46716302-227f-47c9-b1e8-a72096a6459c service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:01.233240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc805e3-7a23-4346-b252-9dfd6d8bab66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:01.233878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc805e3-7a23-4346-b252-9dfd6d8bab66 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:01.234165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc805e3-7a23-4346-b252-9dfd6d8bab66 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:01.234439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc805e3-7a23-4346-b252-9dfd6d8bab66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:01.245137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc805e3-7a23-4346-b252-9dfd6d8bab66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:01.247100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc805e3-7a23-4346-b252-9dfd6d8bab66 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:01.263643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1723/3452] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:58:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:58:02.411902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b314f5-da5f-4d93-a397-cd03740ebff7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:02.412800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b314f5-da5f-4d93-a397-cd03740ebff7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:02.413035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b314f5-da5f-4d93-a397-cd03740ebff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:02.442853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-69b314f5-da5f-4d93-a397-cd03740ebff7 None None] MFA Rules not processed for user `f38321284ec449e599d645625ed11139`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:02.445442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-69b314f5-da5f-4d93-a397-cd03740ebff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:02.456311 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1730/3453] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:02.749421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:02.749926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:02.750401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:02.784425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] MFA Rules not processed for user `687eef270165450a97b4d260b2724b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:02.786181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:02.852429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9e8a-6266-4228-83f8-f7cc2678b7fe None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:02.852947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9e8a-6266-4228-83f8-f7cc2678b7fe None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:02.853428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9e8a-6266-4228-83f8-f7cc2678b7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:02.889367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-682b9e8a-6266-4228-83f8-f7cc2678b7fe None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:02.890992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-682b9e8a-6266-4228-83f8-f7cc2678b7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:02.912446 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1731/3454] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:02.926506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b9e8a-6266-4228-83f8-f7cc2678b7fe None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:02.937298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1724/3455] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:02.952016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b9e8a-6266-4228-83f8-f7cc2678b7fe None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:02.970299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:02.972977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:02.978970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:02.981094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:03.001613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:03.003385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:03.021713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1732/3456] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 20:58:10.461940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:10.464034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:10.502214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:10.532871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa80da9-9a6f-4cb7-9744-aacd21370229 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:10.536670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6db3e5-7dd4-4bbd-b018-e04354df9489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:10.537289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db3e5-7dd4-4bbd-b018-e04354df9489 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:10.538607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db3e5-7dd4-4bbd-b018-e04354df9489 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:10.538887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6db3e5-7dd4-4bbd-b018-e04354df9489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:10.540680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cc6db3e5-7dd4-4bbd-b018-e04354df9489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:10.559395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6db3e5-7dd4-4bbd-b018-e04354df9489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:10.560863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6db3e5-7dd4-4bbd-b018-e04354df9489 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:10.573206 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1725/3457] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:12.540323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:12.544968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:12.567804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f600b-8f69-4798-81a1-3cf6da958d36 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:12.574303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baea6a-2b9c-4ba6-a90c-a6cb964489ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:12.575204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baea6a-2b9c-4ba6-a90c-a6cb964489ef service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:12.575713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baea6a-2b9c-4ba6-a90c-a6cb964489ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:12.576155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baea6a-2b9c-4ba6-a90c-a6cb964489ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:12.578395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24baea6a-2b9c-4ba6-a90c-a6cb964489ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:12.622258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baea6a-2b9c-4ba6-a90c-a6cb964489ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:12.624078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baea6a-2b9c-4ba6-a90c-a6cb964489ef service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:12.639515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1733/3458] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:15.152796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198db2d-a83b-4366-99db-e7010986e045 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:15.153347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198db2d-a83b-4366-99db-e7010986e045 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:15.153347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198db2d-a83b-4366-99db-e7010986e045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:15.187633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c198db2d-a83b-4366-99db-e7010986e045 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:15.190122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c198db2d-a83b-4366-99db-e7010986e045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:15.210793 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1726/3459] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:15.254072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:15.254975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:15.255259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:15.319853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:15.321343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:15.340362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1734/3460] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:15.358131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198db2d-a83b-4366-99db-e7010986e045 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:15.387534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198db2d-a83b-4366-99db-e7010986e045 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:15.393748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:15.394681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:15.395228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:15.395704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:15.409690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:15.412034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:15.426424 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1727/3461] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:58:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:58:16.689571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:16.691747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:16.838166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305739a3-fc53-42d9-ba36-9daaca7f2a86 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:16.848087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c47a0-59db-4a07-abed-c14721b7d6f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:16.849395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c47a0-59db-4a07-abed-c14721b7d6f3 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:16.849697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c47a0-59db-4a07-abed-c14721b7d6f3 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:16.850639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c47a0-59db-4a07-abed-c14721b7d6f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:16.853298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b80c47a0-59db-4a07-abed-c14721b7d6f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:16.872871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c47a0-59db-4a07-abed-c14721b7d6f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:16.875789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c47a0-59db-4a07-abed-c14721b7d6f3 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:16.891108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1735/3462] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:20.664520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:20.679173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7fd24-cd2f-48d6-80a1-64da4e10a651 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:20.688316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:20.689277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:20.689566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:20.689874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:20.700336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:20.723221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:20.725648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:20.758439 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1728/3463] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:22.946735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e2de-a64c-4a96-9050-b4966b7b2c9a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:22.947683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e2de-a64c-4a96-9050-b4966b7b2c9a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:22.948156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3e2de-a64c-4a96-9050-b4966b7b2c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:23.015208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-48c3e2de-a64c-4a96-9050-b4966b7b2c9a None None] MFA Rules not processed for user `4c3e0565f75c4f558a8936f547ae9d76`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:23.023452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-48c3e2de-a64c-4a96-9050-b4966b7b2c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:23.027993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:23.030600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:23.199252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bc17c-ad7c-4396-8132-df7aebb8f35e service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:23.205543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:23.206484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:23.206940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:23.208076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] PATH_INFO: `/v3/users/609617a03387478b9b72cbe52d25dbf0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:23.211310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=609617a03387478b9b72cbe52d25dbf0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:23.220219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:23.241364 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1736/3464] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:23.300251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609617a03387478b9b72cbe52d25dbf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:23.353924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609617a03387478b9b72cbe52d25dbf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:23.354931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609617a03387478b9b72cbe52d25dbf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:23.384368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609617a03387478b9b72cbe52d25dbf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:23.385237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570f7610-0369-45de-902c-b489fb615b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609617a03387478b9b72cbe52d25dbf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:23.441008 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1729/3465] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:58:23 2022] DELETE /identity/v3/users/609617a03387478b9b72cbe52d25dbf0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:24.032302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3e2de-a64c-4a96-9050-b4966b7b2c9a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:24.058887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3e2de-a64c-4a96-9050-b4966b7b2c9a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:24.182440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:24.231204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:24.231987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:24.232264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:24.232547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] PATH_INFO: `/v3/projects/f1f9b47aca3845d6a3fc34662d34ccf1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:24.234011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f9b47aca3845d6a3fc34662d34ccf1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:24.236124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:24.371120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f9b47aca3845d6a3fc34662d34ccf1', 'request_id': 'req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:24.378368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f9b47aca3845d6a3fc34662d34ccf1', 'request_id': 'req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:24.406737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f9b47aca3845d6a3fc34662d34ccf1', 'request_id': 'req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:24.448812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] The token cache is being invalidate because project f1f9b47aca3845d6a3fc34662d34ccf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:58:24.449727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:24.452017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-492aa84e-f252-4d8c-b1fb-9b5dccc49f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:24.457704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1737/3466] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:58:24 2022] DELETE /identity/v3/projects/f1f9b47aca3845d6a3fc34662d34ccf1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:24.463407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dccc9-3e98-4320-9fc7-1eb00e5925f8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:24.463903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dccc9-3e98-4320-9fc7-1eb00e5925f8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:24.463995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dccc9-3e98-4320-9fc7-1eb00e5925f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:24.531105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-551dccc9-3e98-4320-9fc7-1eb00e5925f8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:24.532704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-551dccc9-3e98-4320-9fc7-1eb00e5925f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:24.538862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:24.698166 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1730/3467] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:24.706637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d118-4b17-401b-8f08-51e3b02529c1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:24.707168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d118-4b17-401b-8f08-51e3b02529c1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:24.707682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40d118-4b17-401b-8f08-51e3b02529c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:24.749112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6b40d118-4b17-401b-8f08-51e3b02529c1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:24.750747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b40d118-4b17-401b-8f08-51e3b02529c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:24.760904 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1738/3468] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:24.767476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dccc9-3e98-4320-9fc7-1eb00e5925f8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:24.797782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dccc9-3e98-4320-9fc7-1eb00e5925f8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:24.805255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:24.806497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:24.806943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:24.807666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:24.808483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:58:24.809570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:24.811691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:24.856522 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1731/3469] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:58:24.861044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40d118-4b17-401b-8f08-51e3b02529c1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:24.893606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40d118-4b17-401b-8f08-51e3b02529c1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:24.910167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:24.911814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:24.914608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:24.915142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:24.917179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:24.922488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:25.002332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1739/3470] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:58:24 2022] POST /identity/v3/projects => generated 403 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:25.006413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:25.018640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26ed5b-9513-4c2b-8950-40ecb830d5f3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:25.176065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:25.177023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:25.177373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:25.179152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:25.179949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:25.182079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:25.233582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:25.242000 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1732/3471] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:58:25 2022] POST /identity/v3/users => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:25.261303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcb41d-6398-44f2-bd52-071df94744a9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:25.270165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:25.271091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:25.271662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:25.272181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:25.273130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:25.276273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:25.289934 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1740/3472] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:25.294147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:25.309338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a936d8-0698-455e-93c6-c1ea3907b16e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:25.316363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889c1d8-0992-4641-a8c3-3b58cc7916ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:25.317483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889c1d8-0992-4641-a8c3-3b58cc7916ef admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:25.318020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889c1d8-0992-4641-a8c3-3b58cc7916ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:25.318524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889c1d8-0992-4641-a8c3-3b58cc7916ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:25.320214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889c1d8-0992-4641-a8c3-3b58cc7916ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:25.322137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889c1d8-0992-4641-a8c3-3b58cc7916ef admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:25.336899 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1733/3473] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:25.341066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:25.359105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d3d7-2edc-4f04-86f4-f60162e9af27 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:25.366466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:25.367811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:25.369113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:25.370827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] PATH_INFO: `/v3/projects/cd4fdf2f6f994535805d6f8110234477/users/3b5b2397579544ab98e4e871a52cf114/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:25.396345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4fdf2f6f994535805d6f8110234477, user_id=3b5b2397579544ab98e4e871a52cf114, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:25.399364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:25.420011 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1741/3474] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:58:25 2022] PUT /identity/v3/projects/cd4fdf2f6f994535805d6f8110234477/users/3b5b2397579544ab98e4e871a52cf114/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:25.425612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738568-2792-4c94-8fec-e509f7bc6eb0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:25.426179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738568-2792-4c94-8fec-e509f7bc6eb0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:25.426701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55738568-2792-4c94-8fec-e509f7bc6eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:25.453552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-55738568-2792-4c94-8fec-e509f7bc6eb0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:25.455146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55738568-2792-4c94-8fec-e509f7bc6eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:25.634578 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1734/3475] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:25.643833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:25.645702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:25.778653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b2a27-b0ca-42f5-89ef-01b6dc1882ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:25.783256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:25.784020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:25.784896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:25.785188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:25.832429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:25.838673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:25.861978 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1742/3476] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:30.979670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55738568-2792-4c94-8fec-e509f7bc6eb0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:30.995071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55738568-2792-4c94-8fec-e509f7bc6eb0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:30.999456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:31.001291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:31.016064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:31.019791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d5a62-72e1-4769-a47e-751487eea83d service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:31.020056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:31.021074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:31.023433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:31.023993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:31.027019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433fa90-e1bf-4cab-a885-0bbdc26d8d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:31.030066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433fa90-e1bf-4cab-a885-0bbdc26d8d13 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:31.030352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433fa90-e1bf-4cab-a885-0bbdc26d8d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:31.030719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433fa90-e1bf-4cab-a885-0bbdc26d8d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:31.033879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:31.036284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2433fa90-e1bf-4cab-a885-0bbdc26d8d13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:31.066679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433fa90-e1bf-4cab-a885-0bbdc26d8d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:31.068479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:31.070014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:31.070353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433fa90-e1bf-4cab-a885-0bbdc26d8d13 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:31.091784 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1735/3477] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:31.106794 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1743/3478] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:58:32.631552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:32.633533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:32.638856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:32.810646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f339250-e563-4f50-9f1b-4c08d67a457f service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:32.818835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b8a06-a144-4308-bc3d-5a68269125e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:32.819469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b8a06-a144-4308-bc3d-5a68269125e9 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:32.819902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b8a06-a144-4308-bc3d-5a68269125e9 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:32.820186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b8a06-a144-4308-bc3d-5a68269125e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:32.825629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0f9b8a06-a144-4308-bc3d-5a68269125e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:32.858560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b8a06-a144-4308-bc3d-5a68269125e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:32.860203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b8a06-a144-4308-bc3d-5a68269125e9 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:32.898092 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1736/3479] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:34.513522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642af4-8ca7-45ce-81de-4f42027b79f7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:34.514108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642af4-8ca7-45ce-81de-4f42027b79f7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:34.514108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642af4-8ca7-45ce-81de-4f42027b79f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:34.569444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-22642af4-8ca7-45ce-81de-4f42027b79f7 None None] MFA Rules not processed for user `f38321284ec449e599d645625ed11139`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:34.576756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22642af4-8ca7-45ce-81de-4f42027b79f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:34.714714 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1744/3480] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:35.742739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c88dc-ac55-44a7-a36e-bba5aaa27897 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:35.743763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c88dc-ac55-44a7-a36e-bba5aaa27897 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:35.744056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c88dc-ac55-44a7-a36e-bba5aaa27897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:35.799492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-794c88dc-ac55-44a7-a36e-bba5aaa27897 None None] MFA Rules not processed for user `3b5b2397579544ab98e4e871a52cf114`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:35.801181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-794c88dc-ac55-44a7-a36e-bba5aaa27897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:35.925557 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1737/3481] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:35 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:36.098868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642af4-8ca7-45ce-81de-4f42027b79f7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:36.115245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642af4-8ca7-45ce-81de-4f42027b79f7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:36.134420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:36.135049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:36.135398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:36.135602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:36.146267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:36.147855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:36.160148 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1745/3482] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:36 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:58:37.381907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:37.382448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:37.382565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:37.423899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:37.425221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:37.550719 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1738/3483] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 20:58:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:37.569801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:37.571349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:37.730885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1be14-6965-4c7f-936c-8ec08768acc0 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:37.745273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:37.746370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:37.746653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:37.747459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:37.766301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:37.768305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:37.790654 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1746/3484] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:58:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:58:41.371023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:41.374001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:41.424214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4adda-bc02-4d2a-bdfb-4c4174eaede9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:41.449145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:41.449799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:41.450130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:41.450568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:41.452502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:41.481663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:41.483548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:41.510688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1739/3485] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:44.781627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:44.793999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe26b32-27ff-483e-b2a9-f9d285710baa service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:44.799170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:44.799991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:44.800454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:44.801224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] PATH_INFO: `/v3/users/3b5b2397579544ab98e4e871a52cf114` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:44.803221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5b2397579544ab98e4e871a52cf114)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:44.806259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:44.878396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5b2397579544ab98e4e871a52cf114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:44.894679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:44.914133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5b2397579544ab98e4e871a52cf114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:44.923094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5b2397579544ab98e4e871a52cf114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:44.929237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5b2397579544ab98e4e871a52cf114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:44.930117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e34f5335-2f95-4051-b709-d7be3f1c1a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5b2397579544ab98e4e871a52cf114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:44.944310 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1747/3486] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:58:44 2022] DELETE /identity/v3/users/3b5b2397579544ab98e4e871a52cf114 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:45.657113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:45.674840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd8c2e-0df0-4d4d-b1de-baa9564a4198 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:45.698236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:45.856072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:45.856725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:45.857007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:45.857466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] PATH_INFO: `/v3/projects/cd4fdf2f6f994535805d6f8110234477` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:45.861589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd4fdf2f6f994535805d6f8110234477)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:45.863816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:45.977322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd4fdf2f6f994535805d6f8110234477', 'request_id': 'req-171d27fc-e5a3-4843-acb3-027615f9e0d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:45.997664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd4fdf2f6f994535805d6f8110234477', 'request_id': 'req-171d27fc-e5a3-4843-acb3-027615f9e0d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.011148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd4fdf2f6f994535805d6f8110234477', 'request_id': 'req-171d27fc-e5a3-4843-acb3-027615f9e0d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.029469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] The token cache is being invalidate because project cd4fdf2f6f994535805d6f8110234477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:58:46.029762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.030637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171d27fc-e5a3-4843-acb3-027615f9e0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.035323 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1740/3487] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:58:45 2022] DELETE /identity/v3/projects/cd4fdf2f6f994535805d6f8110234477 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:46.040672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.041218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.041218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.069939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:46.072729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:46.162793 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1748/3488] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:46.170567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd407ee-53c9-4ae1-99a2-01aceb06f67e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.170965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd407ee-53c9-4ae1-99a2-01aceb06f67e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.171377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd407ee-53c9-4ae1-99a2-01aceb06f67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.211084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.212703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:46.220743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7dd407ee-53c9-4ae1-99a2-01aceb06f67e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:46.222261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7dd407ee-53c9-4ae1-99a2-01aceb06f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:46.228574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804d0d1-2d35-43c6-b2b4-efa4790d2bc1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.236623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.237952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.237952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.237952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] PATH_INFO: `/v3/users/bec33de4067d4b1bb6e4ad866ddf2d01` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.239327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec33de4067d4b1bb6e4ad866ddf2d01)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.242559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.245594 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1741/3489] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:46.249835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd407ee-53c9-4ae1-99a2-01aceb06f67e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.272642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd407ee-53c9-4ae1-99a2-01aceb06f67e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.286690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.287872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.288382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.288808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.289705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:58:46.290226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.297143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.320684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec33de4067d4b1bb6e4ad866ddf2d01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.322932 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1742/3490] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:58:46.325287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.341308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec33de4067d4b1bb6e4ad866ddf2d01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.345270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13da9b-78b6-4cbf-a994-9f2c2ed139d4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.356688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec33de4067d4b1bb6e4ad866ddf2d01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.367420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.368051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.368339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.368619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.369264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.371369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.374334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec33de4067d4b1bb6e4ad866ddf2d01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.375426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc679aaa-9dea-4983-aa9e-57ff48109be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec33de4067d4b1bb6e4ad866ddf2d01'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:46.385017 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1749/3491] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:58:46 2022] DELETE /identity/v3/users/bec33de4067d4b1bb6e4ad866ddf2d01 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 20:58:46.390752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.390929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.391487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.453365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:46.454767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:46.459464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1743/3492] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:58:46 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:46.465099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.468159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:46.557649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:46.601184 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1750/3493] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:46.629164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:46.635373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a8a9-bda2-4c49-ad99-c8ba3039e7ef admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.643345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.644189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.644587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.645468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.647880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.650611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.672685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.675059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:46.692195 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1744/3494] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:58:46 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:46.696289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.714676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aecc92-f58b-4262-8ba7-9a43424e02a1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.733778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.734611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.736162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.736628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.737725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.742580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.768913 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1745/3495] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:58:46.773807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.785260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263c9dd-8426-44fc-918f-e712a344794e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.798432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.799135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.801848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.804348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.805696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.806906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580cf79-2d4b-4d52-ae93-48bff377ee92 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.810213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.819267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf06f3-e55a-4ae8-8217-a5d4ce59c0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.821354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf06f3-e55a-4ae8-8217-a5d4ce59c0fa service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.821706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf06f3-e55a-4ae8-8217-a5d4ce59c0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.823479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf06f3-e55a-4ae8-8217-a5d4ce59c0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.832842 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1746/3496] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:46.840286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:46.840499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf06f3-e55a-4ae8-8217-a5d4ce59c0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.842466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf06f3-e55a-4ae8-8217-a5d4ce59c0fa service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.856790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a23009-64d2-411d-9c0d-3f5943f2983d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:46.864062 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1751/3497] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:58:46.874684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d22c9-3992-43f9-a31e-6ee868602c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:46.875592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d22c9-3992-43f9-a31e-6ee868602c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.875969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d22c9-3992-43f9-a31e-6ee868602c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.876617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d22c9-3992-43f9-a31e-6ee868602c10 admin admin] PATH_INFO: `/v3/projects/9c091a483f8b466f89ea487d13fe422b/users/81ee4fa83ef649d0bdc273243cd717c4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:46.908613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d22c9-3992-43f9-a31e-6ee868602c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c091a483f8b466f89ea487d13fe422b, user_id=81ee4fa83ef649d0bdc273243cd717c4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:46.915647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d22c9-3992-43f9-a31e-6ee868602c10 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:46.945218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1747/3498] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:58:46 2022] PUT /identity/v3/projects/9c091a483f8b466f89ea487d13fe422b/users/81ee4fa83ef649d0bdc273243cd717c4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:46.956718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:46.957193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:46.957626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:47.006655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:47.007117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:47.008077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:47.021722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] MFA Rules not processed for user `81ee4fa83ef649d0bdc273243cd717c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:47.025483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:47.050102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:47.051571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:47.133125 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1752/3499] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:47.143055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:47.144620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:47.201974 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1748/3500] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:47.341885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30801269-5c5c-4e94-84d9-c5a4f9588432 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:47.347004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355db1ae-86c4-45c6-b3f1-c89a937c250d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:47.348648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355db1ae-86c4-45c6-b3f1-c89a937c250d service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:47.349877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355db1ae-86c4-45c6-b3f1-c89a937c250d service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:47.350458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355db1ae-86c4-45c6-b3f1-c89a937c250d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:47.363391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355db1ae-86c4-45c6-b3f1-c89a937c250d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:47.365391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355db1ae-86c4-45c6-b3f1-c89a937c250d service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:47.387796 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1753/3501] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:47.684559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:47.686102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:47.797670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc853ff-8edb-4ad0-a134-1461294e9424 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:47.809673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:47.810468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:47.814105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:47.814640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] PATH_INFO: `/v3/projects/f57a1c44d4684b2fa210973f062adbce` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:47.816095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f57a1c44d4684b2fa210973f062adbce)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:47.818759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:47.892897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:47.894080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f57a1c44d4684b2fa210973f062adbce', 'request_id': 'req-88e4c424-c52e-46af-a2cb-28f00cb71ef0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:47.906557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f57a1c44d4684b2fa210973f062adbce', 'request_id': 'req-88e4c424-c52e-46af-a2cb-28f00cb71ef0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:47.922272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f57a1c44d4684b2fa210973f062adbce', 'request_id': 'req-88e4c424-c52e-46af-a2cb-28f00cb71ef0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:47.941216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] The token cache is being invalidate because project f57a1c44d4684b2fa210973f062adbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:58:47.941577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:47.945875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:47.948437 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1749/3502] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:58:47 2022] DELETE /identity/v3/projects/f57a1c44d4684b2fa210973f062adbce => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:47.953358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a62d6-de45-4d2b-84e9-1fd6b02d8994 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:47.954251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a62d6-de45-4d2b-84e9-1fd6b02d8994 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:47.954251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a62d6-de45-4d2b-84e9-1fd6b02d8994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:47.960403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:47.970630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:47.986023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:48.009298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e55a62d6-de45-4d2b-84e9-1fd6b02d8994 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:48.011329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e55a62d6-de45-4d2b-84e9-1fd6b02d8994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:48.082773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4c424-c52e-46af-a2cb-28f00cb71ef0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:48.088352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:48.088990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.089268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.089540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] PATH_INFO: `/v3/users/687eef270165450a97b4d260b2724b3e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.090724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687eef270165450a97b4d260b2724b3e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:48.092912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:48.122192 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1754/3503] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:48.127796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183b621-87d1-46e9-b1ba-151d55409677 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.128165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183b621-87d1-46e9-b1ba-151d55409677 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.128573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183b621-87d1-46e9-b1ba-151d55409677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.156129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9183b621-87d1-46e9-b1ba-151d55409677 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:48.157749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9183b621-87d1-46e9-b1ba-151d55409677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:48.178885 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1755/3504] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:48.183100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183b621-87d1-46e9-b1ba-151d55409677 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:48.193248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687eef270165450a97b4d260b2724b3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.193396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183b621-87d1-46e9-b1ba-151d55409677 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:48.198555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:48.199191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.199588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.199882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.200513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:58:48.200835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:48.202577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:48.202765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687eef270165450a97b4d260b2724b3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.203593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687eef270165450a97b4d260b2724b3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.214553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687eef270165450a97b4d260b2724b3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.215331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687eef270165450a97b4d260b2724b3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.221265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1756/3505] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:58:48.222848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:48.225560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:48.240458 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1750/3506] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:58:47 2022] DELETE /identity/v3/users/687eef270165450a97b4d260b2724b3e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:48.242523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67769-101d-4455-8cee-ed752d0bf682 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:48.339688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:48.341170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.341689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.342215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.343260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:48.347089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:48.410775 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1757/3507] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:58:48 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:48.412714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:48.433236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2a3d-2ef9-40d5-8e9d-181a2aee30e6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:48.467703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:48.470570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:48.510354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:48.517868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:48.518582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.519090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.520270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.523448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:48.524798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:48.574667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1751/3508] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:58:48 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:48.579141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:48.591472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae640b8-70ac-4820-84f2-b3f9c14d76ea admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:48.604507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:48.605151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.605496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.605785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.606542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:48.609345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:48.640601 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1752/3509] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 20:58:48.641559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:48.659675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc297f-67e2-4649-9873-d6d3a03a0465 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:48.671932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621532c0-f8bd-43f3-8be9-a5689737b135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:48.672735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621532c0-f8bd-43f3-8be9-a5689737b135 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.674137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621532c0-f8bd-43f3-8be9-a5689737b135 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.674434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621532c0-f8bd-43f3-8be9-a5689737b135 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.675142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621532c0-f8bd-43f3-8be9-a5689737b135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:48.676153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c8478-0f78-48c0-adf2-4a5f5f852a54 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:48.677207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621532c0-f8bd-43f3-8be9-a5689737b135 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:48.681705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:48.682404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.682696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.682965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] PATH_INFO: `/v3/projects/e8f7f9f3112545e9b40d88a7cc48e3fe` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.684163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f7f9f3112545e9b40d88a7cc48e3fe)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:48.686720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:48.695750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1753/3510] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:48.698252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099ee48-2990-4f33-9fa0-197f5a419339 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.698544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099ee48-2990-4f33-9fa0-197f5a419339 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.698827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099ee48-2990-4f33-9fa0-197f5a419339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.708038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:48.729612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d099ee48-2990-4f33-9fa0-197f5a419339 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:48.731063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d099ee48-2990-4f33-9fa0-197f5a419339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:48.777490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f7f9f3112545e9b40d88a7cc48e3fe', 'request_id': 'req-e8e69ddb-2e72-43d9-90e5-23308ae04c11'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.784499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f7f9f3112545e9b40d88a7cc48e3fe', 'request_id': 'req-e8e69ddb-2e72-43d9-90e5-23308ae04c11'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.794684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f7f9f3112545e9b40d88a7cc48e3fe', 'request_id': 'req-e8e69ddb-2e72-43d9-90e5-23308ae04c11'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.812972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] The token cache is being invalidate because project e8f7f9f3112545e9b40d88a7cc48e3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:58:48.813288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.814124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:48.820029 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1758/3511] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:58:48 2022] DELETE /identity/v3/projects/e8f7f9f3112545e9b40d88a7cc48e3fe => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:48.820029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:48.822023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:48.866686 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1754/3512] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:48.867696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf4723-182d-450d-9007-13521e12115c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:48.867986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf4723-182d-450d-9007-13521e12115c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:48.868263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf4723-182d-450d-9007-13521e12115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:48.910945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-57bf4723-182d-450d-9007-13521e12115c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:48.912753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-57bf4723-182d-450d-9007-13521e12115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.000374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69ddb-2e72-43d9-90e5-23308ae04c11 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.008370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125646d4-6f0e-466c-b646-a37498e44c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.009641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125646d4-6f0e-466c-b646-a37498e44c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.010290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125646d4-6f0e-466c-b646-a37498e44c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.010572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125646d4-6f0e-466c-b646-a37498e44c91 admin admin] PATH_INFO: `/v3/projects/9168d775bc9e4e3590b90d4625983a2f/users/a7df05f8c4f449b3b634b6bb2181e9ed/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.032865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125646d4-6f0e-466c-b646-a37498e44c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9168d775bc9e4e3590b90d4625983a2f, user_id=a7df05f8c4f449b3b634b6bb2181e9ed, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.038220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125646d4-6f0e-466c-b646-a37498e44c91 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.045400 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1755/3513] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:49.046517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf4723-182d-450d-9007-13521e12115c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.054917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-57bf4723-182d-450d-9007-13521e12115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.072325 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1759/3514] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:58:48 2022] PUT /identity/v3/projects/9168d775bc9e4e3590b90d4625983a2f/users/a7df05f8c4f449b3b634b6bb2181e9ed/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:49.073930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219bb5-dcc3-4368-a8b6-28f4e0e3cb16 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.074225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219bb5-dcc3-4368-a8b6-28f4e0e3cb16 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.074503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219bb5-dcc3-4368-a8b6-28f4e0e3cb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.124270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-25219bb5-dcc3-4368-a8b6-28f4e0e3cb16 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:49.126053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25219bb5-dcc3-4368-a8b6-28f4e0e3cb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.153801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf4723-182d-450d-9007-13521e12115c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.166507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:49.231468 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1760/3515] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:49.232973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc415336-deb8-49c4-9957-f88bdd43d3ea None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.233263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc415336-deb8-49c4-9957-f88bdd43d3ea None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.233534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc415336-deb8-49c4-9957-f88bdd43d3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.265741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.266838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.267121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.267398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.269976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cc415336-deb8-49c4-9957-f88bdd43d3ea None None] MFA Rules not processed for user `a7df05f8c4f449b3b634b6bb2181e9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:49.272259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc415336-deb8-49c4-9957-f88bdd43d3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.283541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:49.352868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.356786 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1761/3516] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:49.356786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc415336-deb8-49c4-9957-f88bdd43d3ea None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.369941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.370804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc415336-deb8-49c4-9957-f88bdd43d3ea None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.376371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.377154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.377889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.378147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.378803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:58:49.379103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.382024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.382524 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1756/3517] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:58:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:58:49.389722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.391683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.408772 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1762/3518] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:58:49.414521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.427815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c2013-cb6f-4d51-b6e6-d7faad5c9f10 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.434765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.435373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.435651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.435955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.436584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.439640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.476743 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1763/3519] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:58:49 2022] POST /identity/v3/projects => generated 399 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:49.481856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.504001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fb48c-c494-436b-9c54-35c44975a202 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.566289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:49.574268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5c2e9-a278-4fdb-abd0-10d25d35f6f2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.581313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.581985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.582397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.582685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.583515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.585592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.590680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b31a0-875f-454c-9e2e-6c0dc3afe260 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.591331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b31a0-875f-454c-9e2e-6c0dc3afe260 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.591722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b31a0-875f-454c-9e2e-6c0dc3afe260 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.592064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b31a0-875f-454c-9e2e-6c0dc3afe260 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.609757 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1764/3520] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:58:49 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:58:49.615668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.625762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68de8a-1824-4f7e-ae8f-ffbfc44cd100 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.631032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.631655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.631934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.632209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.633247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.635283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.650950 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1765/3521] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:49.652179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.667137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423c118-bdd7-4bc0-9fa2-3c7fdf0fb6d2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.672388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.673085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.673467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.673832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.674698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.676561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.681192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b31a0-875f-454c-9e2e-6c0dc3afe260 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.687468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b31a0-875f-454c-9e2e-6c0dc3afe260 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.705181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1766/3522] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:49.706223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.715205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1757/3523] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:49.718017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76763bd-9920-4015-b9c8-8b281d97c2a5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:49.739238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715b007-c9c5-44e3-8850-1c4065d8aeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:49.740175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b007-c9c5-44e3-8850-1c4065d8aeaa admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.740988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b007-c9c5-44e3-8850-1c4065d8aeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.741263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b007-c9c5-44e3-8850-1c4065d8aeaa admin admin] PATH_INFO: `/v3/projects/cebec33205424ecf92b54428bdf5eb61/users/ae0c7612d933424597e1f176be189a0b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.767871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715b007-c9c5-44e3-8850-1c4065d8aeaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebec33205424ecf92b54428bdf5eb61, user_id=ae0c7612d933424597e1f176be189a0b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:49.771896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715b007-c9c5-44e3-8850-1c4065d8aeaa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:49.790490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1767/3524] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:58:49 2022] PUT /identity/v3/projects/cebec33205424ecf92b54428bdf5eb61/users/ae0c7612d933424597e1f176be189a0b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:49.796490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.796784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.797085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.825308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:49.827313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.831585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29220e-34cf-4c98-85f0-998aac2acab6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:49.831872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29220e-34cf-4c98-85f0-998aac2acab6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:49.832185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29220e-34cf-4c98-85f0-998aac2acab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:49.868368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4f29220e-34cf-4c98-85f0-998aac2acab6 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:49.870495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4f29220e-34cf-4c98-85f0-998aac2acab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.891350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:49.904418 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1758/3525] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:49.916421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:49.918087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:49.960245 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1768/3526] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:50.033720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510c127-31e6-4967-ace2-794e4b21ef16 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:50.037346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34117f-3dc7-4977-ae79-328a09949b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:50.045089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34117f-3dc7-4977-ae79-328a09949b3c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:50.045089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34117f-3dc7-4977-ae79-328a09949b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:50.045089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34117f-3dc7-4977-ae79-328a09949b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:50.053606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34117f-3dc7-4977-ae79-328a09949b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:50.055089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34117f-3dc7-4977-ae79-328a09949b3c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:50.072005 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1759/3527] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:50.411160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29220e-34cf-4c98-85f0-998aac2acab6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:50.424218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29220e-34cf-4c98-85f0-998aac2acab6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:50.430121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:50.430943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:50.431198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:50.431589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] PATH_INFO: `/v3/users/a7df05f8c4f449b3b634b6bb2181e9ed` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:50.432841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7df05f8c4f449b3b634b6bb2181e9ed)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:50.434984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:50.500989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7df05f8c4f449b3b634b6bb2181e9ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:50.517101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7df05f8c4f449b3b634b6bb2181e9ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:50.527537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7df05f8c4f449b3b634b6bb2181e9ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:50.538032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7df05f8c4f449b3b634b6bb2181e9ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:50.539340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7df05f8c4f449b3b634b6bb2181e9ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:50.547364 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1769/3528] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:58:50 2022] DELETE /identity/v3/users/a7df05f8c4f449b3b634b6bb2181e9ed => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:50.552321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f4650-5028-4b7a-9f4e-bc9a8cd5ff8f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:50.552772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f4650-5028-4b7a-9f4e-bc9a8cd5ff8f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:50.553300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f4650-5028-4b7a-9f4e-bc9a8cd5ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:50.594265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-de9f4650-5028-4b7a-9f4e-bc9a8cd5ff8f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:50.595544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de9f4650-5028-4b7a-9f4e-bc9a8cd5ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:50.656570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:50.793385 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1760/3529] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:50.802594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:50.804214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:50.888045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671cdd-0590-46ee-938e-bf12cf584a40 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:50.893213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:50.894045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:50.894430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:50.894805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:50.912929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:50.914956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:50.926770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1770/3530] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:51.071426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:51.072420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:51.072951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:51.119449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:51.122602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:51.266406 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1761/3531] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:51.283700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:51.285675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:51.341119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:51.371947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6b295-9bdb-4931-925c-7ad67f110a71 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:51.378763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fc63e-bbcc-45bf-8b9e-212af8f2d8e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:51.379545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fc63e-bbcc-45bf-8b9e-212af8f2d8e6 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:51.380082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fc63e-bbcc-45bf-8b9e-212af8f2d8e6 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:51.380646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fc63e-bbcc-45bf-8b9e-212af8f2d8e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:51.394075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:51.396239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:51.408979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fc63e-bbcc-45bf-8b9e-212af8f2d8e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:51.411308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fc63e-bbcc-45bf-8b9e-212af8f2d8e6 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:51.425860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1771/3532] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:58:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:58:51.430709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b333-6cf6-4655-b6e2-0d0c49915781 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:51.436098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:51.436739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:51.437026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:51.437302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] PATH_INFO: `/v3/projects/9168d775bc9e4e3590b90d4625983a2f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:51.440325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9168d775bc9e4e3590b90d4625983a2f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:51.442715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:51.505464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9168d775bc9e4e3590b90d4625983a2f', 'request_id': 'req-3958f220-d592-4935-bf62-e9e061895d28'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:51.512486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9168d775bc9e4e3590b90d4625983a2f', 'request_id': 'req-3958f220-d592-4935-bf62-e9e061895d28'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:51.525480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9168d775bc9e4e3590b90d4625983a2f', 'request_id': 'req-3958f220-d592-4935-bf62-e9e061895d28'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:51.544247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] The token cache is being invalidate because project 9168d775bc9e4e3590b90d4625983a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:58:51.544624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:51.545386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3958f220-d592-4935-bf62-e9e061895d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:51.546987 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1762/3533] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:58:51 2022] DELETE /identity/v3/projects/9168d775bc9e4e3590b90d4625983a2f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:51.554797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769787ce-4e1e-410b-8fcc-32bb9d79a89d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:51.555202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769787ce-4e1e-410b-8fcc-32bb9d79a89d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:51.555396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769787ce-4e1e-410b-8fcc-32bb9d79a89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:51.581398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-769787ce-4e1e-410b-8fcc-32bb9d79a89d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:51.583725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-769787ce-4e1e-410b-8fcc-32bb9d79a89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:51.698141 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1772/3534] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:51.706236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f43fb-afa2-4305-94e5-3f985b66d07e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:51.706758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f43fb-afa2-4305-94e5-3f985b66d07e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:51.707280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f43fb-afa2-4305-94e5-3f985b66d07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:51.731018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-618f43fb-afa2-4305-94e5-3f985b66d07e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:51.732469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-618f43fb-afa2-4305-94e5-3f985b66d07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:51.741950 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1763/3535] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:51.747264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769787ce-4e1e-410b-8fcc-32bb9d79a89d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:51.768085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769787ce-4e1e-410b-8fcc-32bb9d79a89d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:51.784065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:51.785857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:51.786184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:51.787906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:51.788584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:58:51.790240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:51.793165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:51.811516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1773/3536] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:58:51.813182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f43fb-afa2-4305-94e5-3f985b66d07e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:51.827824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f43fb-afa2-4305-94e5-3f985b66d07e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:51.840702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:51.842466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:51.842946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:51.844716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:51.846111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:51.848392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:51.901592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1764/3537] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:58:51 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:51.903808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:51.916211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46004e5-d684-45b8-b2e6-1f1d0d483e86 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:52.031996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae39d3-d239-4e6b-8334-6e599fad3af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:52.032770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae39d3-d239-4e6b-8334-6e599fad3af7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.033015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae39d3-d239-4e6b-8334-6e599fad3af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.033241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae39d3-d239-4e6b-8334-6e599fad3af7 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.033796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae39d3-d239-4e6b-8334-6e599fad3af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:52.037099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae39d3-d239-4e6b-8334-6e599fad3af7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:52.066093 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1774/3538] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:58:51 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:52.071606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:52.081474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14e701-609d-40af-8d26-ef2566d507d6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.082074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14e701-609d-40af-8d26-ef2566d507d6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.082356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14e701-609d-40af-8d26-ef2566d507d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.085565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc96d0-e295-4477-9e94-56911fbfb9ac admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:52.094118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:52.095179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.095625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.096046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.097947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:52.101728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:52.106906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6c14e701-609d-40af-8d26-ef2566d507d6 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:52.108864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6c14e701-609d-40af-8d26-ef2566d507d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:52.121807 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1765/3539] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:52.122858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:52.139951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6f7bb-4ddf-4e1a-a498-b129b3e725b5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:52.147265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:52.148295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.148739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.149350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.150376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:52.152856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:52.170982 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1766/3540] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:58:52.175067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:52.192372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bfdc1-8c70-466b-8c21-f1b7e1e3f23e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:52.195021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:52.203781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:52.212412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.213095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.213473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] PATH_INFO: `/v3/projects/69499769ca8d4da0aa350ce7e7735842/users/438a1255ba0946edb1c3140b1fdcf913/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.219312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:52.225188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1775/3541] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:52.253103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69499769ca8d4da0aa350ce7e7735842, user_id=438a1255ba0946edb1c3140b1fdcf913, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:52.253103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:52.286568 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1767/3542] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:58:52 2022] PUT /identity/v3/projects/69499769ca8d4da0aa350ce7e7735842/users/438a1255ba0946edb1c3140b1fdcf913/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:52.289946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dedb6d-ac7e-45f4-a252-67e64caa4ec9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.290445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dedb6d-ac7e-45f4-a252-67e64caa4ec9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.291020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dedb6d-ac7e-45f4-a252-67e64caa4ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.338412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-30dedb6d-ac7e-45f4-a252-67e64caa4ec9 None None] MFA Rules not processed for user `438a1255ba0946edb1c3140b1fdcf913`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:52.339765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-30dedb6d-ac7e-45f4-a252-67e64caa4ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:52.481661 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1776/3543] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:52.566110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:52.569761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:52.677063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd76748-9a4f-4063-8806-bdddc50fe460 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:52.685137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552ce0d-77aa-4885-b63f-8f9cb69072e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:52.685694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ce0d-77aa-4885-b63f-8f9cb69072e0 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.686163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ce0d-77aa-4885-b63f-8f9cb69072e0 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.686445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ce0d-77aa-4885-b63f-8f9cb69072e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.697575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552ce0d-77aa-4885-b63f-8f9cb69072e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:52.699799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552ce0d-77aa-4885-b63f-8f9cb69072e0 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:52.711833 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1768/3544] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:52.916246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:52.916586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:52.916867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:52.951312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:52.952686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:53.000549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:53.001173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:53.001619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:53.036118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:53.038071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:53.110095 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1777/3545] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:53.167069 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1769/3546] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:58:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:53.186054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:53.191580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:53.461236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd7673-7805-4ac6-9d06-d03a21b1b840 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:53.490453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d771bb-1a97-45e1-aec9-899ea0dde080 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:53.491817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d771bb-1a97-45e1-aec9-899ea0dde080 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:53.492265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d771bb-1a97-45e1-aec9-899ea0dde080 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:53.492696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d771bb-1a97-45e1-aec9-899ea0dde080 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:53.530837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d771bb-1a97-45e1-aec9-899ea0dde080 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:53.532724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d771bb-1a97-45e1-aec9-899ea0dde080 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:53.558715 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1778/3547] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:58:53.769398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:53.770996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:53.832682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307c1f9-3bd6-4d5c-9df3-fec33809b305 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:53.854149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d8aed-38f2-4a62-9232-f52be399d676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:53.855347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8aed-38f2-4a62-9232-f52be399d676 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:53.855706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8aed-38f2-4a62-9232-f52be399d676 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:53.856054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d8aed-38f2-4a62-9232-f52be399d676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:53.858218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a78d8aed-38f2-4a62-9232-f52be399d676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:53.879150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d8aed-38f2-4a62-9232-f52be399d676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:53.882399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d8aed-38f2-4a62-9232-f52be399d676 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:53.897507 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1770/3548] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:54.250716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31759b1e-3834-4da3-9313-a381accf7134 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:54.251022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31759b1e-3834-4da3-9313-a381accf7134 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:54.251312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31759b1e-3834-4da3-9313-a381accf7134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:54.295614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-31759b1e-3834-4da3-9313-a381accf7134 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:54.298727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-31759b1e-3834-4da3-9313-a381accf7134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:54.322147 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1779/3549] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:54.400627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab480c-c594-4b35-a6fa-83893325e5cf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:54.400627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab480c-c594-4b35-a6fa-83893325e5cf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:54.400627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab480c-c594-4b35-a6fa-83893325e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:54.471976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-71ab480c-c594-4b35-a6fa-83893325e5cf None None] MFA Rules not processed for user `81ee4fa83ef649d0bdc273243cd717c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:54.478440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-71ab480c-c594-4b35-a6fa-83893325e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:54.623199 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1771/3550] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:55.214092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b1e01-a33c-460b-b1c1-6fee3a577e14 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:55.214630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b1e01-a33c-460b-b1c1-6fee3a577e14 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:55.214763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b1e01-a33c-460b-b1c1-6fee3a577e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:55.235529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c66b1e01-a33c-460b-b1c1-6fee3a577e14 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:55.237458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c66b1e01-a33c-460b-b1c1-6fee3a577e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:55.247124 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1780/3551] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:55.331418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6b40f-20fb-47a8-8676-566873dc050a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:55.332177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6b40f-20fb-47a8-8676-566873dc050a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:55.332613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6b40f-20fb-47a8-8676-566873dc050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:55.355707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:55.388418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e5c6b40f-20fb-47a8-8676-566873dc050a None None] MFA Rules not processed for user `81ee4fa83ef649d0bdc273243cd717c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:55.388418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e5c6b40f-20fb-47a8-8676-566873dc050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:55.408934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1772/3552] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:56.947042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611a2a5-6352-4ba9-be0a-7668444933e8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:56.947472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611a2a5-6352-4ba9-be0a-7668444933e8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:56.947752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611a2a5-6352-4ba9-be0a-7668444933e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:56.956292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:56.988652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9611a2a5-6352-4ba9-be0a-7668444933e8 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:56.992391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9611a2a5-6352-4ba9-be0a-7668444933e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:57.023054 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1781/3553] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:57.155220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6b40f-20fb-47a8-8676-566873dc050a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:57.298400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6b40f-20fb-47a8-8676-566873dc050a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:57.312318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:57.313084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:57.313355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:57.313730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] PATH_INFO: `/v3/users/438a1255ba0946edb1c3140b1fdcf913` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:57.314959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] RBAC: Authorizing `identity:delete_user(user_id=438a1255ba0946edb1c3140b1fdcf913)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:57.319111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:57.452071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438a1255ba0946edb1c3140b1fdcf913'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:57.462909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438a1255ba0946edb1c3140b1fdcf913'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:57.466407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438a1255ba0946edb1c3140b1fdcf913'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:57.490343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438a1255ba0946edb1c3140b1fdcf913'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:57.493284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438a1255ba0946edb1c3140b1fdcf913'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:57.541094 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1773/3554] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:58:57 2022] DELETE /identity/v3/users/438a1255ba0946edb1c3140b1fdcf913 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:57.542087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1556-01ec-4750-a9c4-129fdcaf6618 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:57.542447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1556-01ec-4750-a9c4-129fdcaf6618 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:57.542948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1556-01ec-4750-a9c4-129fdcaf6618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:57.594646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5b3b1556-01ec-4750-a9c4-129fdcaf6618 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:57.596386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5b3b1556-01ec-4750-a9c4-129fdcaf6618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:57.757855 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1782/3555] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:57.834332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:57.837132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:58.060320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3ddfd-ecd8-4696-8441-56bca528beff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:58.077863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:58.078910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:58.079405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:58.080923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:58.107064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:58.115337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:58.129129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1774/3556] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:58:58.505194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11928e44-50f6-49ce-bb7d-0efa267846db None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:58.505741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11928e44-50f6-49ce-bb7d-0efa267846db None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:58.505741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11928e44-50f6-49ce-bb7d-0efa267846db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:58.530621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-11928e44-50f6-49ce-bb7d-0efa267846db None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:58.531796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-11928e44-50f6-49ce-bb7d-0efa267846db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:58.617980 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1783/3557] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:59.224806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:59.226350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:59.242255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd773ee-b2c2-4ebe-bdab-73a3e67fa9cd service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:59.254352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:59.255905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:59.256268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:59.256607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] PATH_INFO: `/v3/projects/69499769ca8d4da0aa350ce7e7735842` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:59.262003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69499769ca8d4da0aa350ce7e7735842)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:59.262003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:59.306326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:58:59.378350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69499769ca8d4da0aa350ce7e7735842', 'request_id': 'req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:59.387148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69499769ca8d4da0aa350ce7e7735842', 'request_id': 'req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:59.400080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69499769ca8d4da0aa350ce7e7735842', 'request_id': 'req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:59.424859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] The token cache is being invalidate because project 69499769ca8d4da0aa350ce7e7735842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:58:59.425221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:59.426467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-890d3cf9-8985-49bc-9f39-1c8fd6c270a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:58:59.429528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1775/3558] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:58:59 2022] DELETE /identity/v3/projects/69499769ca8d4da0aa350ce7e7735842 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:58:59.438962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958b8af-b59e-442c-88e6-175c7f641c72 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:59.439738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958b8af-b59e-442c-88e6-175c7f641c72 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:59.440323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958b8af-b59e-442c-88e6-175c7f641c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:59.471486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a958b8af-b59e-442c-88e6-175c7f641c72 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:59.477424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a958b8af-b59e-442c-88e6-175c7f641c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:59.657974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e758301-5b07-4836-b198-12be6f455f8c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:59.658297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e758301-5b07-4836-b198-12be6f455f8c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:59.658532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e758301-5b07-4836-b198-12be6f455f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:59.706073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1e758301-5b07-4836-b198-12be6f455f8c None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:59.707653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e758301-5b07-4836-b198-12be6f455f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:59.720112 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1784/3559] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:59.724381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fd1d8-b3d3-4a53-bf26-0f7dbfc94db3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:59.724691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fd1d8-b3d3-4a53-bf26-0f7dbfc94db3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:59.724974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fd1d8-b3d3-4a53-bf26-0f7dbfc94db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:59.760885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f79fd1d8-b3d3-4a53-bf26-0f7dbfc94db3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:58:59.762571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f79fd1d8-b3d3-4a53-bf26-0f7dbfc94db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:58:59.779792 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1785/3560] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:58:59.781297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fd1d8-b3d3-4a53-bf26-0f7dbfc94db3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:59.792142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fd1d8-b3d3-4a53-bf26-0f7dbfc94db3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:59.799084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:59.800064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:59.800557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:59.801075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:59.802305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:58:59.802843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:59.805392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:59.836225 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1776/3561] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:58:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:58:59.836225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e758301-5b07-4836-b198-12be6f455f8c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:59.837439 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1786/3562] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:58:59.880848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e758301-5b07-4836-b198-12be6f455f8c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:58:59.896272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2624f37-5843-4c36-843f-912a6729aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:58:59.897219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624f37-5843-4c36-843f-912a6729aeeb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:58:59.897451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624f37-5843-4c36-843f-912a6729aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:58:59.897732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2624f37-5843-4c36-843f-912a6729aeeb admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:58:59.898414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2624f37-5843-4c36-843f-912a6729aeeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:58:59.900332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2624f37-5843-4c36-843f-912a6729aeeb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:58:59.941217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1777/3563] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:58:59 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:58:59.943023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:58:59.982051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87976c7-d3b2-479e-9c14-c69ff234f95e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:00.022983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:00.085783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:00.086517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.086871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.087216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.087919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:00.089930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:00.124093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75f6df-42fe-4472-89a2-81874f0d9857 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.124951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75f6df-42fe-4472-89a2-81874f0d9857 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.125850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75f6df-42fe-4472-89a2-81874f0d9857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.182825 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1787/3564] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:58:59 2022] POST /identity/v3/users => generated 291 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:00.190032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:00.243804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59cbe-3c81-4982-9139-f9c688f9b54e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:00.250176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5f75f6df-42fe-4472-89a2-81874f0d9857 None None] MFA Rules not processed for user `ae0c7612d933424597e1f176be189a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:00.252520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f75f6df-42fe-4472-89a2-81874f0d9857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:00.261792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:00.262543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.262901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.263242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.264208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:00.267404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:00.291614 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1788/3565] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:00.294882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:00.307987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb5016-44b2-42a8-8ad4-558d9a42d909 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:00.314954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:00.316200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.316584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.316948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.318141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:00.321028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:00.339339 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1789/3566] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:00.343034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:00.357470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c73ba-76d1-4c07-9174-4cf34f734183 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:00.363587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4136cca-54f4-4e30-b9c9-613e9b4fdc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:00.364483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136cca-54f4-4e30-b9c9-613e9b4fdc01 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.365114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136cca-54f4-4e30-b9c9-613e9b4fdc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.365674 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1778/3567] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:00.365801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136cca-54f4-4e30-b9c9-613e9b4fdc01 admin admin] PATH_INFO: `/v3/projects/7289ce4dcfe9474ea286bcbe7ef179a1/users/1b5bc12930c946ccaffcc524795ad965/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.375559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.375852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.376198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.400425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4136cca-54f4-4e30-b9c9-613e9b4fdc01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7289ce4dcfe9474ea286bcbe7ef179a1, user_id=1b5bc12930c946ccaffcc524795ad965, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:00.417031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4136cca-54f4-4e30-b9c9-613e9b4fdc01 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:00.429703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] MFA Rules not processed for user `ae0c7612d933424597e1f176be189a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:00.432571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:00.445305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1790/3568] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:00 2022] PUT /identity/v3/projects/7289ce4dcfe9474ea286bcbe7ef179a1/users/1b5bc12930c946ccaffcc524795ad965/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:00.450684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.450967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.451392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.456604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1779/3569] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:00.484285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:00.487181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:00.602492 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1791/3570] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:00.679208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:00.681047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:00.705306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:00.787949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71745e-f641-48e6-b0cc-9e7a40a382c3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:00.846870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:00.849252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:00.935680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:00.936313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:00.936656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:00.937453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:00.951133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:00.952654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:00.972418 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1780/3571] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:01.034443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:01.043603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323d1f5-5a09-4c2a-991b-2a8d9ff1cf72 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:01.052986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:01.053720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:01.054195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:01.054548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:01.160278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:01.163066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:01.178436 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1792/3572] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:01.742599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:01.744073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:01.770081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787b6c-5f0b-4948-8768-b0668044a2bd service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:01.778138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bead66-722b-49f2-b3ee-5e8a86d3b14b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:01.778774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bead66-722b-49f2-b3ee-5e8a86d3b14b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:01.779266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bead66-722b-49f2-b3ee-5e8a86d3b14b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:01.779548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bead66-722b-49f2-b3ee-5e8a86d3b14b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:01.784514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e1bead66-722b-49f2-b3ee-5e8a86d3b14b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:01.817004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bead66-722b-49f2-b3ee-5e8a86d3b14b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:01.820189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bead66-722b-49f2-b3ee-5e8a86d3b14b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:01.834836 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1781/3573] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:02.920045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:02.923039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:03.122414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d57ae-d575-42ee-91d2-af3322aa472f service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:03.127688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec745a4c-db31-4e55-ad24-16dfd84740cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:03.128724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec745a4c-db31-4e55-ad24-16dfd84740cc service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:03.129427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec745a4c-db31-4e55-ad24-16dfd84740cc service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:03.130473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec745a4c-db31-4e55-ad24-16dfd84740cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:03.132923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ec745a4c-db31-4e55-ad24-16dfd84740cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:03.299239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec745a4c-db31-4e55-ad24-16dfd84740cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:03.303441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec745a4c-db31-4e55-ad24-16dfd84740cc service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:03.322278 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1793/3574] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:03.843192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:03.844514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:03.850310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:03.888097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:03.891775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:04.050441 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1782/3575] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:04.795606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6994e-8b24-4c1c-94de-7fe1af8b8af6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:04.796110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6994e-8b24-4c1c-94de-7fe1af8b8af6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:04.796205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6994e-8b24-4c1c-94de-7fe1af8b8af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:04.859593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9af6994e-8b24-4c1c-94de-7fe1af8b8af6 None None] MFA Rules not processed for user `f38321284ec449e599d645625ed11139`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:04.860831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9af6994e-8b24-4c1c-94de-7fe1af8b8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:04.935988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:05.000188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1794/3576] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:05.624104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:05.626751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:05.654508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d40eb0-ead4-4eab-ae75-c949eb6e3c50 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:05.673472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:05.674459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:05.675016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:05.676072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] PATH_INFO: `/v3/users/4c3e0565f75c4f558a8936f547ae9d76` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:05.683213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3e0565f75c4f558a8936f547ae9d76)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:05.686175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:05.796353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3e0565f75c4f558a8936f547ae9d76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:05.815270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3e0565f75c4f558a8936f547ae9d76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:05.816286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3e0565f75c4f558a8936f547ae9d76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:05.819172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:05.843057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3e0565f75c4f558a8936f547ae9d76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:05.844133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc8ed032-b07f-4d3b-9127-e9a8b5ff411f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3e0565f75c4f558a8936f547ae9d76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:05.894685 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1783/3577] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:59:05 2022] DELETE /identity/v3/users/4c3e0565f75c4f558a8936f547ae9d76 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:07.059602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6994e-8b24-4c1c-94de-7fe1af8b8af6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:07.077925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6994e-8b24-4c1c-94de-7fe1af8b8af6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:07.264854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:07.265608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:07.265914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:07.266197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] PATH_INFO: `/v3/projects/c14add960abe42f580aeeade3e6f27d9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:07.269239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14add960abe42f580aeeade3e6f27d9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:07.272147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:07.454583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14add960abe42f580aeeade3e6f27d9', 'request_id': 'req-2e1bdf29-0cba-470c-9abe-54eae9462bce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:07.474129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14add960abe42f580aeeade3e6f27d9', 'request_id': 'req-2e1bdf29-0cba-470c-9abe-54eae9462bce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:07.489919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14add960abe42f580aeeade3e6f27d9', 'request_id': 'req-2e1bdf29-0cba-470c-9abe-54eae9462bce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:07.526813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] The token cache is being invalidate because project c14add960abe42f580aeeade3e6f27d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:59:07.527111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:07.528315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1bdf29-0cba-470c-9abe-54eae9462bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:07.535404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1795/3578] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:59:07 2022] DELETE /identity/v3/projects/c14add960abe42f580aeeade3e6f27d9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:07.537915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138dbd-46ca-4e1d-a9bb-9f7b138d5df7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:07.538728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138dbd-46ca-4e1d-a9bb-9f7b138d5df7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:07.539230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138dbd-46ca-4e1d-a9bb-9f7b138d5df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:07.583884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2b138dbd-46ca-4e1d-a9bb-9f7b138d5df7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:07.585706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b138dbd-46ca-4e1d-a9bb-9f7b138d5df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:07.719592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1784/3579] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:07.724410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0677-0d07-4cfa-88b4-3d80f66e28ea None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:07.724943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0677-0d07-4cfa-88b4-3d80f66e28ea None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:07.725380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0677-0d07-4cfa-88b4-3d80f66e28ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:07.781298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-566d0677-0d07-4cfa-88b4-3d80f66e28ea None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:07.786640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-566d0677-0d07-4cfa-88b4-3d80f66e28ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:07.806338 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1796/3580] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:07.810828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b138dbd-46ca-4e1d-a9bb-9f7b138d5df7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:07.826276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b138dbd-46ca-4e1d-a9bb-9f7b138d5df7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:07.838540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:07.839217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:07.839508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:07.839786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:07.841990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:07.842335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:07.845324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:07.870239 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1785/3581] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:07.870954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d0677-0d07-4cfa-88b4-3d80f66e28ea None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:07.881344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d0677-0d07-4cfa-88b4-3d80f66e28ea None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:07.887628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:07.888447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:07.889054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:07.889543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:07.890401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:07.899392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:07.970308 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1797/3582] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:07 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:07.972160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:07.993421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707458e9-7515-45a8-8fb2-4ae88e79cc2d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:08.091829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:08.092874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:08.093288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:08.093818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:08.094789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:08.097218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:08.136212 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1786/3583] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:07 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:08.138919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:08.154373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98de33-dff0-4541-a2af-e24e132b111d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:08.160356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:08.162187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:08.162720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:08.163641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:08.165001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:08.167722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:08.199371 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1798/3584] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:08.203359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:08.219338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d303eb1-3442-444d-adfd-e7fe808b41e8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:08.231690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20faadb-518d-46d9-a4c6-b36894be9817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:08.232548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20faadb-518d-46d9-a4c6-b36894be9817 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:08.232679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20faadb-518d-46d9-a4c6-b36894be9817 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:08.232891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20faadb-518d-46d9-a4c6-b36894be9817 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:08.235471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20faadb-518d-46d9-a4c6-b36894be9817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:08.237388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20faadb-518d-46d9-a4c6-b36894be9817 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:08.271296 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1787/3585] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:08.272436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:08.284026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733658-4c92-44a5-9778-10b4c1e6d36b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:08.292761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:08.293649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:08.293973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:08.294418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] PATH_INFO: `/v3/projects/329b758edee649078291d9cb0eda3feb/users/fd0c6e0e7b1a44c69ec9f4685ee39398/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:08.322081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] RBAC: Authorizing `identity:create_grant(project_id=329b758edee649078291d9cb0eda3feb, user_id=fd0c6e0e7b1a44c69ec9f4685ee39398, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:08.326833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:08.354968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1799/3586] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:08 2022] PUT /identity/v3/projects/329b758edee649078291d9cb0eda3feb/users/fd0c6e0e7b1a44c69ec9f4685ee39398/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:08.359714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f547-9869-4392-9b3e-621ad3f824ff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:08.360098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f547-9869-4392-9b3e-621ad3f824ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:08.360544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f547-9869-4392-9b3e-621ad3f824ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:08.393636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7583f547-9869-4392-9b3e-621ad3f824ff None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:08.394625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7583f547-9869-4392-9b3e-621ad3f824ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:08.475078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:08.553932 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1788/3587] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:08.618642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:08.620888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:08.643394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:08.758919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533ae1a-7429-4b1a-8154-6cca85cd751a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:08.767158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5723209-648f-4f52-ae35-e9344c2a8203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:08.767838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723209-648f-4f52-ae35-e9344c2a8203 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:08.768121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723209-648f-4f52-ae35-e9344c2a8203 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:08.768395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723209-648f-4f52-ae35-e9344c2a8203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:08.792019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5723209-648f-4f52-ae35-e9344c2a8203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:08.795337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5723209-648f-4f52-ae35-e9344c2a8203 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:08.809633 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1800/3588] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:15.585082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:15.586190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:15.586797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:15.629199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:15.633496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:15.739627 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1789/3589] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:18.304111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8432-8b8d-473b-85fb-141464be1704 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:18.305441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8432-8b8d-473b-85fb-141464be1704 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:18.305441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8432-8b8d-473b-85fb-141464be1704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:18.388378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5e8d8432-8b8d-473b-85fb-141464be1704 None None] MFA Rules not processed for user `1b5bc12930c946ccaffcc524795ad965`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:18.391139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5e8d8432-8b8d-473b-85fb-141464be1704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:18.621656 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1801/3590] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:19.135193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:19.136716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:19.154715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e86b-0ae6-4d5b-8616-773c4f1c45ab None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:19.155281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e86b-0ae6-4d5b-8616-773c4f1c45ab None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:19.155735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6e86b-0ae6-4d5b-8616-773c4f1c45ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:19.197906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-38d6e86b-0ae6-4d5b-8616-773c4f1c45ab None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:19.203213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-38d6e86b-0ae6-4d5b-8616-773c4f1c45ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:19.237337 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1802/3591] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:19.335287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98436f-e33d-4d0c-9617-5eba1a45eb64 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:19.354168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8f22a-33b5-4ffd-8c32-786056cce507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:19.354870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f22a-33b5-4ffd-8c32-786056cce507 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:19.355161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f22a-33b5-4ffd-8c32-786056cce507 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:19.355436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f22a-33b5-4ffd-8c32-786056cce507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:19.387832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8f22a-33b5-4ffd-8c32-786056cce507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:19.389488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8f22a-33b5-4ffd-8c32-786056cce507 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:19.422343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1790/3592] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:20.022477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:20.023033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:20.023489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:20.078059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] MFA Rules not processed for user `fd0c6e0e7b1a44c69ec9f4685ee39398`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:20.079516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:20.224055 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1803/3593] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:20.237118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361626b4-adc0-4882-8306-181b80b8ccc9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:20.237600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361626b4-adc0-4882-8306-181b80b8ccc9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:20.240007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361626b4-adc0-4882-8306-181b80b8ccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:20.283635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-361626b4-adc0-4882-8306-181b80b8ccc9 None None] MFA Rules not processed for user `fd0c6e0e7b1a44c69ec9f4685ee39398`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:20.287037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-361626b4-adc0-4882-8306-181b80b8ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:20.297526 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1791/3594] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:20.650061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:20.651769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:20.680678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc4150-7c1e-4b37-85eb-caf708d8cf48 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:20.689954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:20.690932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:20.691372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:20.692794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:20.705442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:20.708235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:20.738728 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1804/3595] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:20.778288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154f974-e8f5-4f08-9ba5-db81b561c8f0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:20.778573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154f974-e8f5-4f08-9ba5-db81b561c8f0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:20.778846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154f974-e8f5-4f08-9ba5-db81b561c8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:20.802055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3154f974-e8f5-4f08-9ba5-db81b561c8f0 None None] MFA Rules not processed for user `1b5bc12930c946ccaffcc524795ad965`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:20.803300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3154f974-e8f5-4f08-9ba5-db81b561c8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:20.812428 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1792/3596] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:21.829431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:21.849673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b28f0-5eb4-4147-88ea-dd051b6a8835 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:21.858063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:21.859112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:21.859557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:21.859993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:21.864392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:21.888572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:21.890400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:21.902029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:21.917587 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1805/3597] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 20:59:28.068608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5acb67-31eb-4121-a5b5-6ac0216c9853 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:28.069178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5acb67-31eb-4121-a5b5-6ac0216c9853 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:28.069178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5acb67-31eb-4121-a5b5-6ac0216c9853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:28.092292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bc5acb67-31eb-4121-a5b5-6ac0216c9853 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:28.093968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bc5acb67-31eb-4121-a5b5-6ac0216c9853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:28.164112 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1793/3598] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:28.178771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:28.180812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:28.267170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8b80-b2e8-4313-8ac2-8787d80ab9fd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:28.273347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:28.274158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:28.274560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:28.275100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:28.290724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:28.292507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:28.302387 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1806/3599] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:59:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:59:36.417454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:36.418566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:36.419067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:36.431575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:36.460624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] MFA Rules not processed for user `f38321284ec449e599d645625ed11139`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:36.462930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:36.621875 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1794/3600] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:38.298292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=33 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:41.763700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:41.780719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc1fdd-1b7b-40c7-b4f5-23be18feb696 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:41.784661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:41.785475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:41.786766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:41.787086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:41.790065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:41.813458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:41.815596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:41.853238 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1807/3601] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:42.606857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:42.608423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:42.660677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4f643-9fc3-4317-b26a-ff9bda7aefa4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:42.678281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:42.679127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:42.679573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:42.680005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] PATH_INFO: `/v3/users/91d41f09b29b4f20816318805722b2e4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:42.681774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d41f09b29b4f20816318805722b2e4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:42.684384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:42.864314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d41f09b29b4f20816318805722b2e4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:42.876119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d41f09b29b4f20816318805722b2e4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:42.877532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d41f09b29b4f20816318805722b2e4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:42.897882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d41f09b29b4f20816318805722b2e4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:42.898541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b78617b-4f28-4f41-91de-e8b1dfe8da50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d41f09b29b4f20816318805722b2e4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:42.935503 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1795/3602] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:59:42 2022] DELETE /identity/v3/users/91d41f09b29b4f20816318805722b2e4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:43.524026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:43.548349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dfc55-63ec-475a-a4e2-fbbd9d642534 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:43.782892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:43.783801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:43.784091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:43.784366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] PATH_INFO: `/v3/projects/38b49720bd644bed9a3504ff26f01768` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:43.797603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b49720bd644bed9a3504ff26f01768)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:43.799890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae5a5-f768-4816-b3c6-c8074723d64a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:43.800903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae5a5-f768-4816-b3c6-c8074723d64a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:43.801236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae5a5-f768-4816-b3c6-c8074723d64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:43.802510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:43.842100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6ffae5a5-f768-4816-b3c6-c8074723d64a None None] MFA Rules not processed for user `2bed1a15bbff41c6928afd3c057e7dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:43.843737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6ffae5a5-f768-4816-b3c6-c8074723d64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:43.986369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b49720bd644bed9a3504ff26f01768', 'request_id': 'req-d73cae89-b6dc-4219-8ba7-186025c5083d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:44.000413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b49720bd644bed9a3504ff26f01768', 'request_id': 'req-d73cae89-b6dc-4219-8ba7-186025c5083d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:44.011940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b49720bd644bed9a3504ff26f01768', 'request_id': 'req-d73cae89-b6dc-4219-8ba7-186025c5083d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:44.029888 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1796/3603] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:44.042259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] The token cache is being invalidate because project 38b49720bd644bed9a3504ff26f01768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:59:44.042567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:44.044676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d73cae89-b6dc-4219-8ba7-186025c5083d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:44.052636 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1808/3604] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:59:43 2022] DELETE /identity/v3/projects/38b49720bd644bed9a3504ff26f01768 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:44.056465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816877-127c-4239-80f6-681fbebf5dd2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.056765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816877-127c-4239-80f6-681fbebf5dd2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.057065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed816877-127c-4239-80f6-681fbebf5dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.095145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ed816877-127c-4239-80f6-681fbebf5dd2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.097626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed816877-127c-4239-80f6-681fbebf5dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:44.133929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:44.202991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1797/3605] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:44.211857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c2c7-ce56-43e6-8ee4-684b20462514 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.212213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c2c7-ce56-43e6-8ee4-684b20462514 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.212420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c2c7-ce56-43e6-8ee4-684b20462514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.244315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3535c2c7-ce56-43e6-8ee4-684b20462514 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.246045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3535c2c7-ce56-43e6-8ee4-684b20462514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:44.264639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1809/3606] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:44.266015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816877-127c-4239-80f6-681fbebf5dd2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.281945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed816877-127c-4239-80f6-681fbebf5dd2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.307716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.308468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.309946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.311007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.311819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:44.312228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.314136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.339156 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1798/3607] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:44.344596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c2c7-ce56-43e6-8ee4-684b20462514 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.356015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c2c7-ce56-43e6-8ee4-684b20462514 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.361373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.362028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.362311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.363687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.364388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.366594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.437595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.438147 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1810/3608] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:44.452782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc0a97-abd6-407c-adeb-adffa901830d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.618201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.618852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.619133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.619470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.620116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.622118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.670200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1799/3609] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:59:44.673909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.696649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c63679-a3a4-4e9e-af25-4b21adb0e698 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.705820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.708466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.708958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.709338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.710357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.712722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.759726 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1811/3610] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:44.762314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.776347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d4d34-d5d4-48c5-a8fd-f2350a6f97bc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.790355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bffc3-faad-40fe-9ee4-3d2b67ad923c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.791036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bffc3-faad-40fe-9ee4-3d2b67ad923c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.791322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bffc3-faad-40fe-9ee4-3d2b67ad923c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.791683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bffc3-faad-40fe-9ee4-3d2b67ad923c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.792355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bffc3-faad-40fe-9ee4-3d2b67ad923c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.796493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bffc3-faad-40fe-9ee4-3d2b67ad923c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.826484 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1800/3611] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:44.828067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:44.840268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002a19-6bfe-4a8b-ae27-aadb1194be37 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:44.846136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:44.846810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.848186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.848438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] PATH_INFO: `/v3/projects/d696ed7cdc3945b78d638693a83ac081/users/d57829f24d3841eb9c1d6f5c6a5ea7ee/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.867444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d696ed7cdc3945b78d638693a83ac081, user_id=d57829f24d3841eb9c1d6f5c6a5ea7ee, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:44.870863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:44.906717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44924b-cc55-4196-ba38-630247930c38 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:44.907021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44924b-cc55-4196-ba38-630247930c38 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:44.907276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44924b-cc55-4196-ba38-630247930c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:44.910610 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1812/3612] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:44 2022] PUT /identity/v3/projects/d696ed7cdc3945b78d638693a83ac081/users/d57829f24d3841eb9c1d6f5c6a5ea7ee/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:44.960419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-db44924b-cc55-4196-ba38-630247930c38 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:44.983657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-db44924b-cc55-4196-ba38-630247930c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.080229 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1801/3613] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:45.149334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:45.149334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:45.182858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:45.264573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2dcc2-5d9e-431d-8a83-17b47f8cf117 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:45.268782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:45.269982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:45.270270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:45.270546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:45.294216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:45.295701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:45.312713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1813/3614] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:50.652078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44924b-cc55-4196-ba38-630247930c38 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:50.654465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-db44924b-cc55-4196-ba38-630247930c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:50.676681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44924b-cc55-4196-ba38-630247930c38 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:50.687919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:50.689028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:50.689435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:50.692353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] PATH_INFO: `/v3/users/d8b0a160808f4adda7be57cd535f0631` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:50.694480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b0a160808f4adda7be57cd535f0631)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:50.696533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:50.754581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:50.801791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b0a160808f4adda7be57cd535f0631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:50.814562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b0a160808f4adda7be57cd535f0631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:50.816338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b0a160808f4adda7be57cd535f0631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:50.850794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b0a160808f4adda7be57cd535f0631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:50.854066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c5aaee2-3c6d-43e1-b396-691ad7221ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b0a160808f4adda7be57cd535f0631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:50.898828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1802/3615] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 20:59:50 2022] DELETE /identity/v3/users/d8b0a160808f4adda7be57cd535f0631 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:51.321866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:51.337197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac5bea-596f-4246-8a49-e6c2889d6f92 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:51.449116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:51.449989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.450376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.450759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] PATH_INFO: `/v3/projects/f4b1732129f3414fbc3f26fff6d039ec` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.453176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b1732129f3414fbc3f26fff6d039ec)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:51.455285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:51.532992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b1732129f3414fbc3f26fff6d039ec', 'request_id': 'req-071acb0e-8ab2-4932-8679-5f471b3891ec'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.539536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b1732129f3414fbc3f26fff6d039ec', 'request_id': 'req-071acb0e-8ab2-4932-8679-5f471b3891ec'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.563047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b1732129f3414fbc3f26fff6d039ec', 'request_id': 'req-071acb0e-8ab2-4932-8679-5f471b3891ec'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.590048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] The token cache is being invalidate because project f4b1732129f3414fbc3f26fff6d039ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 20:59:51.590600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.592193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-071acb0e-8ab2-4932-8679-5f471b3891ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 20:59:51.594174 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1814/3616] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 20:59:51 2022] DELETE /identity/v3/projects/f4b1732129f3414fbc3f26fff6d039ec => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:51.600146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332c9c9-db16-4e7b-b03f-bf96f83eb973 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.600785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332c9c9-db16-4e7b-b03f-bf96f83eb973 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.601232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332c9c9-db16-4e7b-b03f-bf96f83eb973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.645142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b332c9c9-db16-4e7b-b03f-bf96f83eb973 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:51.647670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b332c9c9-db16-4e7b-b03f-bf96f83eb973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:51.824088 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1803/3617] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:51.831765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cabc6-d1b8-4dca-a284-42f2508e7f6f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.834401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cabc6-d1b8-4dca-a284-42f2508e7f6f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.835242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cabc6-d1b8-4dca-a284-42f2508e7f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.864501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fe3cabc6-d1b8-4dca-a284-42f2508e7f6f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:51.866529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fe3cabc6-d1b8-4dca-a284-42f2508e7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:51.882217 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1815/3618] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:51.883013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332c9c9-db16-4e7b-b03f-bf96f83eb973 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:51.899189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332c9c9-db16-4e7b-b03f-bf96f83eb973 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:51.905844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:51.907006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.907388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.907763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.908521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 20:59:51.908923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:51.911400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:51.931241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1804/3619] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 20:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 20:59:51.931972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cabc6-d1b8-4dca-a284-42f2508e7f6f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:51.957456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cabc6-d1b8-4dca-a284-42f2508e7f6f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:51.963819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:51.964469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:51.964758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:51.965035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:51.965741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:51.968171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.011923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1816/3620] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:51 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:59:52.017254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.033446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732fcc9-5831-48a7-9a8e-3af1d137cd38 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.159970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.160750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.161246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.162344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.163076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.166925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.209198 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1805/3621] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:52 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:52.210008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.226547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b099a-8a82-40ad-b6b2-be34a0fc401e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.255420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.256210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.256781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.260077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.260077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.275077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.289206 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1817/3622] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:52.293811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.314647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516fbf-8637-43f5-a373-11d7d1b700b3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.336503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ebc9-8133-46ee-b8b5-629bbc4c390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.337123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ebc9-8133-46ee-b8b5-629bbc4c390b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.337408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ebc9-8133-46ee-b8b5-629bbc4c390b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.337683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ebc9-8133-46ee-b8b5-629bbc4c390b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.338433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ebc9-8133-46ee-b8b5-629bbc4c390b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.340020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ebc9-8133-46ee-b8b5-629bbc4c390b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.356389 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1806/3623] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:52.360326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.372099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc1fd5-f7e6-4a35-ac35-e0ffa75365ec admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.377228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.378646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.378937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.379267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] PATH_INFO: `/v3/projects/594351b0dfd749ba84bbd061f16b76dd/users/f71dcabaa905490ea6c869426d2f64ee/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.382078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:52.401178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594351b0dfd749ba84bbd061f16b76dd, user_id=f71dcabaa905490ea6c869426d2f64ee, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.405083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.424871 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1818/3624] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:52 2022] PUT /identity/v3/projects/594351b0dfd749ba84bbd061f16b76dd/users/f71dcabaa905490ea6c869426d2f64ee/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:52.430354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a208800-2419-4a4b-9b1b-13183b523141 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.430740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a208800-2419-4a4b-9b1b-13183b523141 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.431113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a208800-2419-4a4b-9b1b-13183b523141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.454081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5a208800-2419-4a4b-9b1b-13183b523141 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:52.455468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a208800-2419-4a4b-9b1b-13183b523141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:52.460900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:52.541905 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1807/3625] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 20:59:52.576352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:52.578218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:52.662605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dadbe-b377-46ee-bee3-93f7ae2ae044 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:52.667307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:52.668775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:52.669293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:52.669584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:52.683066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:52.684920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:52.697819 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1819/3626] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 20:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 20:59:55.089558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10a13-c04b-4447-ab40-9fc69905ff6f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:55.091688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10a13-c04b-4447-ab40-9fc69905ff6f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:55.091983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10a13-c04b-4447-ab40-9fc69905ff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:55.135910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-19f10a13-c04b-4447-ab40-9fc69905ff6f None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:55.141033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-19f10a13-c04b-4447-ab40-9fc69905ff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:55.266113 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1808/3627] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 20:59:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:55.295957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:55.304206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:55.497524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb329658-19b4-401e-8a8b-2921c2f562b9 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:55.509120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:55.510011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:55.510463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:55.510931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:55.520472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:55.522786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:55.532898 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1820/3628] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 20:59:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 20:59:57.101618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78858-bcff-4370-ac89-813ba242407c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.102266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78858-bcff-4370-ac89-813ba242407c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.102266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78858-bcff-4370-ac89-813ba242407c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.213552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-65d78858-bcff-4370-ac89-813ba242407c None None] MFA Rules not processed for user `d57829f24d3841eb9c1d6f5c6a5ea7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 20:59:57.215129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-65d78858-bcff-4370-ac89-813ba242407c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:57.392951 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1809/3629] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 20:59:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 20:59:57.395121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.397510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 20:59:57.442549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e3edf-9043-466a-813a-82c40f5c190d service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.448893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.449745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.450250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.450694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.451480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.453632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.526100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d78858-bcff-4370-ac89-813ba242407c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.530314 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1821/3630] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 20:59:57 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 20:59:57.551827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d78858-bcff-4370-ac89-813ba242407c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.668481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:57.726625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.727280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.728829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.729115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.730852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.735267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.823639 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1810/3631] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 20:59:57 2022] POST /identity/v3/users => generated 295 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 20:59:57.826274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.841385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf85ac-54f4-4b1e-b722-be6ea884cab7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.855575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.856375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.856666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.856946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.857603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.859302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:57.900026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:57.914273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186a9a-f66b-4f79-b12c-d9a8d88f2722 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:57.915033 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1822/3632] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:57.929756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:57.930469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:57.930756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:57.931033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] PATH_INFO: `/v3/projects/983d065238544c7ea99324500c083b1a/users/a94bd9fd73484ac8932017748d00896b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:57.974285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=983d065238544c7ea99324500c083b1a, user_id=a94bd9fd73484ac8932017748d00896b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:57.983224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:58.010301 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1811/3633] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:57 2022] PUT /identity/v3/projects/983d065238544c7ea99324500c083b1a/users/a94bd9fd73484ac8932017748d00896b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 20:59:58.015255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:58.032437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff63e1-3d72-421f-89cb-e1d6b81a289c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:58.148038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 20:59:58.248589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aaeca1-6f43-4ca9-8eb3-78299666b2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:58.249120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aaeca1-6f43-4ca9-8eb3-78299666b2dd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:58.249407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aaeca1-6f43-4ca9-8eb3-78299666b2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:58.249667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aaeca1-6f43-4ca9-8eb3-78299666b2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:58.252170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aaeca1-6f43-4ca9-8eb3-78299666b2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:58.255020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aaeca1-6f43-4ca9-8eb3-78299666b2dd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:58.286858 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1823/3634] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 20:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 20:59:58.291492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 20:59:58.313042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f06355-9b4b-4d51-9897-2c80b219a82b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 20:59:58.343943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941c262-a8c4-4d89-b703-67a079dd7442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 20:59:58.343943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941c262-a8c4-4d89-b703-67a079dd7442 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 20:59:58.343943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941c262-a8c4-4d89-b703-67a079dd7442 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 20:59:58.343943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941c262-a8c4-4d89-b703-67a079dd7442 admin admin] PATH_INFO: `/v3/projects/983d065238544c7ea99324500c083b1a/users/a94bd9fd73484ac8932017748d00896b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 20:59:58.354849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941c262-a8c4-4d89-b703-67a079dd7442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983d065238544c7ea99324500c083b1a, user_id=a94bd9fd73484ac8932017748d00896b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 20:59:58.366020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941c262-a8c4-4d89-b703-67a079dd7442 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 20:59:58.424092 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1812/3635] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 20:59:58 2022] PUT /identity/v3/projects/983d065238544c7ea99324500c083b1a/users/a94bd9fd73484ac8932017748d00896b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:08.271016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:08.393902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=24,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:09.499205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34affb-cb1a-4c61-be22-e042272aef81 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:09.500078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34affb-cb1a-4c61-be22-e042272aef81 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:09.500540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34affb-cb1a-4c61-be22-e042272aef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:09.525986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9c34affb-cb1a-4c61-be22-e042272aef81 None None] MFA Rules not processed for user `f38321284ec449e599d645625ed11139`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:09.541147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9c34affb-cb1a-4c61-be22-e042272aef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:09.686104 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1824/3636] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:12.008138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edb3a6-6a7d-4a08-a47e-550609c549aa None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:12.008766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edb3a6-6a7d-4a08-a47e-550609c549aa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:12.008941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edb3a6-6a7d-4a08-a47e-550609c549aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:12.058642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-13edb3a6-6a7d-4a08-a47e-550609c549aa None None] MFA Rules not processed for user `ae0c7612d933424597e1f176be189a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:12.060573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-13edb3a6-6a7d-4a08-a47e-550609c549aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:12.275265 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1813/3637] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:14.724798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89757008-833b-41db-8a53-29f8def7eae1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:14.725308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89757008-833b-41db-8a53-29f8def7eae1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:14.725765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89757008-833b-41db-8a53-29f8def7eae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:14.932274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-89757008-833b-41db-8a53-29f8def7eae1 None None] MFA Rules not processed for user `1b5bc12930c946ccaffcc524795ad965`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:14.934730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-89757008-833b-41db-8a53-29f8def7eae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:15.317294 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1825/3638] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 590 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:15.564497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880bc5-1f8e-4123-8a2a-858a732cba82 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:15.565656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880bc5-1f8e-4123-8a2a-858a732cba82 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:15.565922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880bc5-1f8e-4123-8a2a-858a732cba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:15.703549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ed880bc5-1f8e-4123-8a2a-858a732cba82 None None] MFA Rules not processed for user `f71dcabaa905490ea6c869426d2f64ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:15.724450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed880bc5-1f8e-4123-8a2a-858a732cba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:15.880787 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1814/3639] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:16.382560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88188222-6a0e-4dcc-b81b-e67e9cfd2a22 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.383109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88188222-6a0e-4dcc-b81b-e67e9cfd2a22 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.383371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88188222-6a0e-4dcc-b81b-e67e9cfd2a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.454213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-88188222-6a0e-4dcc-b81b-e67e9cfd2a22 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:16.466782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-88188222-6a0e-4dcc-b81b-e67e9cfd2a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:16.563360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:16.564102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:16.564605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:16.695649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] MFA Rules not processed for user `a94bd9fd73484ac8932017748d00896b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:16.700346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:16.796698 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1826/3640] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:16.809159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88188222-6a0e-4dcc-b81b-e67e9cfd2a22 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.008945 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1815/3641] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:17.018699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.020672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:17.045282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:17.103169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88188222-6a0e-4dcc-b81b-e67e9cfd2a22 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:17.107847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:17.122888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:17.123547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:17.123818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:17.124102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:17.144377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:17.146997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:17.169983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1827/3642] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:00:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 21:00:17.184337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.187067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:17.239260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558b835-3dc1-4b1e-a678-fbe88e9a5973 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:17.254617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:17.256098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:17.258144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:17.258536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:17.442188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:17.446106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:17.487187 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1816/3643] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:17.506166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa46fb3-c337-4784-abf5-d5c7166b08a2 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:17.541231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:17.543744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:17.544647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:17.546581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:17.556343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:17.610829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:17.615576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:17.691120 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1828/3644] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:17.717344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:17.719812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:18.116556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacbb29-cb81-4aec-8a0a-104750324b30 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:18.135965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:18.136960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:18.138311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:18.138599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] PATH_INFO: `/v3/users/fd0c6e0e7b1a44c69ec9f4685ee39398` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:18.147437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0c6e0e7b1a44c69ec9f4685ee39398)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:18.161662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:18.322615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0c6e0e7b1a44c69ec9f4685ee39398'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:18.360068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0c6e0e7b1a44c69ec9f4685ee39398'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:18.364106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0c6e0e7b1a44c69ec9f4685ee39398'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:18.399819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0c6e0e7b1a44c69ec9f4685ee39398'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:18.402984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8b06038e-0c91-4b24-9f8e-ebd7f735283b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0c6e0e7b1a44c69ec9f4685ee39398'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:18.428544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:18.451241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1817/3645] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:17 2022] DELETE /identity/v3/users/fd0c6e0e7b1a44c69ec9f4685ee39398 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:19.093160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:19.126114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d169ff1-8e5d-4618-b54d-c356c6ef76bb service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:19.306793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:19.307497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.307802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.309111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] PATH_INFO: `/v3/projects/329b758edee649078291d9cb0eda3feb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.314803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=329b758edee649078291d9cb0eda3feb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:19.316953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:19.435741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329b758edee649078291d9cb0eda3feb', 'request_id': 'req-4f642d9b-f4cc-47d0-92fc-070359505d9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.450411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329b758edee649078291d9cb0eda3feb', 'request_id': 'req-4f642d9b-f4cc-47d0-92fc-070359505d9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.460547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329b758edee649078291d9cb0eda3feb', 'request_id': 'req-4f642d9b-f4cc-47d0-92fc-070359505d9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.493380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] The token cache is being invalidate because project 329b758edee649078291d9cb0eda3feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:19.494016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.496004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4f642d9b-f4cc-47d0-92fc-070359505d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:19.509655 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1829/3646] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:19 2022] DELETE /identity/v3/projects/329b758edee649078291d9cb0eda3feb => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:19.527691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e4f82-0e27-4494-9da2-02cadef6b22a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.528364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e4f82-0e27-4494-9da2-02cadef6b22a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.528364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e4f82-0e27-4494-9da2-02cadef6b22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.606176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f39e4f82-0e27-4494-9da2-02cadef6b22a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:19.607464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f39e4f82-0e27-4494-9da2-02cadef6b22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:19.796276 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1818/3647] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:19.804200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518066-cc4c-4deb-9e08-c36c2b3b8716 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.804490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518066-cc4c-4deb-9e08-c36c2b3b8716 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.804781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a518066-cc4c-4deb-9e08-c36c2b3b8716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.830770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:19.871909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9a518066-cc4c-4deb-9e08-c36c2b3b8716 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:19.873167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9a518066-cc4c-4deb-9e08-c36c2b3b8716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:19.901669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1830/3648] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:19.902903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e4f82-0e27-4494-9da2-02cadef6b22a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:19.917408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e4f82-0e27-4494-9da2-02cadef6b22a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:19.924594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:19.925618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:19.926180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:19.928699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:19.929943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:19.930522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:19.933661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:19.980613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518066-cc4c-4deb-9e08-c36c2b3b8716 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:19.983292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1819/3649] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:00:19.998705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a518066-cc4c-4deb-9e08-c36c2b3b8716 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:20.027423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:20.028067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:20.028349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:20.028622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:20.029266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:20.031692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:20.131955 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1831/3650] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:19 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:20.132642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:20.155798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85b96f-91df-4747-8131-3dd6018cef26 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:20.367165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:20.367804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:20.368216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:20.368352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:20.368998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:20.371412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:20.450979 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1820/3651] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:20 2022] POST /identity/v3/users => generated 294 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:00:20.452689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:20.475518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab40d9-b4e6-4c46-9734-891b698107f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:20.485587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:20.486279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:20.487154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:20.487436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:20.488132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:20.491323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:20.518540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:20.518833 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1832/3652] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:20.548818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc3504-383c-41a8-a37e-e4b3668edf4a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:20.583235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8e225-7542-4ddf-a401-27c37faae202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:20.583871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e225-7542-4ddf-a401-27c37faae202 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:20.584154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e225-7542-4ddf-a401-27c37faae202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:20.584619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e225-7542-4ddf-a401-27c37faae202 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:20.585448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8e225-7542-4ddf-a401-27c37faae202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:20.588434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8e225-7542-4ddf-a401-27c37faae202 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:20.635098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1821/3653] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:20.636960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:20.660800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7dff4-6967-4b2e-9a6b-0c2e9f88c3ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:20.675551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:20.678613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:20.678613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:20.678613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] PATH_INFO: `/v3/projects/243cfbc91bb24e7ca96f348cd21f1adb/users/9b9b59fb1a184b958ea619062412cf2d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:20.719292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=243cfbc91bb24e7ca96f348cd21f1adb, user_id=9b9b59fb1a184b958ea619062412cf2d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:20.731333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:20.781374 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1833/3654] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:20 2022] PUT /identity/v3/projects/243cfbc91bb24e7ca96f348cd21f1adb/users/9b9b59fb1a184b958ea619062412cf2d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:20.785117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a70c1-9753-444e-89b0-074c5a10c2ed None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:20.785419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a70c1-9753-444e-89b0-074c5a10c2ed None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:20.785700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a70c1-9753-444e-89b0-074c5a10c2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:20.829387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f15a70c1-9753-444e-89b0-074c5a10c2ed None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:20.831032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f15a70c1-9753-444e-89b0-074c5a10c2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:21.001450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:21.026216 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1822/3655] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:21.033950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:21.036031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:21.259764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e574-ecfc-4f43-aac2-26ccdd588a8d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:21.267167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa62abc-bbbd-43d6-92dd-9be7c423e39b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:21.269535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62abc-bbbd-43d6-92dd-9be7c423e39b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:21.270027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62abc-bbbd-43d6-92dd-9be7c423e39b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:21.271180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa62abc-bbbd-43d6-92dd-9be7c423e39b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:21.316424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa62abc-bbbd-43d6-92dd-9be7c423e39b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:21.318595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa62abc-bbbd-43d6-92dd-9be7c423e39b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:21.363805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1834/3656] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:23.816613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7604a-39bd-4ecc-806f-8b069da5f096 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:23.817216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7604a-39bd-4ecc-806f-8b069da5f096 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:23.817216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7604a-39bd-4ecc-806f-8b069da5f096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:23.881684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-37b7604a-39bd-4ecc-806f-8b069da5f096 None None] MFA Rules not processed for user `9b9b59fb1a184b958ea619062412cf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:23.883131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-37b7604a-39bd-4ecc-806f-8b069da5f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:24.089881 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1823/3657] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:24.103919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:24.105136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:24.105589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:24.141968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] MFA Rules not processed for user `9b9b59fb1a184b958ea619062412cf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:24.143650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:24.163451 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1835/3658] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:24.165043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5a699-137f-4308-943b-5035de5ab775 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:24.165331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5a699-137f-4308-943b-5035de5ab775 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:24.165608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5a699-137f-4308-943b-5035de5ab775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:24.203136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c2a5a699-137f-4308-943b-5035de5ab775 None None] MFA Rules not processed for user `9b9b59fb1a184b958ea619062412cf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:24.208880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c2a5a699-137f-4308-943b-5035de5ab775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:24.217890 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1824/3659] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:26.063124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:26.066042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:26.173952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ee0e0-5020-4303-954f-f4a6fcb278a6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:26.181228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d73a0-68c0-47a6-8ce6-3c12df4f1930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:26.182193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d73a0-68c0-47a6-8ce6-3c12df4f1930 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:26.182946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d73a0-68c0-47a6-8ce6-3c12df4f1930 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:26.183604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d73a0-68c0-47a6-8ce6-3c12df4f1930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:26.199005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d73a0-68c0-47a6-8ce6-3c12df4f1930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:26.201121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d73a0-68c0-47a6-8ce6-3c12df4f1930 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:26.233895 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1836/3660] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:29.370326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ad66-eacb-431c-a2e2-254e4fdc5f96 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:29.371377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ad66-eacb-431c-a2e2-254e4fdc5f96 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:29.371822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723ad66-eacb-431c-a2e2-254e4fdc5f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:29.459380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d723ad66-eacb-431c-a2e2-254e4fdc5f96 None None] MFA Rules not processed for user `9b9b59fb1a184b958ea619062412cf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:29.460759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d723ad66-eacb-431c-a2e2-254e4fdc5f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:29.502621 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1825/3661] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:31.453402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:31.454426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:31.454606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:31.523792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] MFA Rules not processed for user `9b9b59fb1a184b958ea619062412cf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:31.530096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:31.567783 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1837/3662] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:34.238424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.238912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.239019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.366308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:34.368147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.531027 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1826/3663] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:00:34.551888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:34.554132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:34.627268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:34.868498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049ae45-cfd6-4611-8294-50fa0170eb1c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:34.889484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:34.890173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:34.890450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:34.890724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:34.905389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:34.908131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:34.934363 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1838/3664] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:00:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:00:42.702888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:42.704432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:42.757590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4ec47-9083-41dd-b7d8-2d8416308135 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:42.779546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:42.794290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:42.795973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:42.796435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] PATH_INFO: `/v3/users/f71dcabaa905490ea6c869426d2f64ee` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:42.801328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71dcabaa905490ea6c869426d2f64ee)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:42.810930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:42.949988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71dcabaa905490ea6c869426d2f64ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:42.980111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71dcabaa905490ea6c869426d2f64ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:42.981335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71dcabaa905490ea6c869426d2f64ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:43.004363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71dcabaa905490ea6c869426d2f64ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:43.005145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ddd2a87-8b5d-47ab-94e6-cd2fd98480de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71dcabaa905490ea6c869426d2f64ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:43.027205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:43.050435 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1827/3665] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:42 2022] DELETE /identity/v3/users/f71dcabaa905490ea6c869426d2f64ee => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:43.940661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:43.966686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bae35-8f1a-466e-a216-ba808f9797c7 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:44.127973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:44.129053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:44.129593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:44.130133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] PATH_INFO: `/v3/projects/594351b0dfd749ba84bbd061f16b76dd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:44.132052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594351b0dfd749ba84bbd061f16b76dd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:44.136774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:44.265003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594351b0dfd749ba84bbd061f16b76dd', 'request_id': 'req-91278080-ad9f-4657-9698-47dbaf0f6003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.281737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594351b0dfd749ba84bbd061f16b76dd', 'request_id': 'req-91278080-ad9f-4657-9698-47dbaf0f6003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.323307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594351b0dfd749ba84bbd061f16b76dd', 'request_id': 'req-91278080-ad9f-4657-9698-47dbaf0f6003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.353957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] The token cache is being invalidate because project 594351b0dfd749ba84bbd061f16b76dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:44.354535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.355426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-91278080-ad9f-4657-9698-47dbaf0f6003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:44.357162 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1839/3666] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:43 2022] DELETE /identity/v3/projects/594351b0dfd749ba84bbd061f16b76dd => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:44.363183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43a912-e41b-4fef-8e0c-1c7d13fbca83 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:44.364124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43a912-e41b-4fef-8e0c-1c7d13fbca83 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:44.364634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43a912-e41b-4fef-8e0c-1c7d13fbca83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:44.396250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7f43a912-e41b-4fef-8e0c-1c7d13fbca83 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:44.397854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f43a912-e41b-4fef-8e0c-1c7d13fbca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:44.584737 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1828/3667] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:44.593795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12e9dc-c083-4471-9ca1-4c695025a2d4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:44.594265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12e9dc-c083-4471-9ca1-4c695025a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:44.594639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12e9dc-c083-4471-9ca1-4c695025a2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:44.648932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1e12e9dc-c083-4471-9ca1-4c695025a2d4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:44.650486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1e12e9dc-c083-4471-9ca1-4c695025a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:44.673813 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1840/3668] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:44.675094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43a912-e41b-4fef-8e0c-1c7d13fbca83 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:44.691155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43a912-e41b-4fef-8e0c-1c7d13fbca83 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:44.714658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:44.715302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:44.715586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:44.716181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:44.716524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:44.716845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:44.721544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:44.762260 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1829/3669] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:44.767852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12e9dc-c083-4471-9ca1-4c695025a2d4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:44.795465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12e9dc-c083-4471-9ca1-4c695025a2d4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:44.813407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:44.814592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:44.814905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:44.815202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:44.815804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:44.817756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:44.912582 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1841/3670] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:44 2022] POST /identity/v3/projects => generated 361 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:44.916279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:44.932222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196aa1ef-40c5-4264-95fd-91aed68dbf23 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.073511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.075279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.076442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.078353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.079336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.084644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.155683 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1830/3671] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:44 2022] POST /identity/v3/users => generated 281 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:45.160425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.178779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd5d98-db0a-45e2-b5bf-2b8373a0ed6d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.185655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.186399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.186678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.186952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.187590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.189887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.191704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:45.230110 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1842/3672] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:45.234294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.279221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157163b1-650c-4282-b059-6133d2b6f94b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.292435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a13f0-bde2-4ec4-aa68-fc544503e540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.294261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a13f0-bde2-4ec4-aa68-fc544503e540 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.296032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a13f0-bde2-4ec4-aa68-fc544503e540 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.296345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a13f0-bde2-4ec4-aa68-fc544503e540 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.299708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a13f0-bde2-4ec4-aa68-fc544503e540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.304220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a13f0-bde2-4ec4-aa68-fc544503e540 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.339786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:45.342128 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1831/3673] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:00:45.357259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3835531-5e9e-4bd3-80f8-765aeef4d4fb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:45.368317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:45.369274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.369677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.370137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] PATH_INFO: `/v3/projects/6ddbb1ef58b643c9bf457bd602e24069/users/a1d2a1e0b34542faafad73347969b114/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.405147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ddbb1ef58b643c9bf457bd602e24069, user_id=a1d2a1e0b34542faafad73347969b114, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:45.410587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:45.447300 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1843/3674] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:45 2022] PUT /identity/v3/projects/6ddbb1ef58b643c9bf457bd602e24069/users/a1d2a1e0b34542faafad73347969b114/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:45.454240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5a5b-82e2-4932-a430-94d2fe4e478a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:45.455529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5a5b-82e2-4932-a430-94d2fe4e478a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:45.455969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5a5b-82e2-4932-a430-94d2fe4e478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:45.522561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9bae5a5b-82e2-4932-a430-94d2fe4e478a None None] MFA Rules not processed for user `a1d2a1e0b34542faafad73347969b114`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:45.526104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bae5a5b-82e2-4932-a430-94d2fe4e478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:45.658439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:45.702738 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1832/3675] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:46.779021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:46.926357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7e28-0163-4332-a64d-c316e648b09d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:46.942375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:46.945963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:46.946682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:46.947781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] PATH_INFO: `/v3/users/a1d2a1e0b34542faafad73347969b114` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:46.951133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d2a1e0b34542faafad73347969b114)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:46.957429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:47.098724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d2a1e0b34542faafad73347969b114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.135252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d2a1e0b34542faafad73347969b114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.151670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d2a1e0b34542faafad73347969b114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.172395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d2a1e0b34542faafad73347969b114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.174668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d2a1e0b34542faafad73347969b114'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:47.193595 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1844/3676] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:46 2022] DELETE /identity/v3/users/a1d2a1e0b34542faafad73347969b114 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:47.198227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:47.198732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:47.199230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:47.243355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:47.244830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:47.425110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1833/3677] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:47.574954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:47.576364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:47.778026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8f7b4-c2d0-427a-9157-889e7df27d0c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:47.783165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1474232-0bbf-4854-bef9-e0bfa2eccfd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:47.784020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1474232-0bbf-4854-bef9-e0bfa2eccfd7 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:47.784169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1474232-0bbf-4854-bef9-e0bfa2eccfd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:47.784452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1474232-0bbf-4854-bef9-e0bfa2eccfd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:47.791706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:47.798282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1474232-0bbf-4854-bef9-e0bfa2eccfd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:47.801895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1474232-0bbf-4854-bef9-e0bfa2eccfd7 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:47.824096 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1845/3678] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:49.468328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.471918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:49.494192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168440e-3c6b-460f-b88d-878923244aa8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.502569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.503551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.504068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.504578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] PATH_INFO: `/v3/projects/6ddbb1ef58b643c9bf457bd602e24069` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.506062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ddbb1ef58b643c9bf457bd602e24069)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.518186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.610388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ddbb1ef58b643c9bf457bd602e24069', 'request_id': 'req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:49.619205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ddbb1ef58b643c9bf457bd602e24069', 'request_id': 'req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:49.632385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ddbb1ef58b643c9bf457bd602e24069', 'request_id': 'req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:49.649386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] The token cache is being invalidate because project 6ddbb1ef58b643c9bf457bd602e24069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:49.649701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:49.650494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-102130ee-7cb5-4d71-89e0-7a78ccd6d1fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:49.652547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1834/3679] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:49 2022] DELETE /identity/v3/projects/6ddbb1ef58b643c9bf457bd602e24069 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:49.657540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c5432-ebe1-49a5-8532-e0ad3f0dc9ab None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.659305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c5432-ebe1-49a5-8532-e0ad3f0dc9ab None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.659806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c5432-ebe1-49a5-8532-e0ad3f0dc9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.688336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dd0c5432-ebe1-49a5-8532-e0ad3f0dc9ab None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:49.689994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dd0c5432-ebe1-49a5-8532-e0ad3f0dc9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:49.776574 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1846/3680] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:49.778672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7ca27-6cf8-49b7-9dc1-cec1c002f8c2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.779071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7ca27-6cf8-49b7-9dc1-cec1c002f8c2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.779934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7ca27-6cf8-49b7-9dc1-cec1c002f8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.810600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a6c7ca27-6cf8-49b7-9dc1-cec1c002f8c2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:49.813527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a6c7ca27-6cf8-49b7-9dc1-cec1c002f8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:49.823818 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1835/3681] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:49.828274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c5432-ebe1-49a5-8532-e0ad3f0dc9ab None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.853876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c5432-ebe1-49a5-8532-e0ad3f0dc9ab None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.862164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.863130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.863578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.864220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.865087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:49.865618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.867699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:49.892689 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1847/3682] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:49.893760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7ca27-6cf8-49b7-9dc1-cec1c002f8c2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:49.915049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7ca27-6cf8-49b7-9dc1-cec1c002f8c2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:49.943188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:49.943903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:49.946267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:49.946543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:49.950433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:49.955801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.062807 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1836/3683] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:49 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:50.075591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:50.104280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18d707-0281-4b9c-a344-b58ac33b7514 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:50.279262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:50.279885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.280174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.280452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.281365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:50.284043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.320146 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1848/3684] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:50 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:50.327761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:50.346278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:50.348757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:50.350085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31ff51-95cd-4012-a7c1-2f40a8a332d5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:50.360468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:50.361903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.361903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.364094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.365393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:50.369863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.393324 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1837/3685] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:50.395099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:50.408717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d46ad-0650-4e30-bbca-4ac004fba2e8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:50.414010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:50.414703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.415053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.415395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.416195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:50.419136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.431947 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1838/3686] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:50.440593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:50.464805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a15c8-d74e-4ae5-87f1-8b5c60065173 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:50.490641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1d9ef-892c-4073-9ef3-1a59dc8d948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:50.491401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d9ef-892c-4073-9ef3-1a59dc8d948f admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.491667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d9ef-892c-4073-9ef3-1a59dc8d948f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.492156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d9ef-892c-4073-9ef3-1a59dc8d948f admin admin] PATH_INFO: `/v3/projects/7f98b25814e941e09751a589f04e2a1f/users/efa76aa7f26c4d3ca239b31bd5e63fe4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.533173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1d9ef-892c-4073-9ef3-1a59dc8d948f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f98b25814e941e09751a589f04e2a1f, user_id=efa76aa7f26c4d3ca239b31bd5e63fe4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:50.538112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1d9ef-892c-4073-9ef3-1a59dc8d948f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.551488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:50.557915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafb01-00de-474f-8aa3-0383739f7a82 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:50.562748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1839/3687] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:50 2022] PUT /identity/v3/projects/7f98b25814e941e09751a589f04e2a1f/users/efa76aa7f26c4d3ca239b31bd5e63fe4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:50.578149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.578710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.580703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.623276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:50.646627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] MFA Rules not processed for user `efa76aa7f26c4d3ca239b31bd5e63fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:50.648132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:50.693520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:50.697233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:50.698003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:50.698756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:50.705054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:50.805601 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1840/3688] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:50.905866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:50.907355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:50.944075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1849/3689] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:51.169237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:51.172126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:51.271309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:51.273102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:51.296511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d535db7-e41e-4629-8748-1ccb6ce5a01b service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:51.303897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:51.304921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:51.305468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:51.306880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:51.325808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:51.328263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:51.344160 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1850/3690] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:51.355126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a4701-6c23-4667-b4f3-dee5a48230fe None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:51.365092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cc48b-031b-477f-9ade-472efde1091c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:51.366438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc48b-031b-477f-9ade-472efde1091c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:51.366705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc48b-031b-477f-9ade-472efde1091c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:51.366985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc48b-031b-477f-9ade-472efde1091c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:51.368792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d08cc48b-031b-477f-9ade-472efde1091c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:51.380391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cc48b-031b-477f-9ade-472efde1091c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:51.382027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cc48b-031b-477f-9ade-472efde1091c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:51.399789 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1841/3691] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:51.603247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:51.606127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:51.643952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1655748-2a2b-4223-83a0-8f688d1366de service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:51.653762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea61f-2d87-4334-8919-19cfdd870812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:51.654394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea61f-2d87-4334-8919-19cfdd870812 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:51.654677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea61f-2d87-4334-8919-19cfdd870812 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:51.655327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea61f-2d87-4334-8919-19cfdd870812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:51.660039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8b3ea61f-2d87-4334-8919-19cfdd870812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:51.699150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ea61f-2d87-4334-8919-19cfdd870812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:51.704033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ea61f-2d87-4334-8919-19cfdd870812 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:51.736988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1851/3692] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:54.566464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd8201-4a7b-47fe-a562-bde10ba8e975 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:54.566922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd8201-4a7b-47fe-a562-bde10ba8e975 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:54.567029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd8201-4a7b-47fe-a562-bde10ba8e975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:54.614650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2ffd8201-4a7b-47fe-a562-bde10ba8e975 None None] MFA Rules not processed for user `d57829f24d3841eb9c1d6f5c6a5ea7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:54.615880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2ffd8201-4a7b-47fe-a562-bde10ba8e975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:54.705890 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1842/3693] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:54.716014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae97557-b50d-4231-b94e-02b2e178297e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:54.717290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae97557-b50d-4231-b94e-02b2e178297e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:54.717290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae97557-b50d-4231-b94e-02b2e178297e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:54.744828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bae97557-b50d-4231-b94e-02b2e178297e None None] MFA Rules not processed for user `d57829f24d3841eb9c1d6f5c6a5ea7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:54.746248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bae97557-b50d-4231-b94e-02b2e178297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:54.764386 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1852/3694] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:54.970775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd8201-4a7b-47fe-a562-bde10ba8e975 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.044396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:55.078588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd8201-4a7b-47fe-a562-bde10ba8e975 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:55.084221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:55.085160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.085524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.086050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] PATH_INFO: `/v3/users/efa76aa7f26c4d3ca239b31bd5e63fe4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.087336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efa76aa7f26c4d3ca239b31bd5e63fe4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:55.089442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:55.222050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efa76aa7f26c4d3ca239b31bd5e63fe4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.253622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efa76aa7f26c4d3ca239b31bd5e63fe4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.254603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efa76aa7f26c4d3ca239b31bd5e63fe4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.296109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efa76aa7f26c4d3ca239b31bd5e63fe4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.297183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efa76aa7f26c4d3ca239b31bd5e63fe4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:55.375401 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1843/3695] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:00:54 2022] DELETE /identity/v3/users/efa76aa7f26c4d3ca239b31bd5e63fe4 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:55.380509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.380890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.381301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.410477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:55.412588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.601271 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1853/3696] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:55.612925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.615172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.736920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:55.738746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.801414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:55.856901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f9deb-b8ee-48cc-8c82-3bedcf78fb00 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:55.858357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e38b3d-af37-454b-a685-c61f8e63f5e8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:55.875158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:55.876014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa4e72-ffac-44f1-9fc5-a3e802a22b4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:55.879742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.880265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa4e72-ffac-44f1-9fc5-a3e802a22b4c service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:55.880636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.880993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa4e72-ffac-44f1-9fc5-a3e802a22b4c service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:55.881470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.881862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa4e72-ffac-44f1-9fc5-a3e802a22b4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:55.886873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8aaa4e72-ffac-44f1-9fc5-a3e802a22b4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:55.904698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:55.907262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:55.924300 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1844/3697] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:00:55.927506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa4e72-ffac-44f1-9fc5-a3e802a22b4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:55.929300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa4e72-ffac-44f1-9fc5-a3e802a22b4c service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:55.959806 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1854/3698] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:00:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:00:57.418193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:57.422119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:57.451827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47682bc-e72b-43dc-a97e-eabb69daefa8 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:57.471730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:57.473881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:57.474175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:57.474460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] PATH_INFO: `/v3/projects/7f98b25814e941e09751a589f04e2a1f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:57.479034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f98b25814e941e09751a589f04e2a1f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:57.485363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:57.624640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f98b25814e941e09751a589f04e2a1f', 'request_id': 'req-8277c79e-77d8-48e5-8fd4-7d874a8597d7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:57.627623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:57.641561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f98b25814e941e09751a589f04e2a1f', 'request_id': 'req-8277c79e-77d8-48e5-8fd4-7d874a8597d7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:57.657664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f98b25814e941e09751a589f04e2a1f', 'request_id': 'req-8277c79e-77d8-48e5-8fd4-7d874a8597d7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:57.681882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] The token cache is being invalidate because project 7f98b25814e941e09751a589f04e2a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:00:57.682195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:57.682895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8277c79e-77d8-48e5-8fd4-7d874a8597d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:00:57.684714 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1845/3699] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:00:57 2022] DELETE /identity/v3/projects/7f98b25814e941e09751a589f04e2a1f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:57.695119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e69e1-a74a-4dbc-a9b5-a864263c5b9d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:57.695653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e69e1-a74a-4dbc-a9b5-a864263c5b9d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:57.695653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e69e1-a74a-4dbc-a9b5-a864263c5b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:57.750773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-649e69e1-a74a-4dbc-a9b5-a864263c5b9d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:57.753694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-649e69e1-a74a-4dbc-a9b5-a864263c5b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:57.898229 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1855/3700] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:57.905991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1fbd-d1b1-4cda-b7cc-2a7476648c34 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:57.906859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1fbd-d1b1-4cda-b7cc-2a7476648c34 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:57.906859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1fbd-d1b1-4cda-b7cc-2a7476648c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:57.951823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-767c1fbd-d1b1-4cda-b7cc-2a7476648c34 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:57.956644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-767c1fbd-d1b1-4cda-b7cc-2a7476648c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:57.980592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1846/3701] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:57.985176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e69e1-a74a-4dbc-a9b5-a864263c5b9d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:58.014109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e69e1-a74a-4dbc-a9b5-a864263c5b9d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:58.048859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:58.049494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:58.058111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:58.058455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:58.059180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:00:58.059494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:58.061210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:58.096590 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1856/3702] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:58.101773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c1fbd-d1b1-4cda-b7cc-2a7476648c34 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:58.116430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c1fbd-d1b1-4cda-b7cc-2a7476648c34 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:58.131580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:58.132908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:58.133318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:58.134033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:58.139850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:58.143612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:58.213163 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1847/3703] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:58 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:58.220480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:58.242291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa103dc-4414-454c-879b-3da50256c137 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:58.428986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:58.429621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:58.430120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:58.430575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:58.431381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:58.433503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:58.517362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:58.519811 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1857/3704] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:58 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:58.549813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5701ff0-7708-4e97-8ce3-bbe8920c5ae6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:58.581769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:58.582475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:58.582843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:58.583064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:58.583660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:58.592373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:58.623353 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1848/3705] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:58.627980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:58.645203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eebc1e-4d05-4503-ba15-0b464643dd57 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:58.655937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4626297-40d9-4722-b20d-084486fbc3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:58.656569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4626297-40d9-4722-b20d-084486fbc3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:58.656884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4626297-40d9-4722-b20d-084486fbc3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:58.658147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4626297-40d9-4722-b20d-084486fbc3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:58.660129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4626297-40d9-4722-b20d-084486fbc3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:58.668658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4626297-40d9-4722-b20d-084486fbc3f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:58.695618 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1858/3706] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:58.699844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:58.716495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c8297-b789-43f4-8f48-8dce51c59176 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:58.730982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:58.731823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:58.732155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:58.732579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] PATH_INFO: `/v3/projects/fc1334b2602245f18988b2c57dc56acf/users/d474fa2e9eff44cdba6349ce05f75174/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:58.764877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1334b2602245f18988b2c57dc56acf, user_id=d474fa2e9eff44cdba6349ce05f75174, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:58.768408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:58.795226 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1849/3707] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:58 2022] PUT /identity/v3/projects/fc1334b2602245f18988b2c57dc56acf/users/d474fa2e9eff44cdba6349ce05f75174/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:58.806018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257d6e-7301-4ec9-abe3-d0d08c85e66e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:58.806633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257d6e-7301-4ec9-abe3-d0d08c85e66e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:58.807900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257d6e-7301-4ec9-abe3-d0d08c85e66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:58.843165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-41257d6e-7301-4ec9-abe3-d0d08c85e66e None None] MFA Rules not processed for user `d474fa2e9eff44cdba6349ce05f75174`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:58.845603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-41257d6e-7301-4ec9-abe3-d0d08c85e66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:58.932374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:59.061405 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1859/3708] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:59.066545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.233354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfcf73-9668-447c-9fb1-de84760e38dd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.270951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.271588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.272452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.272741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.273390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.275364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.333285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1850/3709] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:00:59 2022] POST /identity/v3/projects => generated 389 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:59.337144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257d6e-7301-4ec9-abe3-d0d08c85e66e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.352316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257d6e-7301-4ec9-abe3-d0d08c85e66e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.439004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.440118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.440820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.441369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.442228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.445039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.473888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1860/3710] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:00:59 2022] POST /identity/v3/users => generated 301 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:59.475389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.488916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb1f3-5472-42f9-be15-2fbe9b6acb27 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.493769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.494541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.494847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.495195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.496507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.498342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.513146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1851/3711] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:59.517092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.528305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9222552-fd9e-499d-b5d2-59b009b1528c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.537990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.538758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.539255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.539385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] PATH_INFO: `/v3/projects/f6234a9eabee488d9ecb800922c4511e/users/2c305f8d190c4334b586e66f1c40411d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.556693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6234a9eabee488d9ecb800922c4511e, user_id=2c305f8d190c4334b586e66f1c40411d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.559603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.582461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1861/3712] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:59 2022] PUT /identity/v3/projects/f6234a9eabee488d9ecb800922c4511e/users/2c305f8d190c4334b586e66f1c40411d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:59.585113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.597148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de180a-8d28-4388-9842-cbfb80fc43b9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.649291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:00:59.664703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe1721-7825-4c3c-bb25-52d46c3f67a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.665293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1721-7825-4c3c-bb25-52d46c3f67a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.665900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1721-7825-4c3c-bb25-52d46c3f67a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.666359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe1721-7825-4c3c-bb25-52d46c3f67a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.667064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe1721-7825-4c3c-bb25-52d46c3f67a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.668655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe1721-7825-4c3c-bb25-52d46c3f67a5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.688127 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1852/3713] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:00:59.692961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.705111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60416e3-3c3f-48dc-9f94-c5bc2e4591e5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.709882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.710546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.710829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.711403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] PATH_INFO: `/v3/projects/f6234a9eabee488d9ecb800922c4511e/users/2c305f8d190c4334b586e66f1c40411d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.713025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6234a9eabee488d9ecb800922c4511e, user_id=2c305f8d190c4334b586e66f1c40411d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.716373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.733919 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1862/3714] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:00:59 2022] PUT /identity/v3/projects/f6234a9eabee488d9ecb800922c4511e/users/2c305f8d190c4334b586e66f1c40411d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:00:59.741572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76819ea-1db6-4746-b7d5-8282dd40a64b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.741896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76819ea-1db6-4746-b7d5-8282dd40a64b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.742187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76819ea-1db6-4746-b7d5-8282dd40a64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.779210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d76819ea-1db6-4746-b7d5-8282dd40a64b None None] MFA Rules not processed for user `2c305f8d190c4334b586e66f1c40411d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:00:59.780517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d76819ea-1db6-4746-b7d5-8282dd40a64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:00:59.874714 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1853/3715] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:00:59 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:00:59.880322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.892063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522f3d5-a9b2-4de6-887b-0a1c283dd00e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.897529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.898747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.899239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.899722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects/fc1334b2602245f18988b2c57dc56acf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.901386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:get_project(project_id=fc1334b2602245f18988b2c57dc56acf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.903774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.905678 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1863/3716] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:00:59 2022] GET /identity/v3/projects/fc1334b2602245f18988b2c57dc56acf => generated 387 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:00:59.914243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76819ea-1db6-4746-b7d5-8282dd40a64b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.929915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76819ea-1db6-4746-b7d5-8282dd40a64b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:00:59.935621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:00:59.936475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:00:59.936961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:00:59.937294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:00:59.941246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:00:59.942161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:00:59.976897 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1854/3717] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:00:59 2022] POST /identity/v3/projects => generated 338 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:00:59.981309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:00:59.995357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e9ffe-4b39-4db3-b190-b54a8bc98394 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.072009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.072843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.073209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.073556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.075177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.079761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.099362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1864/3718] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:00:59 2022] GET /identity/v3/projects => generated 6976 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:00.103580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.115553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c71e3d-d6d1-4ebb-8ed0-83a49421a515 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.120466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.121103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.121388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.121666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects/fc1334b2602245f18988b2c57dc56acf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.122848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:get_project(project_id=fc1334b2602245f18988b2c57dc56acf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.124887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.127464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1855/3719] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects/fc1334b2602245f18988b2c57dc56acf => generated 387 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:00.131378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.141694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:00.149064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d87e53-4dd9-4e3f-9976-6e5bf6c83b68 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.156087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.156825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.157992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.158581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects/cac21b3ab6ae44e39a50fcac4a33753b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.160410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:get_project(project_id=cac21b3ab6ae44e39a50fcac4a33753b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.162055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.165902 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1865/3720] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects/cac21b3ab6ae44e39a50fcac4a33753b => generated 338 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:00.169537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.182755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3e3e5-1a7c-4076-ae94-f30fa80acf6b tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.190719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.191335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.191621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.191890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.192534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.193737 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:01:00.193737 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:01:00.193737 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:01:00.195376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.221175 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1856/3721] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects => generated 6976 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:00.226083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.243723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67bd7d-c5c9-4550-b8ed-59ad1929f49c tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.248738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.249562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.250066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.250516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.251571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:00.252144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.254762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.292472 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1866/3722] 10.222.0.33 () {58 vars in 1166 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects?domain_id=default => generated 6645 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:00.296801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.309594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.310933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7dd60-dec1-4b7e-87aa-008bd0ab9c07 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.313636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:00.319928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.321142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.321583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.322490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.323262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.334358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.364244 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1857/3723] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects => generated 6976 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:00.367743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.379552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dd2b6-f134-46f4-9bd5-34245e982500 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.384826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.385449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.385941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.386385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.387269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:00.387588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.389703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.401715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c03e6-3a5f-4694-9260-c024d5e78c04 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.409057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.409871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.410017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.410251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.412244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:00.420828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1858/3724] 10.222.0.33 () {58 vars in 1166 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects?domain_id=default => generated 6645 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:00.428818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.449526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139c8ef-d848-432e-83b1-401fb55eef8a tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.456590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.457244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.457534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.457852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.458564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.460581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.507963 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1859/3725] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects => generated 6976 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:00.509537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.520912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98363717-dfdb-4cc5-b537-4e2ab0549a66 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.526189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.526738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.527574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.528062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.528178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.528660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.529523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-748435917) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:00.530073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.533081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.545051 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1867/3726] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:00.555365 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1860/3727] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:01:00 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-748435917 => generated 534 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:00.560790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.580200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac650c-cedf-4a0a-b82c-be82d0559692 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.601021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c305f8d190c4334b586e66f1c40411d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6234a9eabee488d9ecb800922c4511e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.601689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.602024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.602302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] PATH_INFO: `/v3/projects/cac21b3ab6ae44e39a50fcac4a33753b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.603452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cac21b3ab6ae44e39a50fcac4a33753b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.605396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:00.681589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac21b3ab6ae44e39a50fcac4a33753b', 'request_id': 'req-951bec13-12dc-4ea9-824f-2e3be72f828e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.690847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac21b3ab6ae44e39a50fcac4a33753b', 'request_id': 'req-951bec13-12dc-4ea9-824f-2e3be72f828e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.706062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac21b3ab6ae44e39a50fcac4a33753b', 'request_id': 'req-951bec13-12dc-4ea9-824f-2e3be72f828e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.729432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] The token cache is being invalidate because project cac21b3ab6ae44e39a50fcac4a33753b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:00.730074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.732335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-951bec13-12dc-4ea9-824f-2e3be72f828e tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:00.742455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:00.743501 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1868/3728] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:00 2022] DELETE /identity/v3/projects/cac21b3ab6ae44e39a50fcac4a33753b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:00.745157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:00.804640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:00.841959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7452a-88f5-4fa5-ab7b-eb3ea3802aa0 tempest-ListProjectsStaticTestJSON-1627461459 tempest-ListProjectsStaticTestJSON-1627461459-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:00.929093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:00.929970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:00.930256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:00.930538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] PATH_INFO: `/v3/users/d474fa2e9eff44cdba6349ce05f75174` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:00.931885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d474fa2e9eff44cdba6349ce05f75174)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:00.934041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:01.038939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd474fa2e9eff44cdba6349ce05f75174'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.057306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd474fa2e9eff44cdba6349ce05f75174'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.058437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd474fa2e9eff44cdba6349ce05f75174'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.091308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd474fa2e9eff44cdba6349ce05f75174'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.091941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd474fa2e9eff44cdba6349ce05f75174'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:01.186999 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1861/3729] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:00 2022] DELETE /identity/v3/users/d474fa2e9eff44cdba6349ce05f75174 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:01.191444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd5ff4-a4ce-4d45-9822-ad9b0fbf5f2a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.191801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd5ff4-a4ce-4d45-9822-ad9b0fbf5f2a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.192094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd5ff4-a4ce-4d45-9822-ad9b0fbf5f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.230707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5efd5ff4-a4ce-4d45-9822-ad9b0fbf5f2a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:01.232200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5efd5ff4-a4ce-4d45-9822-ad9b0fbf5f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:01.251970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:01.386444 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1869/3730] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:01.403229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:01.405210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:01.582038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171256ed-57c7-419a-8f1d-6771f1a3e294 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:01.593608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:01.594325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:01.596892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:01.596892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:01.622185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:01.624673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:01.642145 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1862/3731] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:02.710032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:02.710909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:02.711343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:02.755330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:02.756634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:02.923145 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1870/3732] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:01:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:02.940823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:02.942517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:02.960912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:03.102413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3eb14-6922-4113-993b-1e0d64ccd231 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:03.115362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:03.116015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:03.116335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:03.116606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:03.136736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:03.138882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:03.163690 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1863/3733] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:03.293463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:03.296442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:03.325677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3d466-9291-4889-9634-8892c5534fbb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:03.338327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:03.339012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:03.339296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:03.339573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] PATH_INFO: `/v3/projects/fc1334b2602245f18988b2c57dc56acf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:03.343652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1334b2602245f18988b2c57dc56acf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:03.350658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:03.466186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1334b2602245f18988b2c57dc56acf', 'request_id': 'req-6f15d997-7f69-4f74-a545-c92f6fc9f416'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.478020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1334b2602245f18988b2c57dc56acf', 'request_id': 'req-6f15d997-7f69-4f74-a545-c92f6fc9f416'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.492847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1334b2602245f18988b2c57dc56acf', 'request_id': 'req-6f15d997-7f69-4f74-a545-c92f6fc9f416'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.516116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] The token cache is being invalidate because project fc1334b2602245f18988b2c57dc56acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:03.523134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.527717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.529938 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1871/3734] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:03 2022] DELETE /identity/v3/projects/fc1334b2602245f18988b2c57dc56acf => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:03.534344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:03.555058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcbdfb-a1f3-4ebf-90df-76b5b1e1fe84 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:03.678415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:03.679110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:03.679393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:03.679819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] PATH_INFO: `/v3/users/2c305f8d190c4334b586e66f1c40411d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:03.682157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c305f8d190c4334b586e66f1c40411d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:03.683923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:03.784000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c305f8d190c4334b586e66f1c40411d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.816951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c305f8d190c4334b586e66f1c40411d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.818066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c305f8d190c4334b586e66f1c40411d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.848624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c305f8d190c4334b586e66f1c40411d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.849523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02a0f91f-48dd-48c0-9c2b-f641b02c6f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c305f8d190c4334b586e66f1c40411d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:03.907972 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1864/3735] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:03 2022] DELETE /identity/v3/users/2c305f8d190c4334b586e66f1c40411d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:05.927365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:05.928937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b46a-3d3b-4f7c-abe9-7813b08be3e5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:05.929352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b46a-3d3b-4f7c-abe9-7813b08be3e5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:05.929938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b46a-3d3b-4f7c-abe9-7813b08be3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:05.930620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:05.972248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15d997-7f69-4f74-a545-c92f6fc9f416 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:05.997622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d453b46a-3d3b-4f7c-abe9-7813b08be3e5 None None] MFA Rules not processed for user `ae0c7612d933424597e1f176be189a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:05.999429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d453b46a-3d3b-4f7c-abe9-7813b08be3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:06.043925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:06.115117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:06.207020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:06.207968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:06.209627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:06.213431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] PATH_INFO: `/v3/projects/f6234a9eabee488d9ecb800922c4511e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:06.219519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6234a9eabee488d9ecb800922c4511e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:06.220450 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1865/3736] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:06.226060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:06.441594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6234a9eabee488d9ecb800922c4511e', 'request_id': 'req-112c8701-e2dd-4645-aec2-12b1061c13da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:06.465926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6234a9eabee488d9ecb800922c4511e', 'request_id': 'req-112c8701-e2dd-4645-aec2-12b1061c13da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:06.478973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6234a9eabee488d9ecb800922c4511e', 'request_id': 'req-112c8701-e2dd-4645-aec2-12b1061c13da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:06.537593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] The token cache is being invalidate because project f6234a9eabee488d9ecb800922c4511e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:06.540626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:06.544457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-112c8701-e2dd-4645-aec2-12b1061c13da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:06.551270 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1872/3737] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:05 2022] DELETE /identity/v3/projects/f6234a9eabee488d9ecb800922c4511e => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:06.555100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5482a96-583a-4372-9a86-93d2efe28714 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:06.558450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5482a96-583a-4372-9a86-93d2efe28714 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:06.558450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5482a96-583a-4372-9a86-93d2efe28714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:06.597352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a5482a96-583a-4372-9a86-93d2efe28714 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:06.599183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a5482a96-583a-4372-9a86-93d2efe28714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:06.731834 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1866/3738] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:06.739292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07dde7-f16e-48bb-b47a-cabcf751456f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:06.739711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07dde7-f16e-48bb-b47a-cabcf751456f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:06.740099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07dde7-f16e-48bb-b47a-cabcf751456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:06.796715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5c07dde7-f16e-48bb-b47a-cabcf751456f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:06.799670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5c07dde7-f16e-48bb-b47a-cabcf751456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:06.827698 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1873/3739] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:06.833664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5482a96-583a-4372-9a86-93d2efe28714 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:06.851040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5482a96-583a-4372-9a86-93d2efe28714 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:06.870888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:06.871804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:06.872280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:06.873210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:06.873210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:06.873533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:06.875210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:06.890208 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1867/3740] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:06.894412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07dde7-f16e-48bb-b47a-cabcf751456f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:06.905098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07dde7-f16e-48bb-b47a-cabcf751456f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:06.911122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:06.911491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:06.911869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:06.912047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:06.912654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:06.914646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:06.957393 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1874/3741] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:06 2022] POST /identity/v3/projects => generated 397 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:06.961138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:06.984368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8f699-1450-4e3d-87b3-f6b02602665d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:07.098130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:07.098977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:07.099263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:07.099624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:07.100200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:07.102253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:07.129654 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1868/3742] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:06 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:07.135901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:07.163120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c5a4c-37c7-42bb-ac17-30bf5c9f3202 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:07.186940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:07.188312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:07.188510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:07.188682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:07.189343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:07.204617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:07.239992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:07.241116 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1875/3743] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:07.257327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb949-f3ee-4cac-a50d-d517a43953a1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:07.271206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59248c5-957b-4282-84a5-5a96e8cc917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:07.272160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59248c5-957b-4282-84a5-5a96e8cc917f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:07.272553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59248c5-957b-4282-84a5-5a96e8cc917f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:07.272940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59248c5-957b-4282-84a5-5a96e8cc917f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:07.273946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59248c5-957b-4282-84a5-5a96e8cc917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:07.276886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59248c5-957b-4282-84a5-5a96e8cc917f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:07.316465 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1869/3744] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:07.318097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:07.333628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5aa109-ea4c-48ba-827f-b6d40f9cfaf5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:07.349967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:07.350872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:07.351237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:07.351600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] PATH_INFO: `/v3/projects/acfe8b124c71401580f338b2438bff66/users/ad76ca5f129443a4a9ef111c2b7cb0ca/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:07.382176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfe8b124c71401580f338b2438bff66, user_id=ad76ca5f129443a4a9ef111c2b7cb0ca, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:07.385514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:07.414791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1876/3745] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:07 2022] PUT /identity/v3/projects/acfe8b124c71401580f338b2438bff66/users/ad76ca5f129443a4a9ef111c2b7cb0ca/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:07.416238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4d46a-b9f9-4c44-b5da-0ec10ba8d1d0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:07.416524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4d46a-b9f9-4c44-b5da-0ec10ba8d1d0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:07.416796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4d46a-b9f9-4c44-b5da-0ec10ba8d1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:07.508424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fee4d46a-b9f9-4c44-b5da-0ec10ba8d1d0 None None] MFA Rules not processed for user `ad76ca5f129443a4a9ef111c2b7cb0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:07.509606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fee4d46a-b9f9-4c44-b5da-0ec10ba8d1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:07.535902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:07.710757 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1870/3746] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:07.712217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:07.864242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880c625-b3b2-440e-9337-a038d7bfe2a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:07.872081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:07.873145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:07.873557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:07.873978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:07.876647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:07.878867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:07.913784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:07.949916 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1877/3747] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:07 2022] POST /identity/v3/projects => generated 397 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:07.953553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4d46a-b9f9-4c44-b5da-0ec10ba8d1d0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:07.973563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4d46a-b9f9-4c44-b5da-0ec10ba8d1d0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:08.080691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:08.080691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:08.080691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:08.080691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:08.081144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:08.082920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:08.112470 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1871/3748] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:07 2022] POST /identity/v3/users => generated 305 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:08.116306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:08.134789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03228add-3963-49f9-b197-abe704631e26 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:08.150657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:08.151280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:08.151569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:08.151879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:08.152771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:08.154578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:08.177666 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1878/3749] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:08.191537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:08.204943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d901b-9776-491c-8075-78f8668dd288 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:08.210332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:08.211140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:08.211429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:08.211762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] PATH_INFO: `/v3/projects/a68af4d0aa75461ca61fa59b83e7f029/users/9299319b37a84e0fae68c8176ba28df5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:08.234732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68af4d0aa75461ca61fa59b83e7f029, user_id=9299319b37a84e0fae68c8176ba28df5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:08.238716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:08.262228 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1872/3750] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:08 2022] PUT /identity/v3/projects/a68af4d0aa75461ca61fa59b83e7f029/users/9299319b37a84e0fae68c8176ba28df5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:08.266893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:08.284263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa075f-f281-4ca4-85ea-35f87a83019f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:08.467386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:08.468049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:08.468304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:08.468577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:08.469226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:08.476419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:08.535407 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1879/3751] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:08.539655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:08.566369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5536a-20a7-4f24-b5ae-cf3dff59d3a4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:08.590872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:08.593070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:08.598082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b968b-b642-4a68-a467-612ff1af17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:08.605268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b968b-b642-4a68-a467-612ff1af17df admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:08.606699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b968b-b642-4a68-a467-612ff1af17df admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:08.607385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b968b-b642-4a68-a467-612ff1af17df admin admin] PATH_INFO: `/v3/projects/a68af4d0aa75461ca61fa59b83e7f029/users/9299319b37a84e0fae68c8176ba28df5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:08.616626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b968b-b642-4a68-a467-612ff1af17df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68af4d0aa75461ca61fa59b83e7f029, user_id=9299319b37a84e0fae68c8176ba28df5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:08.630419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b968b-b642-4a68-a467-612ff1af17df admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:08.650037 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1873/3752] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:08 2022] PUT /identity/v3/projects/a68af4d0aa75461ca61fa59b83e7f029/users/9299319b37a84e0fae68c8176ba28df5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:08.662943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1069-8aca-41a6-b199-04b12cef1545 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:08.663552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1069-8aca-41a6-b199-04b12cef1545 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:08.664217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1069-8aca-41a6-b199-04b12cef1545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:08.792866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-119b1069-8aca-41a6-b199-04b12cef1545 None None] MFA Rules not processed for user `9299319b37a84e0fae68c8176ba28df5`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:08.794177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-119b1069-8aca-41a6-b199-04b12cef1545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:08.827313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9e01-067e-4e78-830f-548b2b85609b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.055998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:09.077344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1874/3753] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:08 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:09.082853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b1069-8aca-41a6-b199-04b12cef1545 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:09.154073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.154706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.154987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.155268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.161537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:09.229147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.231017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:09.271663 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1880/3754] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:09.316549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b1069-8aca-41a6-b199-04b12cef1545 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.335986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad76ca5f129443a4a9ef111c2b7cb0ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acfe8b124c71401580f338b2438bff66', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.336378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.336790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.337183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.337944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Sep 10 21:01:09.367902 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 10 21:01:09.369166 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.377626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:09.387371 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1875/3755] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:01:09 2022] POST /identity/v3/projects => generated 136 bytes in 305 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:01:09.392666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67f1db-fd10-47ee-b754-bc48d35c7d06 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.409060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9299319b37a84e0fae68c8176ba28df5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68af4d0aa75461ca61fa59b83e7f029', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.410132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.410483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.410913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.411661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.414116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Invalid input for field 'name': '' is too short Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: On instance['name']: Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: On instance['name']: Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: '' Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise error Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application '' Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.434556 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:01:09.436458 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application '' Sep 10 21:01:09.437670 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.444722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:09.451188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1881/3756] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:01:09 2022] POST /identity/v3/projects => generated 283 bytes in 70 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 10 21:01:09.465388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1683a-b4c0-4465-8c28-44a383d77dd6 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.474518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9299319b37a84e0fae68c8176ba28df5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68af4d0aa75461ca61fa59b83e7f029', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.475701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.475701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.476202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.478227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.484502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: On instance['name']: Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: On instance['name']: Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise error Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:01:09.504639 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:01:09.506702 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 10 21:01:09.508736 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.511178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1876/3757] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:01:09 2022] POST /identity/v3/projects => generated 412 bytes in 67 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 10 21:01:09.517472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:09.527499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:09.536023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece59a4-da58-4514-811e-86cbe2f862f8 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.548933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9299319b37a84e0fae68c8176ba28df5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68af4d0aa75461ca61fa59b83e7f029', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.554807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.555911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.555911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] PATH_INFO: `/v3/projects/d8bc630aec8e437d9ff3d11537a9c51e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.576652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d8bc630aec8e437d9ff3d11537a9c51e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.578956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Could not find project: d8bc630aec8e437d9ff3d11537a9c51e.: keystone.exception.ProjectNotFound: Could not find project: d8bc630aec8e437d9ff3d11537a9c51e. Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 10 21:01:09.603055 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d8bc630aec8e437d9ff3d11537a9c51e. Sep 10 21:01:09.603986 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:01:09.610034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:09.610202 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1882/3758] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:09 2022] DELETE /identity/v3/projects/d8bc630aec8e437d9ff3d11537a9c51e => generated 113 bytes in 94 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:01:09.627117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33f998-e944-42e7-87bd-09e34fde2efd tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.635973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad76ca5f129443a4a9ef111c2b7cb0ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'acfe8b124c71401580f338b2438bff66', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.636708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.637047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.637327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.638015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Sep 10 21:01:09.648444 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._list_projects() Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 10 21:01:09.649477 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:09.650134 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1877/3759] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:01:09 2022] GET /identity/v3/projects => generated 135 bytes in 41 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:01:09.655442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:09.679709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2cc98-7604-4d7b-94ec-953bf982bf9b tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.687733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9299319b37a84e0fae68c8176ba28df5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68af4d0aa75461ca61fa59b83e7f029', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.688995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.689296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.689609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.690365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.698317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:09.702209 ncdv-master-1 devstack@keystone.service[93527]: WARNING keystone.server.flask.common [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 10 21:01:09.767962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:09.779487 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1883/3760] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:09 2022] POST /identity/v3/projects => generated 357 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:09.787111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a234450-2ca9-4bad-b398-a7e84d0bf702 tempest-ProjectsNegativeStaticTestJSON-1793415352 tempest-ProjectsNegativeStaticTestJSON-1793415352-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:09.930548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9299319b37a84e0fae68c8176ba28df5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68af4d0aa75461ca61fa59b83e7f029', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:09.931199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:09.931488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:09.931775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:09.932410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:09.934484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:10.028642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.sql.core [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-547294647' for key 'project.ixu_project_name_domain_id'") Sep 10 21:01:10.028642 ncdv-master-1 devstack@keystone.service[93528]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 21:01:10.028642 ncdv-master-1 devstack@keystone.service[93528]: [parameters: {'id': '922f714e62024245b36f217d43fbbbef', 'name': 'tempest-project-dup-547294647', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 21:01:10.028642 ncdv-master-1 devstack@keystone.service[93528]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93528) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-547294647, project id 922f714e62024245b36f217d43fbbbef.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-547294647, project id 922f714e62024245b36f217d43fbbbef. Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 21:01:10.061234 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-547294647' for key 'project.ixu_project_name_domain_id'") Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 21:01:10.062865 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application yield resource Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise self.value Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 21:01:10.064132 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application session.commit() Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.flush() Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application compat.raise_( Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 21:01:10.065371 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 21:01:10.066567 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application conn.query(q) Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application result.read() Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 21:01:10.067650 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-547294647' for key 'project.ixu_project_name_domain_id'") Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [parameters: {'id': '922f714e62024245b36f217d43fbbbef', 'name': 'tempest-project-dup-547294647', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:10.068659 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-547294647 at domain ID default. Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:01:10.069680 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-547294647, project id 922f714e62024245b36f217d43fbbbef. Sep 10 21:01:10.077546 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:01:10.080649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:10.083377 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1878/3761] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:01:09 2022] POST /identity/v3/projects => generated 284 bytes in 316 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 21:01:10.094068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e15d-470a-4cea-ad9c-686b739397e2 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:10.109002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9299319b37a84e0fae68c8176ba28df5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a68af4d0aa75461ca61fa59b83e7f029', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:10.110961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:10.111504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:10.112033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] PATH_INFO: `/v3/projects/fcf79f86857842f7b1b4be8682d59751` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:10.122844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fcf79f86857842f7b1b4be8682d59751)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:10.125409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:10.276688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf79f86857842f7b1b4be8682d59751', 'request_id': 'req-0d8e87c6-4245-4502-9b8f-adf9976e1862'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.294055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf79f86857842f7b1b4be8682d59751', 'request_id': 'req-0d8e87c6-4245-4502-9b8f-adf9976e1862'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.313297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf79f86857842f7b1b4be8682d59751', 'request_id': 'req-0d8e87c6-4245-4502-9b8f-adf9976e1862'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.395137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] The token cache is being invalidate because project fcf79f86857842f7b1b4be8682d59751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:10.395439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.396797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d8e87c6-4245-4502-9b8f-adf9976e1862 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.400032 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1884/3762] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:10 2022] DELETE /identity/v3/projects/fcf79f86857842f7b1b4be8682d59751 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:10.405550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:10.408252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:10.560924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab18f42-b8d0-42bb-a051-ffc935247541 tempest-ProjectsNegativeStaticTestJSON-1229395077 tempest-ProjectsNegativeStaticTestJSON-1229395077-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:10.652504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:10.718509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:10.719165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:10.719691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:10.720141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] PATH_INFO: `/v3/users/ad76ca5f129443a4a9ef111c2b7cb0ca` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:10.721583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad76ca5f129443a4a9ef111c2b7cb0ca)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:10.730712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:10.901895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad76ca5f129443a4a9ef111c2b7cb0ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.932954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad76ca5f129443a4a9ef111c2b7cb0ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.934174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad76ca5f129443a4a9ef111c2b7cb0ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.950437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad76ca5f129443a4a9ef111c2b7cb0ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:10.951537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad76ca5f129443a4a9ef111c2b7cb0ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:11.010935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:11.011275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:11.011531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:11.015207 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1879/3763] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:10 2022] DELETE /identity/v3/users/ad76ca5f129443a4a9ef111c2b7cb0ca => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:11.059150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:11.060172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:11.230254 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1885/3764] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:11.248851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:11.254534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:11.479239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6d932-5ce4-4f8b-8de7-e5d2aeaf41f5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:11.497283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:11.498286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:11.498755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:11.501449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:11.523147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:11.525074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:11.545883 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1880/3765] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:13.941498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:13.943771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:13.968822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26679ce-9181-4024-b953-f603fd60bac3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:13.982960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:13.983788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:13.984237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:13.984672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] PATH_INFO: `/v3/projects/acfe8b124c71401580f338b2438bff66` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:13.987044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acfe8b124c71401580f338b2438bff66)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:13.989299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.101054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:14.115115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acfe8b124c71401580f338b2438bff66', 'request_id': 'req-0b475f5e-509f-450e-9d17-63e93fa24d32'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.125119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acfe8b124c71401580f338b2438bff66', 'request_id': 'req-0b475f5e-509f-450e-9d17-63e93fa24d32'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.139080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acfe8b124c71401580f338b2438bff66', 'request_id': 'req-0b475f5e-509f-450e-9d17-63e93fa24d32'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.165561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] The token cache is being invalidate because project acfe8b124c71401580f338b2438bff66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:14.166262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.167218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b475f5e-509f-450e-9d17-63e93fa24d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.169625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1886/3766] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:13 2022] DELETE /identity/v3/projects/acfe8b124c71401580f338b2438bff66 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:14.174018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:14.193735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34487-a9d6-47b7-9d27-906bc7f17e2b service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:14.281594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:14.293797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:14.294739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.295214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.295721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] PATH_INFO: `/v3/users/9299319b37a84e0fae68c8176ba28df5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:14.299869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9299319b37a84e0fae68c8176ba28df5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:14.302133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:14.384110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9299319b37a84e0fae68c8176ba28df5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.402223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9299319b37a84e0fae68c8176ba28df5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.403026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9299319b37a84e0fae68c8176ba28df5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.437815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9299319b37a84e0fae68c8176ba28df5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.440049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9299319b37a84e0fae68c8176ba28df5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:14.487103 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1881/3767] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:14 2022] DELETE /identity/v3/users/9299319b37a84e0fae68c8176ba28df5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:14.998898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:14.999421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:14.999694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.020931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:15.022384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.089197 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1887/3768] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:15.104992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.107179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.261787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e3224-6c58-47e4-bfbf-25a9e02737f4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.302668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55610312-0ec0-4359-9232-c69d448a4ceb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.302668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610312-0ec0-4359-9232-c69d448a4ceb service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.302668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610312-0ec0-4359-9232-c69d448a4ceb service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.303619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:15.312656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.321058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610312-0ec0-4359-9232-c69d448a4ceb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.352831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55610312-0ec0-4359-9232-c69d448a4ceb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.354507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55610312-0ec0-4359-9232-c69d448a4ceb service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.394605 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1882/3769] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:15.482149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05525ea-82fd-4d49-97f1-24e4437aec5f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:15.500009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:15.500862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.501366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.501765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] PATH_INFO: `/v3/projects/a68af4d0aa75461ca61fa59b83e7f029` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.503060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68af4d0aa75461ca61fa59b83e7f029)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:15.504980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:15.624743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68af4d0aa75461ca61fa59b83e7f029', 'request_id': 'req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.651154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68af4d0aa75461ca61fa59b83e7f029', 'request_id': 'req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.671181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68af4d0aa75461ca61fa59b83e7f029', 'request_id': 'req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.700229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] The token cache is being invalidate because project a68af4d0aa75461ca61fa59b83e7f029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:15.701889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.705346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a2a61782-3cc5-46e9-bf73-8d2d8c32f08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:15.708164 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1888/3770] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:15 2022] DELETE /identity/v3/projects/a68af4d0aa75461ca61fa59b83e7f029 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:15.715294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867f34b-d9d1-4877-96f6-b843671ab1fb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.715618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867f34b-d9d1-4877-96f6-b843671ab1fb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.715861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867f34b-d9d1-4877-96f6-b843671ab1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.744267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6867f34b-d9d1-4877-96f6-b843671ab1fb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:15.745391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6867f34b-d9d1-4877-96f6-b843671ab1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:15.898263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1883/3771] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:15.908559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd616-daee-4574-8621-3be006fb51e8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:15.908854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd616-daee-4574-8621-3be006fb51e8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:15.909135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd616-daee-4574-8621-3be006fb51e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:15.958181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7cecd616-daee-4574-8621-3be006fb51e8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:15.977756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7cecd616-daee-4574-8621-3be006fb51e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:16.003285 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1889/3772] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:16.008634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867f34b-d9d1-4877-96f6-b843671ab1fb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.021085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:16.051630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867f34b-d9d1-4877-96f6-b843671ab1fb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.105612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.112537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.115143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.116470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.118125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:16.118603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.122323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.159516 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1884/3773] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:16.160507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd616-daee-4574-8621-3be006fb51e8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.165998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:16.178847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd616-daee-4574-8621-3be006fb51e8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.198862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.199490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.199771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.200045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.200974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.202931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.284138 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1890/3774] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:16 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:16.290138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.307838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c439cb-0bbe-4627-8d31-9084fe45a5f3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.493919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.495286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.497452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.497916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.498780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.501761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.584975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.600571 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1885/3775] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:16 2022] POST /identity/v3/users => generated 297 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:16.601666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe2df8-2dab-4e93-bc69-c3f4bdd1fc3e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.623870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.624865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.626057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.626581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.628861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.637348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.663685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.664807 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1891/3776] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:16.682027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f12b19-cd63-4462-8d6c-97de8fe75c51 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:16.700166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:16.702331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:16.702888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:16.707881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] PATH_INFO: `/v3/projects/c3350b5b6bb24983a589a63de6077ba7/users/ef311c6d70fc48bea0a466d06b8b3c2f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:16.761046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3350b5b6bb24983a589a63de6077ba7, user_id=ef311c6d70fc48bea0a466d06b8b3c2f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:16.768724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:16.823396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1886/3777] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:16 2022] PUT /identity/v3/projects/c3350b5b6bb24983a589a63de6077ba7/users/ef311c6d70fc48bea0a466d06b8b3c2f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:16.824688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:16.843310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e791d7f-a2ae-4f83-9eb0-02541c22f6dc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:17.075553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9a8ac-f9d3-4127-a627-663f0c4b6955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:17.076273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9a8ac-f9d3-4127-a627-663f0c4b6955 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.076455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9a8ac-f9d3-4127-a627-663f0c4b6955 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.076955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9a8ac-f9d3-4127-a627-663f0c4b6955 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.077786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9a8ac-f9d3-4127-a627-663f0c4b6955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:17.079678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9a8ac-f9d3-4127-a627-663f0c4b6955 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:17.138346 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1892/3778] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:17.139643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:17.168521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25439-74d0-4a0e-b9a6-11b77da8cdd4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:17.191693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:17.193185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.193476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.193907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] PATH_INFO: `/v3/projects/c3350b5b6bb24983a589a63de6077ba7/users/ef311c6d70fc48bea0a466d06b8b3c2f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.200151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3350b5b6bb24983a589a63de6077ba7, user_id=ef311c6d70fc48bea0a466d06b8b3c2f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:17.207487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:17.248158 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1887/3779] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:17 2022] PUT /identity/v3/projects/c3350b5b6bb24983a589a63de6077ba7/users/ef311c6d70fc48bea0a466d06b8b3c2f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:17.249935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e81ce-4a24-436d-ae72-e11aedb1d1a4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.250903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e81ce-4a24-436d-ae72-e11aedb1d1a4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.251196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e81ce-4a24-436d-ae72-e11aedb1d1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.302424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-582e81ce-4a24-436d-ae72-e11aedb1d1a4 None None] MFA Rules not processed for user `ef311c6d70fc48bea0a466d06b8b3c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:17.304653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-582e81ce-4a24-436d-ae72-e11aedb1d1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:17.504969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1893/3780] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:17.510059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:17.643586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f201-41bd-48d4-96ff-1706cac6d1f8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:17.649693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:17.650375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.653227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.653681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.654476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:17.657686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:17.708730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e81ce-4a24-436d-ae72-e11aedb1d1a4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:17.713491 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1888/3781] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:17 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:17.723850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e81ce-4a24-436d-ae72-e11aedb1d1a4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:17.806917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:17.818497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:17.819177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.819486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.819870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.820510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:17.822516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:17.848945 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1894/3782] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:17 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:17.853532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:17.865633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f9d66-0a49-4b42-bf48-fc8b07b27ec9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:17.872555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:17.873197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.873486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.873759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.874543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:17.876805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:17.896808 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1889/3783] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:17.900691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:17.911284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2b294-4d1a-44e6-a470-e4bcdf25822b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:17.915990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf86306-3e80-4d7c-bc77-3766fbb08b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:17.916730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf86306-3e80-4d7c-bc77-3766fbb08b3b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.917016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf86306-3e80-4d7c-bc77-3766fbb08b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.917318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf86306-3e80-4d7c-bc77-3766fbb08b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.918176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf86306-3e80-4d7c-bc77-3766fbb08b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:17.919897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf86306-3e80-4d7c-bc77-3766fbb08b3b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:17.933859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1895/3784] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:17.937125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:17.949646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69ba6-0afa-45b2-9244-cad81a848ff0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:17.956335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9715c5-0704-4864-878f-c33ba3aa370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:17.957037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9715c5-0704-4864-878f-c33ba3aa370e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:17.957257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9715c5-0704-4864-878f-c33ba3aa370e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:17.957658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9715c5-0704-4864-878f-c33ba3aa370e admin admin] PATH_INFO: `/v3/projects/97ac9ad64ff441d199608cf45090f8c8/users/1b4f99d5b79644a1a2d9ed0c540242a3/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:17.984313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:17.988883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9715c5-0704-4864-878f-c33ba3aa370e admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ac9ad64ff441d199608cf45090f8c8, user_id=1b4f99d5b79644a1a2d9ed0c540242a3, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:17.992299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9715c5-0704-4864-878f-c33ba3aa370e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:18.008310 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1890/3785] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:17 2022] PUT /identity/v3/projects/97ac9ad64ff441d199608cf45090f8c8/users/1b4f99d5b79644a1a2d9ed0c540242a3/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:18.091706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:18.092265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:18.092379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:18.114738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:18.115225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:18.115510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:18.127648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] MFA Rules not processed for user `1b4f99d5b79644a1a2d9ed0c540242a3`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:18.128862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:18.135395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:18.136595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:18.201687 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1891/3786] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:01:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:18.208524 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1896/3787] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:18.215857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:18.217369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:18.222747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:18.224705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:18.294790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d064c0b-4f31-4665-a3e1-922d00932758 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:18.299175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:18.300034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:18.301413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:18.301413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:18.304343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ab096-5e83-4088-afaf-e2a41ed3b3ad None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:18.316629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabed9d4-761a-4609-b5ca-580182613c96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:18.317262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabed9d4-761a-4609-b5ca-580182613c96 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:18.317546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabed9d4-761a-4609-b5ca-580182613c96 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:18.317863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabed9d4-761a-4609-b5ca-580182613c96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:18.330634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabed9d4-761a-4609-b5ca-580182613c96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:18.331722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabed9d4-761a-4609-b5ca-580182613c96 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:18.343114 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1892/3788] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:18.388523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:18.392743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:18.407506 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1897/3789] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:19.194746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3decf4-369d-4246-8733-766362006ab2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.195427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3decf4-369d-4246-8733-766362006ab2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.195427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3decf4-369d-4246-8733-766362006ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.241009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9c3decf4-369d-4246-8733-766362006ab2 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:19.243934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9c3decf4-369d-4246-8733-766362006ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:19.253129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1893/3790] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:01:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:19.271640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:19.288348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8991de-9bc0-4d12-a4fc-800b0d47fbc5 service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:19.303018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3decf4-369d-4246-8733-766362006ab2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:19.303249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e96a4-1e01-4da9-bdf4-cd8df7f8f09e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:19.304082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e96a4-1e01-4da9-bdf4-cd8df7f8f09e service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.304517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e96a4-1e01-4da9-bdf4-cd8df7f8f09e service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.305059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e96a4-1e01-4da9-bdf4-cd8df7f8f09e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.318539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:19.322441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e96a4-1e01-4da9-bdf4-cd8df7f8f09e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:19.324657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e96a4-1e01-4da9-bdf4-cd8df7f8f09e service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:19.336107 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1898/3791] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:19.411739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3decf4-369d-4246-8733-766362006ab2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:19.417119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:19.418093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.418508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.418906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] PATH_INFO: `/v3/users/ef311c6d70fc48bea0a466d06b8b3c2f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.420682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef311c6d70fc48bea0a466d06b8b3c2f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:19.423209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:19.482396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:19.542356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef311c6d70fc48bea0a466d06b8b3c2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.570243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef311c6d70fc48bea0a466d06b8b3c2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.571570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef311c6d70fc48bea0a466d06b8b3c2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.583658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef311c6d70fc48bea0a466d06b8b3c2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.584402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef311c6d70fc48bea0a466d06b8b3c2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:19.616307 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1894/3792] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:19 2022] DELETE /identity/v3/users/ef311c6d70fc48bea0a466d06b8b3c2f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:19.624427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.624636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.624857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.649396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:19.650720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:19.745188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1899/3793] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:19.752699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:19.754147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:19.871372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94973f-a862-44f2-b98f-c72b31be428f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:19.877120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:19.878070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:19.878396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:19.881367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:19.896357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:19.899674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:19.913090 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1895/3794] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:20.392583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:20.394096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:20.415788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734d6a9-56ac-42ff-a857-c0fda4a6f876 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:20.430412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:20.430955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.431322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.431973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] PATH_INFO: `/v3/projects/c3350b5b6bb24983a589a63de6077ba7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.433615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3350b5b6bb24983a589a63de6077ba7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:20.437788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:20.519066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3350b5b6bb24983a589a63de6077ba7', 'request_id': 'req-f70ad206-6ee6-41b8-b0fe-a561302f5115'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.528072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3350b5b6bb24983a589a63de6077ba7', 'request_id': 'req-f70ad206-6ee6-41b8-b0fe-a561302f5115'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.538082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3350b5b6bb24983a589a63de6077ba7', 'request_id': 'req-f70ad206-6ee6-41b8-b0fe-a561302f5115'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.554172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] The token cache is being invalidate because project c3350b5b6bb24983a589a63de6077ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:20.554469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.555112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.556799 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1900/3795] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:20 2022] DELETE /identity/v3/projects/c3350b5b6bb24983a589a63de6077ba7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:20.563107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:20.579056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd093f-1797-4cbe-9f8a-f28b1dc5622e service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:20.648655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:20.650255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:20.650725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:20.651171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] PATH_INFO: `/v3/users/1b4f99d5b79644a1a2d9ed0c540242a3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:20.652604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4f99d5b79644a1a2d9ed0c540242a3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:20.654749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:20.709624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:20.712816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4f99d5b79644a1a2d9ed0c540242a3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.722670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4f99d5b79644a1a2d9ed0c540242a3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.724618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4f99d5b79644a1a2d9ed0c540242a3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.736879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4f99d5b79644a1a2d9ed0c540242a3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.737719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-30f14c36-91b7-4080-861c-380ceeb3576b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4f99d5b79644a1a2d9ed0c540242a3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:20.773080 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1896/3796] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:20 2022] DELETE /identity/v3/users/1b4f99d5b79644a1a2d9ed0c540242a3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:21.396768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.398440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.414661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ad206-6ee6-41b8-b0fe-a561302f5115 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.480298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.481015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.481374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.481741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] PATH_INFO: `/v3/projects/97ac9ad64ff441d199608cf45090f8c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.483264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ac9ad64ff441d199608cf45090f8c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.485929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.512241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:21.561689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ac9ad64ff441d199608cf45090f8c8', 'request_id': 'req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.570349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ac9ad64ff441d199608cf45090f8c8', 'request_id': 'req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.579984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ac9ad64ff441d199608cf45090f8c8', 'request_id': 'req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.600536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] The token cache is being invalidate because project 97ac9ad64ff441d199608cf45090f8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:21.601057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.602721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b011fcd5-ade7-4b03-afe7-d8e7677cc1ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:21.604960 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1901/3797] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:21 2022] DELETE /identity/v3/projects/97ac9ad64ff441d199608cf45090f8c8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:21.609717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4180e2-7d76-4650-a88b-8971cf83a36f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.610288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4180e2-7d76-4650-a88b-8971cf83a36f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.610572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4180e2-7d76-4650-a88b-8971cf83a36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.639320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ca4180e2-7d76-4650-a88b-8971cf83a36f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:21.642298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ca4180e2-7d76-4650-a88b-8971cf83a36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.743535 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1897/3798] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:21.745484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4dcb-79b0-4b07-8f25-7354a74114c1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.745773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4dcb-79b0-4b07-8f25-7354a74114c1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.746076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4dcb-79b0-4b07-8f25-7354a74114c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.768404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4a4a4dcb-79b0-4b07-8f25-7354a74114c1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:21.769622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4a4a4dcb-79b0-4b07-8f25-7354a74114c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:21.779148 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1902/3799] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:21.785047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4180e2-7d76-4650-a88b-8971cf83a36f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.796377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4180e2-7d76-4650-a88b-8971cf83a36f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.803216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.803828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.808858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.809471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.810398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:21.810789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.813493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.834703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4dcb-79b0-4b07-8f25-7354a74114c1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.839330 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1898/3800] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:01:21.848099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4dcb-79b0-4b07-8f25-7354a74114c1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:21.876953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:21.877700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:21.878339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:21.878642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:21.879455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:21.882937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:21.941966 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1903/3801] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:21 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:21.946937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:21.962699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781221b-44e0-4762-b466-5ae44c5269e6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.050508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.052020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.052520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.052986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.053816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.057090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.086658 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1899/3802] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:21 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:22.091146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.105953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c394c-591d-4079-b828-1abe5fb8f661 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.113362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.114788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.115215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.115883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.117099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.118985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.133877 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1904/3803] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:22.138602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.151415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df43ad-0eab-405c-bd5b-5067dd25492f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.161431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ed02c-7b48-4858-ad31-d0fe00e538ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.162588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ed02c-7b48-4858-ad31-d0fe00e538ab admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.163065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ed02c-7b48-4858-ad31-d0fe00e538ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.163712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ed02c-7b48-4858-ad31-d0fe00e538ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.164651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ed02c-7b48-4858-ad31-d0fe00e538ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.166583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ed02c-7b48-4858-ad31-d0fe00e538ab admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.179502 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1900/3804] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:22.184805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.201182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dff4a-23f2-4968-8761-afd4c1904d23 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.209189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.210281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.210766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.211258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] PATH_INFO: `/v3/projects/572b81304f694b7fac5be063ea9d0a57/users/18412636ce88442eac003e796e50f7c2/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.233667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] RBAC: Authorizing `identity:create_grant(project_id=572b81304f694b7fac5be063ea9d0a57, user_id=18412636ce88442eac003e796e50f7c2, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.237943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.264081 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1905/3805] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:22 2022] PUT /identity/v3/projects/572b81304f694b7fac5be063ea9d0a57/users/18412636ce88442eac003e796e50f7c2/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:22.267876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91d360-b5e1-4d95-af11-651078c07f04 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.268268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91d360-b5e1-4d95-af11-651078c07f04 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.268798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91d360-b5e1-4d95-af11-651078c07f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.311857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6f91d360-b5e1-4d95-af11-651078c07f04 None None] MFA Rules not processed for user `18412636ce88442eac003e796e50f7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:22.314339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6f91d360-b5e1-4d95-af11-651078c07f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:22.332737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:22.415392 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1901/3806] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:22.419781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.498581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0969e34-de62-4cc8-aa77-c0deb667131a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.505641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.506595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.506961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.507167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.507856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.510874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.553845 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1906/3807] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:22.555566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91d360-b5e1-4d95-af11-651078c07f04 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.570725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91d360-b5e1-4d95-af11-651078c07f04 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.671146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.673405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.674277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.674829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.675773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.679490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.736561 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1902/3808] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:22 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:22.740721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.752232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b6a57-b64e-4784-b1d3-1355f142795b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.785384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.790604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.790904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.791188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.791884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.793474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.832138 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1907/3809] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:22.833414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.873621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea9ae5-32a6-4e0c-8b1c-3041fdd7a488 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.888870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394aedb-ea0f-421b-84a8-1a7f4d43f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.890462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394aedb-ea0f-421b-84a8-1a7f4d43f491 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.890777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394aedb-ea0f-421b-84a8-1a7f4d43f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.891294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394aedb-ea0f-421b-84a8-1a7f4d43f491 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.892008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394aedb-ea0f-421b-84a8-1a7f4d43f491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.893739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394aedb-ea0f-421b-84a8-1a7f4d43f491 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.924520 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1903/3810] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:22.925650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:22.944575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026fa28-efb2-4b45-bd51-62cb6f7426bd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:22.964364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:22.965109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:22.965355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:22.968433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] PATH_INFO: `/v3/projects/a321d006fce44d88b7abbd392cde861c/users/79af725123844f66883e6ebdca944f44/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:22.992029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a321d006fce44d88b7abbd392cde861c, user_id=79af725123844f66883e6ebdca944f44, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:22.994838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:22.998162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:23.013685 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1908/3811] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:22 2022] PUT /identity/v3/projects/a321d006fce44d88b7abbd392cde861c/users/79af725123844f66883e6ebdca944f44/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:23.025530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5b950-c183-4777-a6d4-fc5c576d1ea0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.026017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5b950-c183-4777-a6d4-fc5c576d1ea0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.026395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5b950-c183-4777-a6d4-fc5c576d1ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.073068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-49a5b950-c183-4777-a6d4-fc5c576d1ea0 None None] MFA Rules not processed for user `79af725123844f66883e6ebdca944f44`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:23.075257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-49a5b950-c183-4777-a6d4-fc5c576d1ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.184308 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1904/3812] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:23.193368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.195809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:23.275434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426ef8-7530-4a2a-8b55-72c0c1ebe8fa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.280107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.280408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.280674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.282756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.400245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.403420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.434156 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1909/3813] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:23.648088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5b950-c183-4777-a6d4-fc5c576d1ea0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:23.670030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5b950-c183-4777-a6d4-fc5c576d1ea0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:23.678020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e6013-207b-4f86-a510-30ad717a913c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:23.680802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e6013-207b-4f86-a510-30ad717a913c service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:23.681264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e6013-207b-4f86-a510-30ad717a913c service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:23.681719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e6013-207b-4f86-a510-30ad717a913c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:23.701226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e6013-207b-4f86-a510-30ad717a913c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:23.705696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e6013-207b-4f86-a510-30ad717a913c service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:23.719984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1905/3814] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:25.872557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:25.972524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5185c-879a-4664-b5a7-399d81231083 service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:25.995709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:25.995709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:25.995709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:25.996131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] PATH_INFO: `/v3/users/18412636ce88442eac003e796e50f7c2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:25.998403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18412636ce88442eac003e796e50f7c2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.005568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.088019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18412636ce88442eac003e796e50f7c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:26.095857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:26.108339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18412636ce88442eac003e796e50f7c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:26.121713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18412636ce88442eac003e796e50f7c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:26.137430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18412636ce88442eac003e796e50f7c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:26.139271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18412636ce88442eac003e796e50f7c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:26.151749 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1910/3815] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:25 2022] DELETE /identity/v3/users/18412636ce88442eac003e796e50f7c2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:26.153472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.154041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.154041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.183874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:26.185107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:26.192256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:26.336181 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1906/3816] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:26.352047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:26.354556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:26.472461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c64f-5f04-440f-bc62-4e0c4cbf44e4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:26.478471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:26.479194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:26.479473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:26.479746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:26.491149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:26.492747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:26.503735 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1911/3817] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:27.501239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:27.502879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:27.518501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4444046-a71a-4c98-b831-50f9afb51010 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:27.525083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:27.526074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:27.526573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:27.526893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] PATH_INFO: `/v3/projects/572b81304f694b7fac5be063ea9d0a57` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:27.528777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] RBAC: Authorizing `identity:delete_project(project_id=572b81304f694b7fac5be063ea9d0a57)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:27.532533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:27.622305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572b81304f694b7fac5be063ea9d0a57', 'request_id': 'req-108e0693-6e33-4a94-9f6b-c60040b9683c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.629548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572b81304f694b7fac5be063ea9d0a57', 'request_id': 'req-108e0693-6e33-4a94-9f6b-c60040b9683c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.639274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572b81304f694b7fac5be063ea9d0a57', 'request_id': 'req-108e0693-6e33-4a94-9f6b-c60040b9683c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.652032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] The token cache is being invalidate because project 572b81304f694b7fac5be063ea9d0a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:27.652513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.653174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.654793 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1907/3818] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:27 2022] DELETE /identity/v3/projects/572b81304f694b7fac5be063ea9d0a57 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:27.660599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:27.671234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc700e92-30b8-416f-8ce5-effdfe9b7194 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:27.741486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:27.742157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:27.742565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:27.742770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] PATH_INFO: `/v3/users/79af725123844f66883e6ebdca944f44` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:27.743875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79af725123844f66883e6ebdca944f44)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:27.745844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:27.799345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79af725123844f66883e6ebdca944f44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.810820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79af725123844f66883e6ebdca944f44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.818212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79af725123844f66883e6ebdca944f44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.824122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:27.830841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79af725123844f66883e6ebdca944f44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.831917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab77a5d5-fe01-4a5f-add0-eb4fbc1c1fa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79af725123844f66883e6ebdca944f44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:27.841179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1912/3819] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:27 2022] DELETE /identity/v3/users/79af725123844f66883e6ebdca944f44 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:28.267194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:28.269052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:28.290088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e0693-6e33-4a94-9f6b-c60040b9683c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:28.380177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:28.380965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.381438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.382427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] PATH_INFO: `/v3/projects/a321d006fce44d88b7abbd392cde861c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.383881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a321d006fce44d88b7abbd392cde861c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:28.386252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:28.431367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:28.444911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a321d006fce44d88b7abbd392cde861c', 'request_id': 'req-951db940-6946-40ba-b680-8b325c7a18e9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:28.451731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a321d006fce44d88b7abbd392cde861c', 'request_id': 'req-951db940-6946-40ba-b680-8b325c7a18e9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:28.460719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a321d006fce44d88b7abbd392cde861c', 'request_id': 'req-951db940-6946-40ba-b680-8b325c7a18e9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:28.472799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] The token cache is being invalidate because project a321d006fce44d88b7abbd392cde861c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:28.473171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:28.474135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951db940-6946-40ba-b680-8b325c7a18e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:28.475806 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1908/3820] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:28 2022] DELETE /identity/v3/projects/a321d006fce44d88b7abbd392cde861c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:28.481601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5dc91-9321-414e-914d-79976c0b06d3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.481975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5dc91-9321-414e-914d-79976c0b06d3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.482251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5dc91-9321-414e-914d-79976c0b06d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.502913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d3c5dc91-9321-414e-914d-79976c0b06d3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:28.504039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3c5dc91-9321-414e-914d-79976c0b06d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:28.566823 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1913/3821] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:28.575118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02d66-7ba3-46b5-abaa-1ca62ed5c848 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.575627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02d66-7ba3-46b5-abaa-1ca62ed5c848 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.575729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e02d66-7ba3-46b5-abaa-1ca62ed5c848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.596835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-65e02d66-7ba3-46b5-abaa-1ca62ed5c848 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:28.597924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-65e02d66-7ba3-46b5-abaa-1ca62ed5c848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:28.606494 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1909/3822] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:28.611179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5dc91-9321-414e-914d-79976c0b06d3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:28.620818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5dc91-9321-414e-914d-79976c0b06d3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:28.625628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:28.626398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.626678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.626972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.627597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:28.627899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:28.629474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:28.641193 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1914/3823] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:28.645757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e02d66-7ba3-46b5-abaa-1ca62ed5c848 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:28.655692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e02d66-7ba3-46b5-abaa-1ca62ed5c848 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:28.661481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:28.662212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.662556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.662894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.663538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:28.665435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:28.693667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1910/3824] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:28 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:28.697904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:28.709641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da07fc-b054-47cf-bcd9-99026d811ab5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:28.792201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:28.792843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.793236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.793577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.794329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:28.796784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:28.827266 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1915/3825] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:28 2022] POST /identity/v3/users => generated 280 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:28.831596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:28.843800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b1a3d-b4df-4a99-88d3-244042f14a0c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:28.850478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:28.850698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.850966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.851247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.851903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:28.853564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:28.866042 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1911/3826] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:28.869804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:28.884105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afd9ed-0759-45e8-8753-f71a2e332df3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:28.891302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516b238-4aa6-4582-80aa-f8bd3b778607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:28.892164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516b238-4aa6-4582-80aa-f8bd3b778607 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.892473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516b238-4aa6-4582-80aa-f8bd3b778607 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.892804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516b238-4aa6-4582-80aa-f8bd3b778607 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.893453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516b238-4aa6-4582-80aa-f8bd3b778607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:28.895317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516b238-4aa6-4582-80aa-f8bd3b778607 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:28.914089 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1916/3827] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:28.918473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:28.931452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75da33-344a-4a0c-a9b9-5cc1a8fd403c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:28.947125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:28.948059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:28.948153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:28.948450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] PATH_INFO: `/v3/projects/216c6a6a58a0464bbe59e5def45f5c39/users/382a6be6d42149e0b8d084da89e75eb8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:28.967112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] RBAC: Authorizing `identity:create_grant(project_id=216c6a6a58a0464bbe59e5def45f5c39, user_id=382a6be6d42149e0b8d084da89e75eb8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:28.975952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:28.994604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1912/3828] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:28 2022] PUT /identity/v3/projects/216c6a6a58a0464bbe59e5def45f5c39/users/382a6be6d42149e0b8d084da89e75eb8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:29.007650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9e98-409a-4e84-a374-35695a9bb7b4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:29.007990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9e98-409a-4e84-a374-35695a9bb7b4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:29.008209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9e98-409a-4e84-a374-35695a9bb7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:29.171720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-52ae9e98-409a-4e84-a374-35695a9bb7b4 None None] MFA Rules not processed for user `382a6be6d42149e0b8d084da89e75eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:29.173534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-52ae9e98-409a-4e84-a374-35695a9bb7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:29.219262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:29.245712 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1917/3829] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:29.255160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:29.323096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7f24-83b8-4cf0-b2a1-68a6eea5baef admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:29.328789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:29.329529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:29.329809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:29.330111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] PATH_INFO: `/v3/users/382a6be6d42149e0b8d084da89e75eb8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:29.331748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382a6be6d42149e0b8d084da89e75eb8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:29.333705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:29.392301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382a6be6d42149e0b8d084da89e75eb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:29.400093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382a6be6d42149e0b8d084da89e75eb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:29.400819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382a6be6d42149e0b8d084da89e75eb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:29.411421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382a6be6d42149e0b8d084da89e75eb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:29.412920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382a6be6d42149e0b8d084da89e75eb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:29.414710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:29.453991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1913/3830] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:29 2022] DELETE /identity/v3/users/382a6be6d42149e0b8d084da89e75eb8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:29.458579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:29.458869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:29.459142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:29.492325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:29.501252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:29.623323 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1918/3831] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:29.639082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:29.641014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:29.749797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06c216-4e1b-4b02-9492-9177c40bcf80 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:29.755703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda4471-00d4-4742-b850-1122030afe11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:29.756464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4471-00d4-4742-b850-1122030afe11 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:29.756843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4471-00d4-4742-b850-1122030afe11 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:29.757214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4471-00d4-4742-b850-1122030afe11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:29.768128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda4471-00d4-4742-b850-1122030afe11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:29.770325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda4471-00d4-4742-b850-1122030afe11 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:29.784187 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1914/3832] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:30.317541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:30.319194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:30.333520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bff62-d0c7-44d0-98ac-b3df3ba457de None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:30.345582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:30.346267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:30.346670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:30.347177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] PATH_INFO: `/v3/projects/216c6a6a58a0464bbe59e5def45f5c39` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:30.348812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=216c6a6a58a0464bbe59e5def45f5c39)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:30.352524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:30.449670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216c6a6a58a0464bbe59e5def45f5c39', 'request_id': 'req-6f667a11-2601-4a5a-bcb5-a68e90269654'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:30.464412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216c6a6a58a0464bbe59e5def45f5c39', 'request_id': 'req-6f667a11-2601-4a5a-bcb5-a68e90269654'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:30.479923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216c6a6a58a0464bbe59e5def45f5c39', 'request_id': 'req-6f667a11-2601-4a5a-bcb5-a68e90269654'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:30.497772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] The token cache is being invalidate because project 216c6a6a58a0464bbe59e5def45f5c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:30.498613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:30.500937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f667a11-2601-4a5a-bcb5-a68e90269654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:30.503137 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1919/3833] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:30 2022] DELETE /identity/v3/projects/216c6a6a58a0464bbe59e5def45f5c39 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:30.508334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19257126-ad3b-4e4f-8e64-eee56821b0a0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:30.508895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19257126-ad3b-4e4f-8e64-eee56821b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:30.512535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19257126-ad3b-4e4f-8e64-eee56821b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:30.537054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-19257126-ad3b-4e4f-8e64-eee56821b0a0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:30.539238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-19257126-ad3b-4e4f-8e64-eee56821b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:30.641512 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1915/3834] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:30.649645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc19ed-2c16-4a21-a303-278999b1a74e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:30.650322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc19ed-2c16-4a21-a303-278999b1a74e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:30.650853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc19ed-2c16-4a21-a303-278999b1a74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:30.686364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-63dc19ed-2c16-4a21-a303-278999b1a74e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:30.688101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-63dc19ed-2c16-4a21-a303-278999b1a74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:30.706775 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1920/3835] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:30.707126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19257126-ad3b-4e4f-8e64-eee56821b0a0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:30.720429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19257126-ad3b-4e4f-8e64-eee56821b0a0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:30.728180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:30.731530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:30.732019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:30.732467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:30.733389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:30.733869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:30.738963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:30.755745 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1916/3836] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:30.760290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc19ed-2c16-4a21-a303-278999b1a74e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:30.774552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc19ed-2c16-4a21-a303-278999b1a74e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:30.780214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:30.782239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:30.782532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:30.782850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:30.784144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:30.786025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:30.822243 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1921/3837] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:30 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:30.826243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:30.840159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b580f-092d-48c8-8ea0-71ab1ff663c8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:30.928204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:30.929493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:30.929786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:30.930107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:30.931476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:30.934119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:30.977953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:30.986826 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1917/3838] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:30 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:30.991957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.005734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c49a5-c25c-4e92-9874-8f40c85cab44 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.011532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.012809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.013325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.013671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.014418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.017309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.032136 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1922/3839] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:31.036207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.049371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9a2cb-f74d-46ae-907a-703ee5367829 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.056744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ba3e7-6be6-4f40-bdf4-3f9b3cfd97f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.057555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba3e7-6be6-4f40-bdf4-3f9b3cfd97f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.057868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba3e7-6be6-4f40-bdf4-3f9b3cfd97f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.058164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ba3e7-6be6-4f40-bdf4-3f9b3cfd97f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.058819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ba3e7-6be6-4f40-bdf4-3f9b3cfd97f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.061753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ba3e7-6be6-4f40-bdf4-3f9b3cfd97f6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.079346 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1918/3840] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:31.083786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.097576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e3eb5-996e-4c63-b535-86d6f418d10b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.106129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.107107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.107426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.107904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] PATH_INFO: `/v3/projects/e196f8b976a64c70b73768215f312943/users/9bccaed010664129961e499b613a416d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.142259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e196f8b976a64c70b73768215f312943, user_id=9bccaed010664129961e499b613a416d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.145815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.181244 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1923/3841] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:31 2022] PUT /identity/v3/projects/e196f8b976a64c70b73768215f312943/users/9bccaed010664129961e499b613a416d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:31.193638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c74d9-db36-4c46-94ff-44aed7131dcc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.194014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c74d9-db36-4c46-94ff-44aed7131dcc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.194217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c74d9-db36-4c46-94ff-44aed7131dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.229216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f56c74d9-db36-4c46-94ff-44aed7131dcc None None] MFA Rules not processed for user `9bccaed010664129961e499b613a416d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:31.230548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f56c74d9-db36-4c46-94ff-44aed7131dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:31.324696 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1919/3842] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:31.325877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.336812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:31.410645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e2b8f-82aa-49e2-9568-bee7640edf48 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.416535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.417677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.417677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.417987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.418912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.421668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.463756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c74d9-db36-4c46-94ff-44aed7131dcc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.475054 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1924/3843] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:31 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:31.485064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c74d9-db36-4c46-94ff-44aed7131dcc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.548990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.549737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.550723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.550723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.551356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.564835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.592010 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1920/3844] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:31 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:31.596622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.608536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5b698-b96b-43cb-abf1-86de3549fe4d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.614540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.614752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.615101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.615441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.616859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.618403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.633609 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1925/3845] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:31.637686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.649151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04111c31-1d12-4492-b413-cd53835a4a27 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.656301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.657674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.658194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.658538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] PATH_INFO: `/v3/projects/4acba97f54454c5fafbbd04d38dbc5e9/users/e812a3299d954eb1b3a060df26bcf803/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.677474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4acba97f54454c5fafbbd04d38dbc5e9, user_id=e812a3299d954eb1b3a060df26bcf803, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.680719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.697430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1921/3846] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:31 2022] PUT /identity/v3/projects/4acba97f54454c5fafbbd04d38dbc5e9/users/e812a3299d954eb1b3a060df26bcf803/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:31.701565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.715818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f21ba-3334-48b4-b047-ab8f20d484e3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.803710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ec6d6-a76c-4fed-8424-22778bae2eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.804338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec6d6-a76c-4fed-8424-22778bae2eed admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.804634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec6d6-a76c-4fed-8424-22778bae2eed admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.805134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec6d6-a76c-4fed-8424-22778bae2eed admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.805701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec6d6-a76c-4fed-8424-22778bae2eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.809709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec6d6-a76c-4fed-8424-22778bae2eed admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.841140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1926/3847] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:31.846063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:31.863338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd350cd0-7f93-4f85-a0ae-c7097bd199fa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:31.876712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6289c7-2659-464e-bdeb-8952ba6f8c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:31.877328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6289c7-2659-464e-bdeb-8952ba6f8c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.877745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6289c7-2659-464e-bdeb-8952ba6f8c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.878083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6289c7-2659-464e-bdeb-8952ba6f8c6f admin admin] PATH_INFO: `/v3/projects/4acba97f54454c5fafbbd04d38dbc5e9/users/e812a3299d954eb1b3a060df26bcf803/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.884822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6289c7-2659-464e-bdeb-8952ba6f8c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4acba97f54454c5fafbbd04d38dbc5e9, user_id=e812a3299d954eb1b3a060df26bcf803, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:31.888558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6289c7-2659-464e-bdeb-8952ba6f8c6f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:31.912865 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1922/3848] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:31 2022] PUT /identity/v3/projects/4acba97f54454c5fafbbd04d38dbc5e9/users/e812a3299d954eb1b3a060df26bcf803/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:31.914176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867806a-02b1-4883-9794-aff002e052bc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:31.914684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867806a-02b1-4883-9794-aff002e052bc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:31.914832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867806a-02b1-4883-9794-aff002e052bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:31.963654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9867806a-02b1-4883-9794-aff002e052bc None None] MFA Rules not processed for user `e812a3299d954eb1b3a060df26bcf803`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:31.965356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9867806a-02b1-4883-9794-aff002e052bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:32.101677 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1927/3849] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:32.112007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:32.112464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:32.112840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:32.127060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867806a-02b1-4883-9794-aff002e052bc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:32.129711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9867806a-02b1-4883-9794-aff002e052bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:32.136884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:32.149033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:32.151685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:32.192512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:32.213630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867806a-02b1-4883-9794-aff002e052bc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:32.221100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9499131-2d0f-4b2a-be7b-99a636258c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:32.221881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9499131-2d0f-4b2a-be7b-99a636258c55 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:32.222221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9499131-2d0f-4b2a-be7b-99a636258c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:32.222722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9499131-2d0f-4b2a-be7b-99a636258c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:32.238314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9499131-2d0f-4b2a-be7b-99a636258c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:32.239836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9499131-2d0f-4b2a-be7b-99a636258c55 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:32.252478 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1923/3850] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:01:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:32.272622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:32.274293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:32.279792 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1928/3851] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:32.381878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53249e-8947-4f67-9a2a-6ee2aab632a1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:32.393291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:32.393978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:32.394320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:32.394601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:32.405846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:32.409187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:32.434205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1924/3852] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:35.308495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9325d-c8bb-4cca-ac4b-a03a63f47a96 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:35.309199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9325d-c8bb-4cca-ac4b-a03a63f47a96 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:35.309199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9325d-c8bb-4cca-ac4b-a03a63f47a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:35.336016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-59f9325d-c8bb-4cca-ac4b-a03a63f47a96 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:35.338521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59f9325d-c8bb-4cca-ac4b-a03a63f47a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:35.367414 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1929/3853] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:35.368764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:35.403871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f542d3-a151-4ba5-8e7c-e1f266c785ff service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:35.411938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:35.413449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:35.414591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:35.415311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:35.440522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:35.445762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:35.462035 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1925/3854] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:35.879695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502bf93-9be2-4e40-80d8-e58421688b17 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:35.879984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502bf93-9be2-4e40-80d8-e58421688b17 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:35.880265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502bf93-9be2-4e40-80d8-e58421688b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:35.901709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4502bf93-9be2-4e40-80d8-e58421688b17 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:35.903187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4502bf93-9be2-4e40-80d8-e58421688b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:35.911929 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1930/3855] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:35.925550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:35.936714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c3e39-4fd7-4181-9095-f21e9e605445 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:35.945094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60426697-0402-408a-9c59-db9bd0d5da28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:35.945844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60426697-0402-408a-9c59-db9bd0d5da28 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:35.946204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60426697-0402-408a-9c59-db9bd0d5da28 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:35.946555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60426697-0402-408a-9c59-db9bd0d5da28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:35.959490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60426697-0402-408a-9c59-db9bd0d5da28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:35.961371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60426697-0402-408a-9c59-db9bd0d5da28 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:35.986646 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1926/3856] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:36.396424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502bf93-9be2-4e40-80d8-e58421688b17 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:36.480396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502bf93-9be2-4e40-80d8-e58421688b17 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:36.486695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:36.487436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:36.487800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:36.488152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] PATH_INFO: `/v3/users/9bccaed010664129961e499b613a416d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:36.489553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bccaed010664129961e499b613a416d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:36.491631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:36.540233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75c32-3c0f-4e17-b4bc-7b84b07415a4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:36.540702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75c32-3c0f-4e17-b4bc-7b84b07415a4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:36.540822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75c32-3c0f-4e17-b4bc-7b84b07415a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:36.556336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bccaed010664129961e499b613a416d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:36.561313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-97e75c32-3c0f-4e17-b4bc-7b84b07415a4 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:36.563505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-97e75c32-3c0f-4e17-b4bc-7b84b07415a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:36.570340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bccaed010664129961e499b613a416d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:36.571833 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1927/3857] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:01:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:36.582954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bccaed010664129961e499b613a416d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:36.585566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e75c32-3c0f-4e17-b4bc-7b84b07415a4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:36.591576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bccaed010664129961e499b613a416d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:36.592267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bbb75f4b-1ab8-429e-aa60-600d5a1fa7b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bccaed010664129961e499b613a416d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:36.599726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e75c32-3c0f-4e17-b4bc-7b84b07415a4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:36.600320 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1931/3858] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:36 2022] DELETE /identity/v3/users/9bccaed010664129961e499b613a416d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:36.604726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:36.605205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:36.605679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:36.627875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:36.629796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:36.637182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:36.672175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:36.672939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:36.673220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:36.673493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:36.675343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:36.700184 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1932/3859] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:36.710518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:36.712138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:36.721736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:36.762382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:36.764385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:36.776123 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1928/3860] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:01:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:01:36.779517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cd36d-ffc2-4fe4-863b-256cdaf6ebda None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:36.783375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f54e8a-2488-4f26-89f4-f6587ff418f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:36.783994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f54e8a-2488-4f26-89f4-f6587ff418f5 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:36.784275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f54e8a-2488-4f26-89f4-f6587ff418f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:36.784546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f54e8a-2488-4f26-89f4-f6587ff418f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:36.795845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f54e8a-2488-4f26-89f4-f6587ff418f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:36.799143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f54e8a-2488-4f26-89f4-f6587ff418f5 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:36.811260 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1933/3861] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:36.912172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:36.912172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:36.929804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9425d-e721-42aa-8523-d7632e257a4a service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:36.935397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:36.936380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:36.936915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:36.937390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] PATH_INFO: `/v3/users/81ee4fa83ef649d0bdc273243cd717c4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:36.939164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ee4fa83ef649d0bdc273243cd717c4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:36.942121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:37.053291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ee4fa83ef649d0bdc273243cd717c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.061490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ee4fa83ef649d0bdc273243cd717c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.062386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ee4fa83ef649d0bdc273243cd717c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.073536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ee4fa83ef649d0bdc273243cd717c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.074405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ee4fa83ef649d0bdc273243cd717c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.102943 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1929/3862] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:36 2022] DELETE /identity/v3/users/81ee4fa83ef649d0bdc273243cd717c4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:37.107852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:37.108148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:37.108425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:37.135709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:37.137908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:37.266963 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1934/3863] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:37.284775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:37.286732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:37.385015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924ad7b-c94b-4844-a734-928984cdf057 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:37.389438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:37.390209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:37.390646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:37.391024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:37.404189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:37.408475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:37.421174 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1930/3864] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:37.444615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:37.446972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:37.459765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494178a-e6e3-4b32-92af-b9e0a2717745 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:37.466193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:37.466919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:37.467246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:37.483202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] PATH_INFO: `/v3/projects/e196f8b976a64c70b73768215f312943` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:37.485892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e196f8b976a64c70b73768215f312943)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:37.488250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:37.563392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e196f8b976a64c70b73768215f312943', 'request_id': 'req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.564932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:37.571302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e196f8b976a64c70b73768215f312943', 'request_id': 'req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.579376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e196f8b976a64c70b73768215f312943', 'request_id': 'req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.597693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] The token cache is being invalidate because project e196f8b976a64c70b73768215f312943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:37.598385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.599886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.602061 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1935/3865] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:37 2022] DELETE /identity/v3/projects/e196f8b976a64c70b73768215f312943 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:37.605693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:37.616722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d23041-e832-4049-9cd4-267f775846a0 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:37.667368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:37.683558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:37.684203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:37.684735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:37.685023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] PATH_INFO: `/v3/users/e812a3299d954eb1b3a060df26bcf803` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:37.686208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e812a3299d954eb1b3a060df26bcf803)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:37.688973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:37.777795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e812a3299d954eb1b3a060df26bcf803'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.787522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e812a3299d954eb1b3a060df26bcf803'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.788592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e812a3299d954eb1b3a060df26bcf803'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.804556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e812a3299d954eb1b3a060df26bcf803'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.806959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-349ca08b-40bd-4f19-adc2-a2e624cf0da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e812a3299d954eb1b3a060df26bcf803'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:37.832307 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1931/3866] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:37 2022] DELETE /identity/v3/users/e812a3299d954eb1b3a060df26bcf803 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:38.008534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.012163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:38.028510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ff4ee-fd38-436d-ad0e-64a7c708c4ee admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.118383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.119322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.119680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.120105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] PATH_INFO: `/v3/projects/9c091a483f8b466f89ea487d13fe422b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.121516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c091a483f8b466f89ea487d13fe422b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:38.123499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:38.182571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c091a483f8b466f89ea487d13fe422b', 'request_id': 'req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.194629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c091a483f8b466f89ea487d13fe422b', 'request_id': 'req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.207032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c091a483f8b466f89ea487d13fe422b', 'request_id': 'req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.229377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] The token cache is being invalidate because project 9c091a483f8b466f89ea487d13fe422b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:38.229730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.230430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b72b8c5-5561-4bd7-bb86-3c9a3a5ff2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.232167 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1936/3867] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:38 2022] DELETE /identity/v3/projects/9c091a483f8b466f89ea487d13fe422b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:38.238928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52ec4b-956b-484b-872b-019bcb6765f4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.239217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52ec4b-956b-484b-872b-019bcb6765f4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.239684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52ec4b-956b-484b-872b-019bcb6765f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.265346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-af52ec4b-956b-484b-872b-019bcb6765f4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:38.267048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af52ec4b-956b-484b-872b-019bcb6765f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:38.346878 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1932/3868] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:38.355253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac1e3-38bd-4652-a72c-21d048b0284a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.355537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac1e3-38bd-4652-a72c-21d048b0284a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.355825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac1e3-38bd-4652-a72c-21d048b0284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.391121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-05fac1e3-38bd-4652-a72c-21d048b0284a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:38.391874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-05fac1e3-38bd-4652-a72c-21d048b0284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:38.403686 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1937/3869] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:38.405560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52ec4b-956b-484b-872b-019bcb6765f4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.417721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52ec4b-956b-484b-872b-019bcb6765f4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.428029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.428978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.430127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.430587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.431815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:38.432285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:38.455126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:38.470424 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1933/3870] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:38.475134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac1e3-38bd-4652-a72c-21d048b0284a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.486635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac1e3-38bd-4652-a72c-21d048b0284a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.493666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.495111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.495391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.495667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.496302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:38.498640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:38.553233 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1938/3871] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:38 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:38.554588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.574404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d698f-b605-48eb-a94b-7d00c792905c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.646679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:38.656163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.657003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.657667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.658550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.662242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:38.664220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.665579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:38.667055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:38.718904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc7f9-fc64-42da-abaf-cdb06983133d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.730368 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1934/3872] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:38 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:38.734622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.738477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.753307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.759472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.760377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] PATH_INFO: `/v3/projects/4acba97f54454c5fafbbd04d38dbc5e9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.762641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4acba97f54454c5fafbbd04d38dbc5e9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:38.767896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:38.779572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6f645-d537-4210-a8e3-179a00829476 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.785191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.786398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.787035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.787499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.788329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:38.791801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:38.807896 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1935/3873] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:38.815451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.839203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ed2bb-915b-49af-85ce-cac68140132f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.850711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.851730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.852110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.852703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.853413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:38.856613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:38.868689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:38.873405 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1936/3874] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:38.878882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:38.891140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4acba97f54454c5fafbbd04d38dbc5e9', 'request_id': 'req-f50a940f-630b-4335-8e23-a485f76ea326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.895405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c2771-9698-41f1-aad1-4fa90385e559 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:38.904289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4acba97f54454c5fafbbd04d38dbc5e9', 'request_id': 'req-f50a940f-630b-4335-8e23-a485f76ea326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.916423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4acba97f54454c5fafbbd04d38dbc5e9', 'request_id': 'req-f50a940f-630b-4335-8e23-a485f76ea326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.934366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] The token cache is being invalidate because project 4acba97f54454c5fafbbd04d38dbc5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:38.934815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.935688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f50a940f-630b-4335-8e23-a485f76ea326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:38.941276 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1939/3875] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:38 2022] DELETE /identity/v3/projects/4acba97f54454c5fafbbd04d38dbc5e9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:38.944192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d190a-7b8e-4864-ac32-624fae3be71f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.944476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d190a-7b8e-4864-ac32-624fae3be71f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.944753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d190a-7b8e-4864-ac32-624fae3be71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:38.976404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ee9d190a-7b8e-4864-ac32-624fae3be71f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:38.978702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ee9d190a-7b8e-4864-ac32-624fae3be71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:38.987921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58b3d2-2702-41af-ba65-8b9aba090f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:38.988667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b3d2-2702-41af-ba65-8b9aba090f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:38.988866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b3d2-2702-41af-ba65-8b9aba090f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:38.989145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b3d2-2702-41af-ba65-8b9aba090f68 admin admin] PATH_INFO: `/v3/projects/75ff64ac8631450bb5e2d269118982a8/users/b638a9e7f0934e47a3883be65e42d558/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.018542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58b3d2-2702-41af-ba65-8b9aba090f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ff64ac8631450bb5e2d269118982a8, user_id=b638a9e7f0934e47a3883be65e42d558, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.021593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58b3d2-2702-41af-ba65-8b9aba090f68 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.045814 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1937/3876] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:38 2022] PUT /identity/v3/projects/75ff64ac8631450bb5e2d269118982a8/users/b638a9e7f0934e47a3883be65e42d558/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:39.049317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e309a-54f4-4edf-a85f-5e7cf1e43dcf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.049609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e309a-54f4-4edf-a85f-5e7cf1e43dcf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.049978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e309a-54f4-4edf-a85f-5e7cf1e43dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.073409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1940/3877] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:39.073817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5b1e309a-54f4-4edf-a85f-5e7cf1e43dcf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:39.075723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b1e309a-54f4-4edf-a85f-5e7cf1e43dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:39.080882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cb142-fd94-4e1b-8dc6-ab479c87ac3e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.081502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cb142-fd94-4e1b-8dc6-ab479c87ac3e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.081978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cb142-fd94-4e1b-8dc6-ab479c87ac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.108701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-694cb142-fd94-4e1b-8dc6-ab479c87ac3e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:39.110928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-694cb142-fd94-4e1b-8dc6-ab479c87ac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:39.164423 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1938/3878] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:39.198153 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1941/3879] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:39.203811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e309a-54f4-4edf-a85f-5e7cf1e43dcf None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.219740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e309a-54f4-4edf-a85f-5e7cf1e43dcf None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.225375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.226167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.226549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.226952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.228124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:39.228529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.230778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.251581 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1939/3880] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:39.255909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cb142-fd94-4e1b-8dc6-ab479c87ac3e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.266279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.269603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:39.270199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cb142-fd94-4e1b-8dc6-ab479c87ac3e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.277740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.278344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.279852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.280244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.282442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.284570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.320650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:39.329222 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1942/3881] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:39 2022] POST /identity/v3/projects => generated 405 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:39.334561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.362864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd5b98-2e35-48c5-b13a-b8bf85deb481 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.408137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:39.440327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8ebd9-bc36-430b-8dec-f4ebf272adcb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.447009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.447762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.448106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.448387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.449183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.451191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.461872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.463355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.465136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.465507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.483705 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1943/3882] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:39 2022] POST /identity/v3/users => generated 303 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:01:39.485640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.497402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f6289-bf56-4bc9-98f6-209c1746a1d2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.504309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.505369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.505560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.506090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.506595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.507694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.507813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.509772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.526538 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1944/3883] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:39.527824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.541855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f122a-c1d7-48fe-9db9-2b9b7b5797e1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.544396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1940/3884] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:39.550073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25976eff-29aa-48f1-b38b-86e232c2ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.550972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25976eff-29aa-48f1-b38b-86e232c2ee19 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.551248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25976eff-29aa-48f1-b38b-86e232c2ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.551573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25976eff-29aa-48f1-b38b-86e232c2ee19 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.552319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25976eff-29aa-48f1-b38b-86e232c2ee19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.554070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25976eff-29aa-48f1-b38b-86e232c2ee19 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.570819 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1945/3885] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:39.582846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.594429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccf06d-5e28-4ddb-b696-62391c4dedc7 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.600726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.601650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.602126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.602592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] PATH_INFO: `/v3/projects/cd988a3e226749e09043d36d3d610d58/users/2eb69f3b88f840c09d4bb2bf69217f96/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.630268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd988a3e226749e09043d36d3d610d58, user_id=2eb69f3b88f840c09d4bb2bf69217f96, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.633140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.649948 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1941/3886] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:39 2022] PUT /identity/v3/projects/cd988a3e226749e09043d36d3d610d58/users/2eb69f3b88f840c09d4bb2bf69217f96/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:39.659987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9511a59-f900-4e1a-9679-01b7081918a0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.660274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9511a59-f900-4e1a-9679-01b7081918a0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.661056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9511a59-f900-4e1a-9679-01b7081918a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.704288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e9511a59-f900-4e1a-9679-01b7081918a0 None None] MFA Rules not processed for user `2eb69f3b88f840c09d4bb2bf69217f96`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:39.705526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e9511a59-f900-4e1a-9679-01b7081918a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:39.788189 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1946/3887] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:39 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:39.792618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.895509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb66c91-588f-4927-9934-7a40e753c5a4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:39.900616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:39.901258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:39.902026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:39.902026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:39.902440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:39.904713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:39.940330 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1942/3888] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:39 2022] POST /identity/v3/projects => generated 405 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:39.941587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9511a59-f900-4e1a-9679-01b7081918a0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:39.965947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9511a59-f900-4e1a-9679-01b7081918a0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:40.064217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:40.065127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:40.065876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:40.065876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:40.066662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:40.068821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:40.099398 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1947/3889] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:39 2022] POST /identity/v3/users => generated 309 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:40.105435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:40.117464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3e41c-1287-4ecf-b798-e2e05d5a82ce admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:40.126703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:40.127487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:40.127798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:40.128080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:40.128739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:40.130623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:40.148168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:40.149363 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1943/3890] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:40.160228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3d8a-6c83-4902-b1d6-a7f793b0a18c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:40.166010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:40.167030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:40.167705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:40.168131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] PATH_INFO: `/v3/projects/10d84a67488044eaaee66bd000de05de/users/4b3431297f8f4c2cbe65d8c6372125aa/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:40.187460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d84a67488044eaaee66bd000de05de, user_id=4b3431297f8f4c2cbe65d8c6372125aa, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:40.193370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:40.216083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:40.217746 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1948/3891] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:40 2022] PUT /identity/v3/projects/10d84a67488044eaaee66bd000de05de/users/4b3431297f8f4c2cbe65d8c6372125aa/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:40.232568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffa57b-3472-4e99-a6ec-a9673c1f0000 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:40.316554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:40.338293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f037e-5c1c-4e4c-808c-5093d717d738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:40.339184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f037e-5c1c-4e4c-808c-5093d717d738 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:40.339618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f037e-5c1c-4e4c-808c-5093d717d738 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:40.340663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f037e-5c1c-4e4c-808c-5093d717d738 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:40.342433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f037e-5c1c-4e4c-808c-5093d717d738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:40.344379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f037e-5c1c-4e4c-808c-5093d717d738 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:40.365908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:40.367980 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1944/3892] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:40.377889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:40.383681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a8ac1-df3e-4ef6-80a8-262962e0e114 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:40.397931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:40.400326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:40.401388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:40.402235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] PATH_INFO: `/v3/projects/10d84a67488044eaaee66bd000de05de/users/4b3431297f8f4c2cbe65d8c6372125aa/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:40.405509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d84a67488044eaaee66bd000de05de, user_id=4b3431297f8f4c2cbe65d8c6372125aa, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:40.409607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:40.435437 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1949/3893] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:40 2022] PUT /identity/v3/projects/10d84a67488044eaaee66bd000de05de/users/4b3431297f8f4c2cbe65d8c6372125aa/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:40.437407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:40.437407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:40.437774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:40.482863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] MFA Rules not processed for user `4b3431297f8f4c2cbe65d8c6372125aa`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:40.486660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:40.637369 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1945/3894] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:40 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:40.709673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:40.865691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2e54e-bb98-4415-94dd-f7db7a901a0c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:40.876443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:40.877417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:40.878083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:40.878552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:41.050809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:41.052416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:41.065457 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1950/3895] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:40 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:42.445686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:42.447318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:42.646145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb93d1-ff85-4bd3-aed9-c4e7d01ad781 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:42.654992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:42.656339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:42.657588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:42.658029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:42.669571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:42.672913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:42.689884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1946/3896] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:42 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:47.464704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:47.466854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:47.576925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080609a0-e55b-4ea8-9168-e920b0361680 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:47.584530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:47.585280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:47.585656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:47.586055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:47.588110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:47.604976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:47.606885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:47.647790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1951/3897] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:47.725629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:47.727346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:47.797133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:47.822990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7608df0-1b73-45ff-b304-89e2761d21ac service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:47.830124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:47.831536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:47.831803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:47.832306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] PATH_INFO: `/v3/users/ae0c7612d933424597e1f176be189a0b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:47.833641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0c7612d933424597e1f176be189a0b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:47.835760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:47.934174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0c7612d933424597e1f176be189a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:47.943413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0c7612d933424597e1f176be189a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:47.944226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0c7612d933424597e1f176be189a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:47.956114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0c7612d933424597e1f176be189a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:47.956750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-817a7bd7-6341-4dab-b5dd-48ce95e350e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0c7612d933424597e1f176be189a0b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:47.994896 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1947/3898] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:47 2022] DELETE /identity/v3/users/ae0c7612d933424597e1f176be189a0b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:48.479378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:48.499098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a36d5b-b3cb-43b0-8f04-1c9d3375196a service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:48.576953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:48.633459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:48.637068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:48.637068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:48.637443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] PATH_INFO: `/v3/projects/cebec33205424ecf92b54428bdf5eb61` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:48.640883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebec33205424ecf92b54428bdf5eb61)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:48.644498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:48.751855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebec33205424ecf92b54428bdf5eb61', 'request_id': 'req-973cb964-bb3c-4481-badc-49f2abd0c42d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:48.770440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebec33205424ecf92b54428bdf5eb61', 'request_id': 'req-973cb964-bb3c-4481-badc-49f2abd0c42d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:48.791104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebec33205424ecf92b54428bdf5eb61', 'request_id': 'req-973cb964-bb3c-4481-badc-49f2abd0c42d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:48.814218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] The token cache is being invalidate because project cebec33205424ecf92b54428bdf5eb61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:48.814823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:48.815788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cb964-bb3c-4481-badc-49f2abd0c42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:48.818980 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1952/3899] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:48 2022] DELETE /identity/v3/projects/cebec33205424ecf92b54428bdf5eb61 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:48.823525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8419c-ebf7-4951-80c0-920a8cb8dd09 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:48.824055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8419c-ebf7-4951-80c0-920a8cb8dd09 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:48.824055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8419c-ebf7-4951-80c0-920a8cb8dd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:48.872916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3bf8419c-ebf7-4951-80c0-920a8cb8dd09 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:48.874182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3bf8419c-ebf7-4951-80c0-920a8cb8dd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:49.063374 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1948/3900] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:49.074465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd0f3-f548-4252-88bb-df5b63b4ecf1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.075025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd0f3-f548-4252-88bb-df5b63b4ecf1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.075406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dd0f3-f548-4252-88bb-df5b63b4ecf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.104193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dd5dd0f3-f548-4252-88bb-df5b63b4ecf1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:49.105849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dd5dd0f3-f548-4252-88bb-df5b63b4ecf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:49.141663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8419c-ebf7-4951-80c0-920a8cb8dd09 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.151130 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1953/3901] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:49.158569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8419c-ebf7-4951-80c0-920a8cb8dd09 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.169412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.170301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.170851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.172467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.174173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:49.178578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.181651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.209607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd0f3-f548-4252-88bb-df5b63b4ecf1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.215742 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1949/3902] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:01:49.221006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dd0f3-f548-4252-88bb-df5b63b4ecf1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.227923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.229236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.229624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.230057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.230800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.241657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.285483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.295092 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1954/3903] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:49 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:49.308019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1768f-3ad1-4817-bcb2-e674bcf2b01b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.375497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:49.385564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.386889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.389106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.389106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.390231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.393018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.427458 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1950/3904] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:49 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:49.431018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.442795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb362ed0-64a6-45e5-bedc-e325b5397029 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.447899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.448528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.448808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.449081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.449726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.451385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.466273 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1955/3905] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:49.467188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.480736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d7afc-1a4c-4bca-8179-9bb354b0d5f1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.486564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95280fc3-3686-4f72-86a5-efed199dcf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.487183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95280fc3-3686-4f72-86a5-efed199dcf86 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.487466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95280fc3-3686-4f72-86a5-efed199dcf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.487741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95280fc3-3686-4f72-86a5-efed199dcf86 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.488387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95280fc3-3686-4f72-86a5-efed199dcf86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.490033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95280fc3-3686-4f72-86a5-efed199dcf86 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.506805 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1951/3906] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:49.508196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.519690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b549fcf-fcf4-4672-b136-a00f86768ab8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:49.525085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:49.526077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.526533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.527675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] PATH_INFO: `/v3/projects/dd2cd4bb350d452eb8c5e2533642c0f0/users/6f50ca756d594703846f1dade2d38eac/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.546997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2cd4bb350d452eb8c5e2533642c0f0, user_id=6f50ca756d594703846f1dade2d38eac, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:49.551113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:49.576136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:49.576699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:49.577211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:49.585921 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1956/3907] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:49 2022] PUT /identity/v3/projects/dd2cd4bb350d452eb8c5e2533642c0f0/users/6f50ca756d594703846f1dade2d38eac/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:49.614664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:49.616478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:49.731208 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1952/3908] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:49.763706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:49.767065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:49.966746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298d6b3-b932-41fb-973f-7d88f5d169ac admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.006362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d34149-659d-43ac-a514-178057971528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.008797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34149-659d-43ac-a514-178057971528 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.010577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34149-659d-43ac-a514-178057971528 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.011678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d34149-659d-43ac-a514-178057971528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.058077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.059840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:50.070023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d34149-659d-43ac-a514-178057971528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.071704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d34149-659d-43ac-a514-178057971528 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.094959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978021ee-8c40-48d6-84ca-de1b247a47cd None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.132148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.132385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.132648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.132929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] PATH_INFO: `/v3/users/f38321284ec449e599d645625ed11139` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.141613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38321284ec449e599d645625ed11139)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.143628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.145569 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1957/3909] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:50.224788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38321284ec449e599d645625ed11139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.235581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38321284ec449e599d645625ed11139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.236004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38321284ec449e599d645625ed11139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.252570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38321284ec449e599d645625ed11139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.253616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38321284ec449e599d645625ed11139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:50.309523 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1953/3910] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:50 2022] DELETE /identity/v3/users/f38321284ec449e599d645625ed11139 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:50.314391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.314699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.315806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.329197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:50.347437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:50.349976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:50.444710 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1958/3911] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:50.474058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:50.474058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:50.623364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:50.630064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19124032-e173-48d5-893f-cf05f28cd938 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:50.634968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:50.635993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:50.636635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:50.637778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:50.651038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:50.653369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:50.669083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1954/3912] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:51.645086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.646996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:51.665010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb012a3-1a9e-4fd9-b6cc-02fd516cc747 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:51.670478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:51.671165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:51.671596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:51.672160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] PATH_INFO: `/v3/projects/d30ef9b7525e4784be26268f46121e62` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:51.673379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30ef9b7525e4784be26268f46121e62)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:51.675505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:51.779046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30ef9b7525e4784be26268f46121e62', 'request_id': 'req-85d133f0-8f8f-4549-be05-11b2ab83b1d3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:51.795145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30ef9b7525e4784be26268f46121e62', 'request_id': 'req-85d133f0-8f8f-4549-be05-11b2ab83b1d3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:51.814295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30ef9b7525e4784be26268f46121e62', 'request_id': 'req-85d133f0-8f8f-4549-be05-11b2ab83b1d3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:51.848443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] The token cache is being invalidate because project d30ef9b7525e4784be26268f46121e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:51.849267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:51.851766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:51.859854 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1959/3913] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:51 2022] DELETE /identity/v3/projects/d30ef9b7525e4784be26268f46121e62 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:51.865123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:51.878827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ef71-1956-4dda-ba11-b25e5ceeadad service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:52.023395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:52.024673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:52.024967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:52.025654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] PATH_INFO: `/v3/users/9d79404e087f4e3e87f52ac0941a33c9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:52.027532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d79404e087f4e3e87f52ac0941a33c9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:52.033087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:52.188882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d79404e087f4e3e87f52ac0941a33c9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:52.204761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d79404e087f4e3e87f52ac0941a33c9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:52.220645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d79404e087f4e3e87f52ac0941a33c9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:52.237440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d79404e087f4e3e87f52ac0941a33c9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:52.238825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b5659a1-69ab-465b-a692-2e10f1756ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d79404e087f4e3e87f52ac0941a33c9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:52.271848 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1955/3914] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:51 2022] DELETE /identity/v3/users/9d79404e087f4e3e87f52ac0941a33c9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:53.298234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:53.299702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:53.324532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d133f0-8f8f-4549-be05-11b2ab83b1d3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:53.472520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:53.473873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:53.474298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:53.474654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] PATH_INFO: `/v3/projects/80bf9de322d94dfcaaf62eb62aeba095` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:53.476127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80bf9de322d94dfcaaf62eb62aeba095)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:53.478524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:53.576481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:53.620818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80bf9de322d94dfcaaf62eb62aeba095', 'request_id': 'req-19878d69-c6ea-4294-96ca-e3dcf6d05656'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:53.638750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80bf9de322d94dfcaaf62eb62aeba095', 'request_id': 'req-19878d69-c6ea-4294-96ca-e3dcf6d05656'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:53.655420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80bf9de322d94dfcaaf62eb62aeba095', 'request_id': 'req-19878d69-c6ea-4294-96ca-e3dcf6d05656'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:53.684987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] The token cache is being invalidate because project 80bf9de322d94dfcaaf62eb62aeba095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:53.685290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:53.687365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19878d69-c6ea-4294-96ca-e3dcf6d05656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:53.690663 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1960/3915] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:53 2022] DELETE /identity/v3/projects/80bf9de322d94dfcaaf62eb62aeba095 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:53.696203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3e8c-a41c-40f9-b57f-b558e620a5cf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:53.696770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3e8c-a41c-40f9-b57f-b558e620a5cf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:53.696894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3e8c-a41c-40f9-b57f-b558e620a5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:53.735944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c21e3e8c-a41c-40f9-b57f-b558e620a5cf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:53.737213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c21e3e8c-a41c-40f9-b57f-b558e620a5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:53.865374 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1956/3916] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:53.866410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d579fdb-80f2-47c2-a083-9083b654aa94 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:53.866907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d579fdb-80f2-47c2-a083-9083b654aa94 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:53.867349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d579fdb-80f2-47c2-a083-9083b654aa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:53.897170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1d579fdb-80f2-47c2-a083-9083b654aa94 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:53.899513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d579fdb-80f2-47c2-a083-9083b654aa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:53.911777 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1961/3917] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:01:53.916114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3e8c-a41c-40f9-b57f-b558e620a5cf None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:53.927671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3e8c-a41c-40f9-b57f-b558e620a5cf None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:53.932619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:53.933521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:53.934038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:53.934528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:53.935387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:53.935913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:53.938604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:53.941615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:53.960386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d579fdb-80f2-47c2-a083-9083b654aa94 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:53.966359 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1957/3918] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:53.990156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d579fdb-80f2-47c2-a083-9083b654aa94 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:53.996213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:53.997085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:53.997522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:53.998644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:53.999563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:54.002074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:54.065177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1962/3919] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:53 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:54.069844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:54.085457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f1cc0-8787-44e2-a994-12eed8052736 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:54.197759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:54.200053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:54.200464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:54.207314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:54.208229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:54.210401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:54.252441 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1958/3920] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:54 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:54.255705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:54.282894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a68ee-66c3-4f46-a3a6-44272c6a6c88 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:54.293226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:54.295438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:54.297313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:54.298013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:54.300580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:54.304216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:54.339610 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1963/3921] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:54.340402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:54.362399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456c7c6-9730-4c87-b12c-2a2defbe5d5d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:54.375308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5295cd-f34a-463c-876d-814d293098da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:54.375940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5295cd-f34a-463c-876d-814d293098da admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:54.376228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5295cd-f34a-463c-876d-814d293098da admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:54.376825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5295cd-f34a-463c-876d-814d293098da admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:54.377422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5295cd-f34a-463c-876d-814d293098da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:54.379159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5295cd-f34a-463c-876d-814d293098da admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:54.398191 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1959/3922] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:01:54.402795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:54.419000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7651a3-e451-4f97-b036-5761f3807fe9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:54.439618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:54.440375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:54.442046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:54.442482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] PATH_INFO: `/v3/projects/d777e13fce174529bea607084fd7ddd6/users/1d619203301549838fddbca84b32e667/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:54.483439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d777e13fce174529bea607084fd7ddd6, user_id=1d619203301549838fddbca84b32e667, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:54.486594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:54.526025 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1964/3923] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:54 2022] PUT /identity/v3/projects/d777e13fce174529bea607084fd7ddd6/users/1d619203301549838fddbca84b32e667/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:54.532088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5936389-7517-480f-8500-5ff3699ef49e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:54.532402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5936389-7517-480f-8500-5ff3699ef49e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:54.532649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5936389-7517-480f-8500-5ff3699ef49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:54.599002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e5936389-7517-480f-8500-5ff3699ef49e None None] MFA Rules not processed for user `1d619203301549838fddbca84b32e667`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:54.600250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e5936389-7517-480f-8500-5ff3699ef49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:54.791357 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1960/3924] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:54.792400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:54.948882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8afde5-f82a-45dd-a6ad-5c05b74c723f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:54.961580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:54.962292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:54.962576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:54.962848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:54.963499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:54.965432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:55.036208 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1965/3925] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:54 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:55.040499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5936389-7517-480f-8500-5ff3699ef49e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:55.058759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5936389-7517-480f-8500-5ff3699ef49e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:55.173089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:55.174085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:55.174584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:55.175042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:55.176543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:55.179536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:55.195339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:55.221861 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1961/3926] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:55 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:55.225416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:55.243019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d78978-9293-46db-97e6-5381b4b0d8e4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:55.251721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:55.252845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:55.253433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:55.254044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:55.255778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:55.257623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:55.311546 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1966/3927] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:55.313108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:55.329076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285c6ce-15f6-4ee5-9f66-15a783b97869 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:55.349021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:55.355280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:55.355577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:55.355877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] PATH_INFO: `/v3/projects/a5e92c5334ae48199c1da0210043847d/users/f311912f0d4c4260bcda6e2c25c8a513/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:55.414383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e92c5334ae48199c1da0210043847d, user_id=f311912f0d4c4260bcda6e2c25c8a513, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:55.418557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:55.459537 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1962/3928] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:55 2022] PUT /identity/v3/projects/a5e92c5334ae48199c1da0210043847d/users/f311912f0d4c4260bcda6e2c25c8a513/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:01:55.464572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:55.484825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edde76-c335-400b-91d5-2c286cf1f73a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:55.578228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:55.652587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488adbdf-0c43-45ce-8b92-48ff92631a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:55.653918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488adbdf-0c43-45ce-8b92-48ff92631a71 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:55.654423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488adbdf-0c43-45ce-8b92-48ff92631a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:55.654912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488adbdf-0c43-45ce-8b92-48ff92631a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:55.657536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488adbdf-0c43-45ce-8b92-48ff92631a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:55.660529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488adbdf-0c43-45ce-8b92-48ff92631a71 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:55.709261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1967/3929] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:55.716689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:55.748001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa32de-09b0-483a-8a14-b83f7bdab746 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:55.757163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046c3cb-5ae4-4b7b-bb86-0e0bd0ebaf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:55.758130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046c3cb-5ae4-4b7b-bb86-0e0bd0ebaf37 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:55.758758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046c3cb-5ae4-4b7b-bb86-0e0bd0ebaf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:55.759194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046c3cb-5ae4-4b7b-bb86-0e0bd0ebaf37 admin admin] PATH_INFO: `/v3/projects/a5e92c5334ae48199c1da0210043847d/users/f311912f0d4c4260bcda6e2c25c8a513/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:55.761361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046c3cb-5ae4-4b7b-bb86-0e0bd0ebaf37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e92c5334ae48199c1da0210043847d, user_id=f311912f0d4c4260bcda6e2c25c8a513, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:55.764335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046c3cb-5ae4-4b7b-bb86-0e0bd0ebaf37 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:55.790357 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1963/3930] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:55 2022] PUT /identity/v3/projects/a5e92c5334ae48199c1da0210043847d/users/f311912f0d4c4260bcda6e2c25c8a513/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:55.803393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:55.804794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:55.805180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:55.842295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] MFA Rules not processed for user `f311912f0d4c4260bcda6e2c25c8a513`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:55.844302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:55.960028 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1968/3931] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:56.304427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:56.304953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:56.305242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:56.348465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] MFA Rules not processed for user `b638a9e7f0934e47a3883be65e42d558`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:56.350183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:56.499227 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1964/3932] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:56.651769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:56.653940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:56.865699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91686e11-d73a-46b3-a3cb-9a9f22dee04b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:56.887254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:56.888061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:56.888345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:56.888694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:56.911826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:56.913452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:56.981102 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1969/3933] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:57.081873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:57.083479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:57.141685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:57.158845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb207cbf-6c5a-44b8-a59f-be2e85d8385a service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:57.184391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:57.186875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:57.187354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:57.187643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:57.211133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:57.212800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:57.242499 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1970/3934] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:01:57.276743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97775cc8-8987-4326-83e3-a6a8c33a0ef8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:57.302201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:57.302893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:57.303337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:57.303461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] PATH_INFO: `/v3/users/1b5bc12930c946ccaffcc524795ad965` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:57.307782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b5bc12930c946ccaffcc524795ad965)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:57.312453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:57.392208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b5bc12930c946ccaffcc524795ad965'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:57.396264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:57.413479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b5bc12930c946ccaffcc524795ad965'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:57.417751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b5bc12930c946ccaffcc524795ad965'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:57.439106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b5bc12930c946ccaffcc524795ad965'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:57.439949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d07a897e-47db-4c6f-8d59-a506fa5fb7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b5bc12930c946ccaffcc524795ad965'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:57.484334 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1965/3935] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:57 2022] DELETE /identity/v3/users/1b5bc12930c946ccaffcc524795ad965 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:57.994285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:58.012268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324dd81-f08d-4e15-9089-3b9163abafa4 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:58.051403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:58.113942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:58.115130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:58.115130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:58.115410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] PATH_INFO: `/v3/projects/7289ce4dcfe9474ea286bcbe7ef179a1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:58.117227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7289ce4dcfe9474ea286bcbe7ef179a1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:58.119683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:58.239736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7289ce4dcfe9474ea286bcbe7ef179a1', 'request_id': 'req-f663df0f-70d9-4d4e-a6de-4e402d26c451'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.256965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7289ce4dcfe9474ea286bcbe7ef179a1', 'request_id': 'req-f663df0f-70d9-4d4e-a6de-4e402d26c451'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.269938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7289ce4dcfe9474ea286bcbe7ef179a1', 'request_id': 'req-f663df0f-70d9-4d4e-a6de-4e402d26c451'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.297689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] The token cache is being invalidate because project 7289ce4dcfe9474ea286bcbe7ef179a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:01:58.297971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.299032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.306424 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1971/3936] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:01:57 2022] DELETE /identity/v3/projects/7289ce4dcfe9474ea286bcbe7ef179a1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:58.307668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3f1c1-6cc3-4cd0-a873-f93fae304780 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:58.307963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3f1c1-6cc3-4cd0-a873-f93fae304780 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:58.308501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3f1c1-6cc3-4cd0-a873-f93fae304780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:58.358775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-dbc3f1c1-6cc3-4cd0-a873-f93fae304780 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:58.362769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dbc3f1c1-6cc3-4cd0-a873-f93fae304780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:58.385948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:58.387651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:58.554204 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1966/3937] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:58.555750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73d74-f7ea-4beb-8f0e-04db8762d003 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:58.559307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73d74-f7ea-4beb-8f0e-04db8762d003 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:58.559667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73d74-f7ea-4beb-8f0e-04db8762d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:58.571658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663df0f-70d9-4d4e-a6de-4e402d26c451 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:58.582614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:58.584698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:58.586304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:58.586667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] PATH_INFO: `/v3/users/1d619203301549838fddbca84b32e667` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:58.588830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d619203301549838fddbca84b32e667)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:58.591237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:58.605266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b9f73d74-f7ea-4beb-8f0e-04db8762d003 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:58.606638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b9f73d74-f7ea-4beb-8f0e-04db8762d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:58.628044 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1967/3938] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:58.633735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73d74-f7ea-4beb-8f0e-04db8762d003 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:58.649964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73d74-f7ea-4beb-8f0e-04db8762d003 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:58.661623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:58.662553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:58.663332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:58.663332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:58.664291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:01:58.665254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:58.672563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:58.701789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d619203301549838fddbca84b32e667'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.713795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1968/3939] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:01:58.714850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:58.720385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:58.740843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d619203301549838fddbca84b32e667'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.744836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df625c-77a2-4d77-9022-b48b8ee63ac2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:58.756509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:58.757869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:58.758297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d619203301549838fddbca84b32e667'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.759352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:58.761070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:58.762009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:58.768191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:58.774890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d619203301549838fddbca84b32e667'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.775686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b5a2cee4-bf95-44a3-8c20-3cc35871d843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d619203301549838fddbca84b32e667'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:01:58.793522 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1972/3940] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:01:58 2022] DELETE /identity/v3/users/1d619203301549838fddbca84b32e667 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:58.797096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:58.798236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:58.801084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:58.837623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:58.840986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1969/3941] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:01:58 2022] POST /identity/v3/projects => generated 361 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:58.843788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:58.845229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:58.846928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:59.095568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e97b-33e1-4654-8cfe-2df0c14af38a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.104482 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1973/3942] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:01:59.118552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.120445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:59.122694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.122881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:59.124747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.125499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.127553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.130386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.170083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1970/3943] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:01:58 2022] POST /identity/v3/users => generated 281 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:01:59.180296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:59.227170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f95c-f7a0-4fe3-af11-65be2017d9db None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.235724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.236477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.236858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.238070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.238854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755e339-9d5c-4f2b-a83c-d693fe3f0c82 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.252340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.254633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.254955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.255269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.255553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.255826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.256471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.264287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.269314 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1974/3944] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:01:59.302022 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1971/3945] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:59.306935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:59.322867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083cf51-cb4d-4f84-8233-aae3b1ea3e03 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.336360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adee0d-7f86-4a16-8eb1-21694ed7fb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.337334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adee0d-7f86-4a16-8eb1-21694ed7fb1f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.337334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adee0d-7f86-4a16-8eb1-21694ed7fb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.337563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adee0d-7f86-4a16-8eb1-21694ed7fb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.338449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adee0d-7f86-4a16-8eb1-21694ed7fb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.342413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adee0d-7f86-4a16-8eb1-21694ed7fb1f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.381868 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1975/3946] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:01:59.388741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:01:59.399718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:59.410504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d897b68-f129-468c-b3a9-227d173929b1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:01:59.424647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:01:59.425333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.425619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.426085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] PATH_INFO: `/v3/projects/7fe16a7c4f6e406197baeefbcb6fcb20/users/74d6ad804429434ca952c933970a40c4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.466079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fe16a7c4f6e406197baeefbcb6fcb20, user_id=74d6ad804429434ca952c933970a40c4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:01:59.470330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:01:59.510123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:01:59.510475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:01:59.510676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:01:59.514237 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1972/3947] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:01:59 2022] PUT /identity/v3/projects/7fe16a7c4f6e406197baeefbcb6fcb20/users/74d6ad804429434ca952c933970a40c4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:01:59.578673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] MFA Rules not processed for user `74d6ad804429434ca952c933970a40c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:01:59.586245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:01:59.737117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:01:59.774208 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1976/3948] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:01:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:00.251657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.253728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.569853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c4b8-d600-4177-8b93-c4feecab7439 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.597613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.599460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.603216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.604308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.643117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.644797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.681884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1973/3949] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:00.686377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.688829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:00.805630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:00.866992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08ea9-cf16-44d4-8291-a3835277ac74 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:00.881697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:00.882388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:00.882671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:00.882959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] PATH_INFO: `/v3/projects/d777e13fce174529bea607084fd7ddd6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:00.884788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d777e13fce174529bea607084fd7ddd6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:00.891204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:00.994734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd777e13fce174529bea607084fd7ddd6', 'request_id': 'req-76b326d9-24b3-47e8-8d4c-2a23031bc429'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.006873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd777e13fce174529bea607084fd7ddd6', 'request_id': 'req-76b326d9-24b3-47e8-8d4c-2a23031bc429'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.014006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08d313-5fec-49c8-b6fe-7314ade2817b service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.025752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd777e13fce174529bea607084fd7ddd6', 'request_id': 'req-76b326d9-24b3-47e8-8d4c-2a23031bc429'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.052881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] The token cache is being invalidate because project d777e13fce174529bea607084fd7ddd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:01.054295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.055425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.059976 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1977/3950] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:00 2022] DELETE /identity/v3/projects/d777e13fce174529bea607084fd7ddd6 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:01.065932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.067875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.166534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.168437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.168783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.169068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] PATH_INFO: `/v3/users/74d6ad804429434ca952c933970a40c4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.171435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d6ad804429434ca952c933970a40c4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.176873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.205732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:01.211181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b326d9-24b3-47e8-8d4c-2a23031bc429 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.224006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.224902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.225278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.225682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] PATH_INFO: `/v3/users/f311912f0d4c4260bcda6e2c25c8a513` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.228748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f311912f0d4c4260bcda6e2c25c8a513)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.232155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.277496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:01.301682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d6ad804429434ca952c933970a40c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.318471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d6ad804429434ca952c933970a40c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.320827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d6ad804429434ca952c933970a40c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.340313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d6ad804429434ca952c933970a40c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.341144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-85aa21f8-7a29-4909-96fd-8c1bc69d3633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d6ad804429434ca952c933970a40c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.341268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f311912f0d4c4260bcda6e2c25c8a513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.370498 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1974/3951] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:00 2022] DELETE /identity/v3/users/74d6ad804429434ca952c933970a40c4 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:01.375789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.376350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.376803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.382272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f311912f0d4c4260bcda6e2c25c8a513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.406273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:01.406815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f311912f0d4c4260bcda6e2c25c8a513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.409051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.428707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f311912f0d4c4260bcda6e2c25c8a513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.429480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f311912f0d4c4260bcda6e2c25c8a513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:01.447613 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1978/3952] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:01 2022] DELETE /identity/v3/users/f311912f0d4c4260bcda6e2c25c8a513 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:01.531113 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1975/3953] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:01.556711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:01.558920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:01.684954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0adb5-d117-427a-ad79-da97385413f2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:01.691154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bf0f3-63f0-44c4-814a-bc7b6b16eb41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:01.692926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bf0f3-63f0-44c4-814a-bc7b6b16eb41 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:01.693655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bf0f3-63f0-44c4-814a-bc7b6b16eb41 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:01.694434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bf0f3-63f0-44c4-814a-bc7b6b16eb41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:01.902138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bf0f3-63f0-44c4-814a-bc7b6b16eb41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:01.904737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bf0f3-63f0-44c4-814a-bc7b6b16eb41 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:01.928766 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1979/3954] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:02.634629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:02.636538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.657472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407062-26d2-4d0f-8738-05b3a256ab50 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:02.678529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:02.685137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:02.686128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:02.690191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] PATH_INFO: `/v3/projects/7fe16a7c4f6e406197baeefbcb6fcb20` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:02.699536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fe16a7c4f6e406197baeefbcb6fcb20)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:02.711645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:02.854063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fe16a7c4f6e406197baeefbcb6fcb20', 'request_id': 'req-b288117e-f1ad-43c0-9743-925821a7e445'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.868075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fe16a7c4f6e406197baeefbcb6fcb20', 'request_id': 'req-b288117e-f1ad-43c0-9743-925821a7e445'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.880398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fe16a7c4f6e406197baeefbcb6fcb20', 'request_id': 'req-b288117e-f1ad-43c0-9743-925821a7e445'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.911889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] The token cache is being invalidate because project 7fe16a7c4f6e406197baeefbcb6fcb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:02.912938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.913650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:02.916124 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1976/3955] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:02 2022] DELETE /identity/v3/projects/7fe16a7c4f6e406197baeefbcb6fcb20 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:02.924265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcce48a-7a74-45fc-8826-b04e02749e29 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:02.924265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcce48a-7a74-45fc-8826-b04e02749e29 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:02.924265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcce48a-7a74-45fc-8826-b04e02749e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:02.941478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:02.950970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.952463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8fcce48a-7a74-45fc-8826-b04e02749e29 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:02.955517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8fcce48a-7a74-45fc-8826-b04e02749e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:02.985764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:03.083759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288117e-f1ad-43c0-9743-925821a7e445 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.102816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.103641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.104033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.104414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] PATH_INFO: `/v3/projects/a5e92c5334ae48199c1da0210043847d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.106032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e92c5334ae48199c1da0210043847d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.118164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.119513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:03.147551 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1980/3956] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:03.234816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e92c5334ae48199c1da0210043847d', 'request_id': 'req-2d2a5517-6475-46f5-baaf-15c071af0044'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:03.235958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953b9a-7482-46df-bd35-65e0c864efe0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.236265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953b9a-7482-46df-bd35-65e0c864efe0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.236527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d953b9a-7482-46df-bd35-65e0c864efe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.255103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e92c5334ae48199c1da0210043847d', 'request_id': 'req-2d2a5517-6475-46f5-baaf-15c071af0044'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:03.258535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d953b9a-7482-46df-bd35-65e0c864efe0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:03.259834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d953b9a-7482-46df-bd35-65e0c864efe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:03.282288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e92c5334ae48199c1da0210043847d', 'request_id': 'req-2d2a5517-6475-46f5-baaf-15c071af0044'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:03.303656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] The token cache is being invalidate because project a5e92c5334ae48199c1da0210043847d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:03.304034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:03.307016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2d2a5517-6475-46f5-baaf-15c071af0044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:03.309328 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1977/3957] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:02 2022] DELETE /identity/v3/projects/a5e92c5334ae48199c1da0210043847d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:03.314656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27455c-4b88-48b4-b7e7-7e5329e4ec95 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.314947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27455c-4b88-48b4-b7e7-7e5329e4ec95 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.315231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27455c-4b88-48b4-b7e7-7e5329e4ec95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.363498 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1981/3958] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:03.364945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d953b9a-7482-46df-bd35-65e0c864efe0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.372903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2e27455c-4b88-48b4-b7e7-7e5329e4ec95 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:03.374567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2e27455c-4b88-48b4-b7e7-7e5329e4ec95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:03.378652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d953b9a-7482-46df-bd35-65e0c864efe0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.386712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.387916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.388419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.388903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.391261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:03.391879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.393266 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1978/3959] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:03.394603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.401392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faab5ea-9302-452a-ba08-91735d5ea643 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.401685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faab5ea-9302-452a-ba08-91735d5ea643 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.402289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faab5ea-9302-452a-ba08-91735d5ea643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.414640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1982/3960] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:03.415875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.429563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662ff28-9855-475b-b68a-118fa89b890b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.435833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.437266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.437731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.438107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.438981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.441139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.444860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3faab5ea-9302-452a-ba08-91735d5ea643 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:03.446116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3faab5ea-9302-452a-ba08-91735d5ea643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:03.467400 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1979/3961] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:03.472755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faab5ea-9302-452a-ba08-91735d5ea643 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.494166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faab5ea-9302-452a-ba08-91735d5ea643 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.500871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.501582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.501892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.502268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.502937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:03.503873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.505558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.537881 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1980/3962] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:02:03.542392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.548336 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1983/3963] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:03.556840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.561158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3246e-64d4-4423-8a41-502277649a55 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.691863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.692793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.693262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.693684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.694955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.697667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.701854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766b46d-22bc-44a3-98a3-65b7d113304e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.712761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.714436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.714436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.716498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.717711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.720592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.757698 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1981/3964] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/projects => generated 393 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:03.761806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.781260 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1984/3965] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:03.783823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:03.786671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.815216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c1349-8971-4bcb-895e-c488d3068814 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.895238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:03.933079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.934211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.934503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.934940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.936529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.940578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.946517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202a4b3-91b9-40b6-9bb5-ff7d816c90ba admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:03.961701 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1985/3966] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:03.965269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:03.966137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:03.966571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:03.967020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:03.967542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:03.968733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:03.971048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:03.991991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d84150-5c0b-4a3f-91f7-e7b37c3e1002 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:04.001462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:04.002419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.002594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.002890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.004779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:04.008270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:04.033027 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1986/3967] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:04.035072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:04.037710 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1982/3968] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:03 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:04.041751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:04.056736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0121d6e-385e-4418-a021-d20901c17962 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:04.062244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf9d82-ca23-4e53-8564-8010aac5e8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:04.062468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd3c23-4225-4249-baad-63f41d88445e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:04.063481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf9d82-ca23-4e53-8564-8010aac5e8ac admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.063763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf9d82-ca23-4e53-8564-8010aac5e8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.064037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf9d82-ca23-4e53-8564-8010aac5e8ac admin admin] PATH_INFO: `/v3/projects/a58f02659fe74b70a8657691faaad6c9/users/4faa458c158141efaf707317411cd544/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.081338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:04.082009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.082373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.082684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.083453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:04.084971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:04.089635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf9d82-ca23-4e53-8564-8010aac5e8ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58f02659fe74b70a8657691faaad6c9, user_id=4faa458c158141efaf707317411cd544, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:04.098051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf9d82-ca23-4e53-8564-8010aac5e8ac admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:04.107381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1983/3969] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:04.108233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:04.134960 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1987/3970] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:04 2022] PUT /identity/v3/projects/a58f02659fe74b70a8657691faaad6c9/users/4faa458c158141efaf707317411cd544/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:04.141859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.143055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.143055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.148446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5d35a-bd72-4f34-897f-9c0a1adc1456 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:04.204230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:04.213273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:04.293742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:04.294413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.294696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.294978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.295628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:04.302674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:04.348362 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1984/3971] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:04.349595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:04.377908 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1988/3972] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:04.378836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96689cf3-9f0a-4b91-b8c0-c963ec424f27 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:04.394428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:04.395116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.395402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.395679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] PATH_INFO: `/v3/projects/86ac1a286f2a468dad5296c209919b88/users/31c9521425ef447b873dbce6841775d8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.428503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:04.430906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:04.456380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86ac1a286f2a468dad5296c209919b88, user_id=31c9521425ef447b873dbce6841775d8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:04.463595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:04.508847 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1985/3973] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:04 2022] PUT /identity/v3/projects/86ac1a286f2a468dad5296c209919b88/users/31c9521425ef447b873dbce6841775d8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:04.510686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:04.513390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:04.558076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eeb4bb-7869-44eb-89b3-3fbb384b4b43 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:04.654534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47d4df-7d90-47fe-9afa-aa38a48fe43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:04.655793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47d4df-7d90-47fe-9afa-aa38a48fe43b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.656378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47d4df-7d90-47fe-9afa-aa38a48fe43b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.656905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47d4df-7d90-47fe-9afa-aa38a48fe43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.684162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:04.706186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:04.709966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb248d-b2e7-405b-ade3-8634feb2f048 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:04.725311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:04.726013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.726351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.726631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] PATH_INFO: `/v3/users/2eb69f3b88f840c09d4bb2bf69217f96` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.730733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb69f3b88f840c09d4bb2bf69217f96)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:04.732782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:04.805980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47d4df-7d90-47fe-9afa-aa38a48fe43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:04.808623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47d4df-7d90-47fe-9afa-aa38a48fe43b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:04.835733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1989/3974] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:04.840069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.840525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.840916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:04.851412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb69f3b88f840c09d4bb2bf69217f96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:04.886933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb69f3b88f840c09d4bb2bf69217f96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:04.887989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb69f3b88f840c09d4bb2bf69217f96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:04.903808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb69f3b88f840c09d4bb2bf69217f96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:04.904668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d95d562c-fe1c-4fdc-a5c7-5a32252057bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb69f3b88f840c09d4bb2bf69217f96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:04.914430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] MFA Rules not processed for user `31c9521425ef447b873dbce6841775d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:04.916354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:04.950112 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1986/3975] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:04 2022] DELETE /identity/v3/users/2eb69f3b88f840c09d4bb2bf69217f96 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:04.955584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:04.955911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:04.956138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.011944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:05.013283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:05.101989 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1990/3976] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:05.107021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:05.113407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:05.161187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ff5c-7d07-40ed-bec5-85e67c7aad71 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:05.174444 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1987/3977] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:05.175142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:05.175882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.179093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.179858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.180231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:05.182375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:05.195237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:05.196771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:05.227474 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1991/3978] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:05 2022] POST /identity/v3/projects => generated 395 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:05.231185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:05.288922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91fcd2-3055-4097-9d78-dea8dfac1d6f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:05.328599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:05.359729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f0ddd-1896-4de6-a608-f8c4ca8d70e1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:05.360048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:05.360608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.360884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.361155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.366401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:05.367374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.367986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.368476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.369330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:05.371630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:05.372837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:05.373522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:05.393166 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1988/3979] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:02:05.419067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:05.428181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1992/3980] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:05 2022] POST /identity/v3/users => generated 304 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:05.433334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade9666-30f0-4d07-a76a-b3d7517530a2 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:05.439012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:05.440663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.441188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.441670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.442544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:05.444606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:05.458562 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1989/3981] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:05.464650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:05.479444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a193f79-ef64-419e-a8a3-03ad9348ffa1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:05.489079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0e710-07b9-40bf-ae72-64ac8e539cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:05.490665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e710-07b9-40bf-ae72-64ac8e539cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.491167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e710-07b9-40bf-ae72-64ac8e539cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.491653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e710-07b9-40bf-ae72-64ac8e539cb8 admin admin] PATH_INFO: `/v3/projects/8974d33f1cbf4f06910af6d996642ecb/users/2c7f59c488c94001a45c8f0cb5f4829d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.516762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:05.524690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0e710-07b9-40bf-ae72-64ac8e539cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8974d33f1cbf4f06910af6d996642ecb, user_id=2c7f59c488c94001a45c8f0cb5f4829d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:05.531898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0e710-07b9-40bf-ae72-64ac8e539cb8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:05.563776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:05.564265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1993/3982] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:05 2022] PUT /identity/v3/projects/8974d33f1cbf4f06910af6d996642ecb/users/2c7f59c488c94001a45c8f0cb5f4829d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:05.581516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a76c8a-d09e-44f3-a5e5-4944110551c9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:05.823918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dbb6f-9cb7-4e8a-bc98-a00271db7e85 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.824209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dbb6f-9cb7-4e8a-bc98-a00271db7e85 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.824483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dbb6f-9cb7-4e8a-bc98-a00271db7e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.831529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:05.845483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.847659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.848045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.849778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:05.852799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:05.870548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-212dbb6f-9cb7-4e8a-bc98-a00271db7e85 None None] MFA Rules not processed for user `b638a9e7f0934e47a3883be65e42d558`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:05.875083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-212dbb6f-9cb7-4e8a-bc98-a00271db7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:05.888464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1990/3983] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:05.891589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:05.899543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:05.928785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc82a01-139d-45b2-8191-21225ef357b2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:05.949108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:05.949740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:05.950190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:05.950608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] PATH_INFO: `/v3/projects/cd988a3e226749e09043d36d3d610d58` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:05.952077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd988a3e226749e09043d36d3d610d58)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:05.955097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:06.034538 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1994/3984] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:06.046751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dbb6f-9cb7-4e8a-bc98-a00271db7e85 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:06.085657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd988a3e226749e09043d36d3d610d58', 'request_id': 'req-821effcc-e441-43e0-a3ba-1a48ff46b631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.090123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dbb6f-9cb7-4e8a-bc98-a00271db7e85 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:06.101916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd988a3e226749e09043d36d3d610d58', 'request_id': 'req-821effcc-e441-43e0-a3ba-1a48ff46b631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.127130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd988a3e226749e09043d36d3d610d58', 'request_id': 'req-821effcc-e441-43e0-a3ba-1a48ff46b631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.150933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] The token cache is being invalidate because project cd988a3e226749e09043d36d3d610d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:06.152177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.153852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.157223 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1991/3985] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:05 2022] DELETE /identity/v3/projects/cd988a3e226749e09043d36d3d610d58 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:06.161148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:06.164952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:06.244809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa74e7-cf78-43e5-bb82-b6ce2be4f152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:06.245557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa74e7-cf78-43e5-bb82-b6ce2be4f152 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:06.245925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa74e7-cf78-43e5-bb82-b6ce2be4f152 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:06.246216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa74e7-cf78-43e5-bb82-b6ce2be4f152 admin admin] PATH_INFO: `/v3/projects/8974d33f1cbf4f06910af6d996642ecb/users/2c7f59c488c94001a45c8f0cb5f4829d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:06.251994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa74e7-cf78-43e5-bb82-b6ce2be4f152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8974d33f1cbf4f06910af6d996642ecb, user_id=2c7f59c488c94001a45c8f0cb5f4829d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:06.254639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa74e7-cf78-43e5-bb82-b6ce2be4f152 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:06.279540 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1995/3986] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:06 2022] PUT /identity/v3/projects/8974d33f1cbf4f06910af6d996642ecb/users/2c7f59c488c94001a45c8f0cb5f4829d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:06.282699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:06.283115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:06.284581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:06.285122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:06.322045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821effcc-e441-43e0-a3ba-1a48ff46b631 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:06.351420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] MFA Rules not processed for user `2c7f59c488c94001a45c8f0cb5f4829d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:06.353053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:06.445966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:06.447187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:06.447187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:06.447454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] PATH_INFO: `/v3/users/4b3431297f8f4c2cbe65d8c6372125aa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:06.449848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b3431297f8f4c2cbe65d8c6372125aa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:06.453797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:06.555557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1996/3987] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:06.592423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b3431297f8f4c2cbe65d8c6372125aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.602952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b3431297f8f4c2cbe65d8c6372125aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.604126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b3431297f8f4c2cbe65d8c6372125aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.616244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b3431297f8f4c2cbe65d8c6372125aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.617074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b3431297f8f4c2cbe65d8c6372125aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:06.644878 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1992/3988] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:06 2022] DELETE /identity/v3/users/4b3431297f8f4c2cbe65d8c6372125aa => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:06.763739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:06.766902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:06.806494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:07.016671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dba98-ae03-4c45-ad85-16748f44c6f8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:07.045126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:07.047765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:07.180081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee6184-5b89-407e-834d-af691c7086ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:07.180962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6184-5b89-407e-834d-af691c7086ce service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:07.180962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6184-5b89-407e-834d-af691c7086ce service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:07.181274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6184-5b89-407e-834d-af691c7086ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:07.183414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-64ee6184-5b89-407e-834d-af691c7086ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:07.246957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d2322-93f5-4d5c-aec8-eb45ca424efe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:07.261757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:07.262438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:07.262721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:07.262991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] PATH_INFO: `/v3/projects/10d84a67488044eaaee66bd000de05de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:07.265809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d84a67488044eaaee66bd000de05de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:07.272504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:07.294308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:07.316672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee6184-5b89-407e-834d-af691c7086ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:07.340166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee6184-5b89-407e-834d-af691c7086ce service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:07.367340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1997/3989] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:07.405852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d84a67488044eaaee66bd000de05de', 'request_id': 'req-80825b78-3795-494f-9844-6efe907d2fe1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:07.424823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d84a67488044eaaee66bd000de05de', 'request_id': 'req-80825b78-3795-494f-9844-6efe907d2fe1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:07.440469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d84a67488044eaaee66bd000de05de', 'request_id': 'req-80825b78-3795-494f-9844-6efe907d2fe1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:07.478626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] The token cache is being invalidate because project 10d84a67488044eaaee66bd000de05de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:07.478928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:07.482120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-80825b78-3795-494f-9844-6efe907d2fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:07.488546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1993/3990] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:07 2022] DELETE /identity/v3/projects/10d84a67488044eaaee66bd000de05de => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:07.490713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5f1e-f93f-4188-9e89-432d0d6dc9ad None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:07.491004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5f1e-f93f-4188-9e89-432d0d6dc9ad None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:07.491421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd5f1e-f93f-4188-9e89-432d0d6dc9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:07.538303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9efd5f1e-f93f-4188-9e89-432d0d6dc9ad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:07.543731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9efd5f1e-f93f-4188-9e89-432d0d6dc9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:07.675749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:07.723311 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1998/3991] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:07.725535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afad4de-d431-490b-a610-071821d1b6b5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:07.725848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afad4de-d431-490b-a610-071821d1b6b5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:07.726133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afad4de-d431-490b-a610-071821d1b6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:07.754085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7afad4de-d431-490b-a610-071821d1b6b5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:07.756893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7afad4de-d431-490b-a610-071821d1b6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:07.779351 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1994/3992] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:07.781679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd5f1e-f93f-4188-9e89-432d0d6dc9ad None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:07.798770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd5f1e-f93f-4188-9e89-432d0d6dc9ad None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:07.805530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:07.806192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:07.806474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:07.806808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:07.807796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:07.808317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:07.811370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:07.836878 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 1999/3993] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:07.838308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afad4de-d431-490b-a610-071821d1b6b5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:07.854660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afad4de-d431-490b-a610-071821d1b6b5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:07.861479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:07.862781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:07.863065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:07.863396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:07.864280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:07.869358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:07.920200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1995/3994] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:07 2022] POST /identity/v3/projects => generated 369 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:07.924744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:07.955625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6e5a6-68ed-46b3-8c86-c987d0839cba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:08.151493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:08.152453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:08.152453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:08.152672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:08.153290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:08.155797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:08.226326 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2000/3995] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:07 2022] POST /identity/v3/users => generated 285 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:08.228683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:08.243761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07c4c-95ba-4b43-bb0a-860d71a550d0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:08.256892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:08.265794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:08.267031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:08.267031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:08.267484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:08.269366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:08.294556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:08.303159 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1996/3996] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:08.315152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ec347-8e40-4494-bd72-88b5be20dc70 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:08.330049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2eee6-1925-44d5-b7ce-fff4b21c63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:08.330808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2eee6-1925-44d5-b7ce-fff4b21c63eb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:08.331196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2eee6-1925-44d5-b7ce-fff4b21c63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:08.331477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2eee6-1925-44d5-b7ce-fff4b21c63eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:08.332130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2eee6-1925-44d5-b7ce-fff4b21c63eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:08.333796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2eee6-1925-44d5-b7ce-fff4b21c63eb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:08.357704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2001/3997] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:02:08.363366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:08.375935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5d681-45cc-4d6b-a83c-26ceb3dcc392 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:08.385087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:08.385747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:08.386184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:08.386472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] PATH_INFO: `/v3/projects/302c207c0fc84ed6ad230678ac644108/users/9d6faed7dfdb4655b0574edcb88fcd9e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:08.410479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=302c207c0fc84ed6ad230678ac644108, user_id=9d6faed7dfdb4655b0574edcb88fcd9e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:08.417490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:08.446286 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1997/3998] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:08 2022] PUT /identity/v3/projects/302c207c0fc84ed6ad230678ac644108/users/9d6faed7dfdb4655b0574edcb88fcd9e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:08.454311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7937eb-78ce-4b4a-859c-0eee39b245fc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:08.454599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7937eb-78ce-4b4a-859c-0eee39b245fc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:08.454928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7937eb-78ce-4b4a-859c-0eee39b245fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:08.502680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7b7937eb-78ce-4b4a-859c-0eee39b245fc None None] MFA Rules not processed for user `9d6faed7dfdb4655b0574edcb88fcd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:08.503935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b7937eb-78ce-4b4a-859c-0eee39b245fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:08.603990 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2002/3999] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:08.606154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:08.694988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab202ff4-5d5b-4800-a3b1-2d88fab276fe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:08.700294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:08.700923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:08.701210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:08.701485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:08.702891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:08.704905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:08.755628 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1998/4000] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:08 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:08.762489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7937eb-78ce-4b4a-859c-0eee39b245fc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:08.829914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7937eb-78ce-4b4a-859c-0eee39b245fc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:09.049079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:09.105044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:09.105675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:09.105987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:09.106264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:09.106903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:09.109279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:09.155827 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2003/4001] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:08 2022] POST /identity/v3/users => generated 291 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:09.161038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:09.182312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254c970-4fb7-482f-a64e-4c5dda452fd9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:09.201645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:09.207000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:09.207293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:09.207573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:09.208243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:09.209890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:09.265375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:09.266352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 1999/4002] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:09.285190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2175219-d932-4b6f-b2ec-0e75bbeee2cc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:09.314058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:09.315165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:09.315782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:09.316294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] PATH_INFO: `/v3/projects/d1c3701208984be8ad70e79602180ab2/users/933b9d039f1a4fd3b3ffc01099887f8f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:09.353699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c3701208984be8ad70e79602180ab2, user_id=933b9d039f1a4fd3b3ffc01099887f8f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:09.357769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:09.423294 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2004/4003] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:09 2022] PUT /identity/v3/projects/d1c3701208984be8ad70e79602180ab2/users/933b9d039f1a4fd3b3ffc01099887f8f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:09.427207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:09.445888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238c51-37bb-49c3-ae79-96883991d56f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:09.458680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:09.612696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a8b9a-f5c6-4d68-bac8-99ac327046a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:09.613330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a8b9a-f5c6-4d68-bac8-99ac327046a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:09.613608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a8b9a-f5c6-4d68-bac8-99ac327046a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:09.613954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a8b9a-f5c6-4d68-bac8-99ac327046a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:09.614607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a8b9a-f5c6-4d68-bac8-99ac327046a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:09.616299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a8b9a-f5c6-4d68-bac8-99ac327046a1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:09.641924 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2000/4004] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:09.643065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:09.658022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ceb83f-f422-4aba-ac57-c31d348880b2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:09.664157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea47ed5-72d8-4306-ab4b-15a146558875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:09.665235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea47ed5-72d8-4306-ab4b-15a146558875 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:09.665528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea47ed5-72d8-4306-ab4b-15a146558875 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:09.665809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea47ed5-72d8-4306-ab4b-15a146558875 admin admin] PATH_INFO: `/v3/projects/d1c3701208984be8ad70e79602180ab2/users/933b9d039f1a4fd3b3ffc01099887f8f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:09.668001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea47ed5-72d8-4306-ab4b-15a146558875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c3701208984be8ad70e79602180ab2, user_id=933b9d039f1a4fd3b3ffc01099887f8f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:09.671031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea47ed5-72d8-4306-ab4b-15a146558875 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:09.694401 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2005/4005] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:09 2022] PUT /identity/v3/projects/d1c3701208984be8ad70e79602180ab2/users/933b9d039f1a4fd3b3ffc01099887f8f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:09.696585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:09.698002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:09.698452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:09.744356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] MFA Rules not processed for user `933b9d039f1a4fd3b3ffc01099887f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:09.745963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:09.850247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe6958-55da-41ca-87e1-4ad7a0ea3dda None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:09.850543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe6958-55da-41ca-87e1-4ad7a0ea3dda None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:09.850824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe6958-55da-41ca-87e1-4ad7a0ea3dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:09.894479 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2001/4006] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:09.927486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:09.927486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:09.928035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5dfe6958-55da-41ca-87e1-4ad7a0ea3dda None None] MFA Rules not processed for user `6f50ca756d594703846f1dade2d38eac`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:09.928035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5dfe6958-55da-41ca-87e1-4ad7a0ea3dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:10.107751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91685ecc-cf31-4286-a08a-5a6bcd965785 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:10.116749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:10.117373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:10.117658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:10.117962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:10.144254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:10.146504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:10.167391 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2006/4007] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:10.169032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:10.169323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:10.169600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:10.184191 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2002/4008] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:10.197264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] MFA Rules not processed for user `6f50ca756d594703846f1dade2d38eac`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:10.198711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:10.211723 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2007/4009] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:10.272897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:10.274621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:10.344926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:10.413386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8add738-26ca-45e2-b186-699051a46f07 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:10.421588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ffd4f-7b24-4f7e-a74b-c018f6d4ce7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:10.422915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ffd4f-7b24-4f7e-a74b-c018f6d4ce7d service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:10.423819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ffd4f-7b24-4f7e-a74b-c018f6d4ce7d service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:10.434303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:10.437408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:10.440361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ffd4f-7b24-4f7e-a74b-c018f6d4ce7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:10.450147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f07ffd4f-7b24-4f7e-a74b-c018f6d4ce7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:10.475350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc02a6-a671-4508-bf2d-a0ab91fb12ab None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:10.492045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:10.492978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:10.493268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:10.493547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:10.509070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:10.511035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:10.528832 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2008/4010] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:10.571347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ffd4f-7b24-4f7e-a74b-c018f6d4ce7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:10.573809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ffd4f-7b24-4f7e-a74b-c018f6d4ce7d service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:10.601960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2003/4011] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:10.618989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:10.623449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:10.808059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2b5a4-e25f-4f2b-9717-55019834c49b service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:10.815382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:10.816330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:10.816802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:10.817282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] PATH_INFO: `/v3/users/31c9521425ef447b873dbce6841775d8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:10.818895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c9521425ef447b873dbce6841775d8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:10.822031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:10.825944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:11.017063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c9521425ef447b873dbce6841775d8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:11.047653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c9521425ef447b873dbce6841775d8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:11.081058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c9521425ef447b873dbce6841775d8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:11.109327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c9521425ef447b873dbce6841775d8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:11.110569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c9521425ef447b873dbce6841775d8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:11.136910 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2009/4012] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:10 2022] DELETE /identity/v3/users/31c9521425ef447b873dbce6841775d8 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:11.138623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:11.139025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:11.139592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:11.184717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:11.186721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:11.333627 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2004/4013] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:11.590410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:11.591876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:11.755814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226bf33-062a-4c37-8e6d-ed9bef68f078 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:11.780693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:11.782106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:11.784538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:11.786373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:11.845244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:11.848617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:11.900103 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2010/4014] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:14.172556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.175075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:14.204135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7aabb7-b5f4-4e25-9d14-f8eccdef6cad None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.210509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.212188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.212761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.213280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] PATH_INFO: `/v3/projects/86ac1a286f2a468dad5296c209919b88` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.215473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86ac1a286f2a468dad5296c209919b88)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.220040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.370587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86ac1a286f2a468dad5296c209919b88', 'request_id': 'req-06584727-e09a-4e3f-aba6-76f092f3a7d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.388274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86ac1a286f2a468dad5296c209919b88', 'request_id': 'req-06584727-e09a-4e3f-aba6-76f092f3a7d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.410187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86ac1a286f2a468dad5296c209919b88', 'request_id': 'req-06584727-e09a-4e3f-aba6-76f092f3a7d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.413857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:14.443879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] The token cache is being invalidate because project 86ac1a286f2a468dad5296c209919b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:14.444265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.446524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.456179 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2005/4015] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:14 2022] DELETE /identity/v3/projects/86ac1a286f2a468dad5296c209919b88 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:14.457651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:14.487648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57e1a0-df81-4dbb-9f54-f3156f79ee9b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:14.665592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:14.666288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:14.666573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:14.666851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] PATH_INFO: `/v3/users/2c7f59c488c94001a45c8f0cb5f4829d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:14.668021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7f59c488c94001a45c8f0cb5f4829d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:14.670020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:14.747507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:14.789510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7f59c488c94001a45c8f0cb5f4829d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.814040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7f59c488c94001a45c8f0cb5f4829d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.826509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7f59c488c94001a45c8f0cb5f4829d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.845950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7f59c488c94001a45c8f0cb5f4829d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.847433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b779b-ad7b-45f7-bf90-c74c2d2b01b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7f59c488c94001a45c8f0cb5f4829d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:14.881988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2011/4016] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:14 2022] DELETE /identity/v3/users/2c7f59c488c94001a45c8f0cb5f4829d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:16.573875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:16.574963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:16.604259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06584727-e09a-4e3f-aba6-76f092f3a7d3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:16.712704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:16.713356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:16.713639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:16.714182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] PATH_INFO: `/v3/projects/8974d33f1cbf4f06910af6d996642ecb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:16.715944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8974d33f1cbf4f06910af6d996642ecb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:16.721191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:16.839352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8974d33f1cbf4f06910af6d996642ecb', 'request_id': 'req-f49f738c-bf13-450f-b177-f15a3aed703f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:16.850610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8974d33f1cbf4f06910af6d996642ecb', 'request_id': 'req-f49f738c-bf13-450f-b177-f15a3aed703f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:16.863141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8974d33f1cbf4f06910af6d996642ecb', 'request_id': 'req-f49f738c-bf13-450f-b177-f15a3aed703f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:16.881260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] The token cache is being invalidate because project 8974d33f1cbf4f06910af6d996642ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:16.881606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:16.883702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f49f738c-bf13-450f-b177-f15a3aed703f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:16.887555 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2006/4017] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:16 2022] DELETE /identity/v3/projects/8974d33f1cbf4f06910af6d996642ecb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:16.896260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04e8d6-bc83-470c-b188-c85c540f2f6f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:16.897295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04e8d6-bc83-470c-b188-c85c540f2f6f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:16.897933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04e8d6-bc83-470c-b188-c85c540f2f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:16.951118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7d04e8d6-bc83-470c-b188-c85c540f2f6f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:16.954264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7d04e8d6-bc83-470c-b188-c85c540f2f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:17.095926 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2012/4018] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:17.097473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63408-23e5-4bfc-8e83-5c6fbf638117 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:17.098395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63408-23e5-4bfc-8e83-5c6fbf638117 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:17.098931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63408-23e5-4bfc-8e83-5c6fbf638117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:17.181728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-99c63408-23e5-4bfc-8e83-5c6fbf638117 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:17.198673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-99c63408-23e5-4bfc-8e83-5c6fbf638117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:17.231426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04e8d6-bc83-470c-b188-c85c540f2f6f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:17.255297 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2007/4019] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:17.260619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04e8d6-bc83-470c-b188-c85c540f2f6f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:17.273941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:17.274556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:17.274839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:17.275115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:17.275771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:17.276429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:17.284243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:17.336527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63408-23e5-4bfc-8e83-5c6fbf638117 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:17.360280 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2013/4020] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:02:17.370456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63408-23e5-4bfc-8e83-5c6fbf638117 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:17.396916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:17.397553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:17.401668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:17.402065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:17.402734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:17.405235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:17.487586 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2008/4021] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:17 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:17.491805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:17.506334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd34668-5f87-4187-9eb5-fe74751a6139 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:17.677983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:17.678613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:17.678904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:17.679232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:17.680498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:17.682634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:17.737327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:17.753641 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2014/4022] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:17 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:17.754001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b543f0-b72b-4d42-a56b-4a68f0d39799 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:17.770291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:17.772433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:17.772900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:17.774509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:17.777617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:17.779621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:17.824448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:17.849372 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2009/4023] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:02:17.854186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7562c4-2f18-4ac4-819e-bcf2dda7861a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:17.865555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437db8c5-9326-4013-9615-4ff0a17ff911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:17.866433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db8c5-9326-4013-9615-4ff0a17ff911 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:17.866719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db8c5-9326-4013-9615-4ff0a17ff911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:17.867044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db8c5-9326-4013-9615-4ff0a17ff911 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:17.867691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437db8c5-9326-4013-9615-4ff0a17ff911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:17.869291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437db8c5-9326-4013-9615-4ff0a17ff911 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:17.906724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:17.927250 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2015/4024] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:17.929914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050340c-abe8-4349-9dca-8f8c8487dbfb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:17.935489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:17.937212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:17.937558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:17.937867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] PATH_INFO: `/v3/projects/1da09fdc751e406f9c3bad4bf0e1b067/users/96372af27d29482491d613b5a783a0d1/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:17.958081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da09fdc751e406f9c3bad4bf0e1b067, user_id=96372af27d29482491d613b5a783a0d1, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:17.960629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:18.003653 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2010/4025] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:17 2022] PUT /identity/v3/projects/1da09fdc751e406f9c3bad4bf0e1b067/users/96372af27d29482491d613b5a783a0d1/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:18.013499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472e086-ed4d-44aa-9dcb-8fd90ec73574 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:18.016536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472e086-ed4d-44aa-9dcb-8fd90ec73574 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:18.017052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472e086-ed4d-44aa-9dcb-8fd90ec73574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:18.075502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2472e086-ed4d-44aa-9dcb-8fd90ec73574 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:18.077440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2472e086-ed4d-44aa-9dcb-8fd90ec73574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:18.090248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:18.205256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2016/4026] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:18.241817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:18.251595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:18.315423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:18.539513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3cfb5-0ec6-4852-aa26-0b34684e1882 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:18.565600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dad4e-7743-4aea-b8a5-c03599c762b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:18.570513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dad4e-7743-4aea-b8a5-c03599c762b9 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:18.570945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dad4e-7743-4aea-b8a5-c03599c762b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:18.572277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dad4e-7743-4aea-b8a5-c03599c762b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:18.626260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dad4e-7743-4aea-b8a5-c03599c762b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:18.638674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dad4e-7743-4aea-b8a5-c03599c762b9 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:18.675210 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2011/4027] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:20.978369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:20.979018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:20.979018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.050347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:21.051859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:21.152324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2017/4028] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:21.156586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.157143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.157143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.177760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:21.179099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:21.200064 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2012/4029] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:21.209764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:21.211293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:21.353395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e0e76-b5a7-4f2d-8843-ca1f4023fbfa None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:21.363703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:21.365742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:21.366245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:21.367389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:21.382096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:21.383932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:21.398953 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2018/4030] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:24.277028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:24.278622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:24.444590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a9e51-ad82-4d14-88fe-15d2c093b1a1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:24.450845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:24.454278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:24.454740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:24.455080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] PATH_INFO: `/v3/users/9b9b59fb1a184b958ea619062412cf2d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:24.456437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9b59fb1a184b958ea619062412cf2d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:24.458901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:24.567611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9b59fb1a184b958ea619062412cf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:24.582538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9b59fb1a184b958ea619062412cf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:24.584657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9b59fb1a184b958ea619062412cf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:24.600640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9b59fb1a184b958ea619062412cf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:24.601433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8a4cf6f6-c91c-407b-84d6-1b8ce13f5bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9b59fb1a184b958ea619062412cf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:24.667028 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2013/4031] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:24 2022] DELETE /identity/v3/users/9b9b59fb1a184b958ea619062412cf2d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:26.494560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:26.514747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:26.538639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d39c5-6897-4a90-a3b0-a74f4ef91781 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:26.610003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:26.610649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:26.610934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:26.611212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] PATH_INFO: `/v3/projects/243cfbc91bb24e7ca96f348cd21f1adb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:26.612639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243cfbc91bb24e7ca96f348cd21f1adb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:26.614610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:26.669639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243cfbc91bb24e7ca96f348cd21f1adb', 'request_id': 'req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:26.677052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243cfbc91bb24e7ca96f348cd21f1adb', 'request_id': 'req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:26.684007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243cfbc91bb24e7ca96f348cd21f1adb', 'request_id': 'req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:26.700673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] The token cache is being invalidate because project 243cfbc91bb24e7ca96f348cd21f1adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:26.700893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:26.702533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-36289f7f-6d30-44ee-b8f3-cd0f455aaba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:26.707706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2019/4032] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:26 2022] DELETE /identity/v3/projects/243cfbc91bb24e7ca96f348cd21f1adb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:26.709116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5753-3e23-4a7c-a834-98f33d781705 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:26.710606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5753-3e23-4a7c-a834-98f33d781705 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:26.711999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5753-3e23-4a7c-a834-98f33d781705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:26.783884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-abed5753-3e23-4a7c-a834-98f33d781705 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:26.785636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-abed5753-3e23-4a7c-a834-98f33d781705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:26.803887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:26.943449 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2014/4033] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:26.948489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f5015-f5a9-4bd5-8842-b2784202b2c0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:26.949185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f5015-f5a9-4bd5-8842-b2784202b2c0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:26.949185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f5015-f5a9-4bd5-8842-b2784202b2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:26.995010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ac9f5015-f5a9-4bd5-8842-b2784202b2c0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:26.996412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ac9f5015-f5a9-4bd5-8842-b2784202b2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:27.031422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2020/4034] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:27.034547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5753-3e23-4a7c-a834-98f33d781705 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:27.045057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5753-3e23-4a7c-a834-98f33d781705 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:27.061296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:27.061987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:27.062425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:27.062711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:27.063525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:27.063873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:27.065535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:27.096616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f5015-f5a9-4bd5-8842-b2784202b2c0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:27.101209 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2015/4035] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:27.130733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f5015-f5a9-4bd5-8842-b2784202b2c0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:27.146650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:27.149640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:27.150120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:27.150592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:27.151397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:27.157883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:27.233292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:27.234570 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2021/4036] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:27 2022] POST /identity/v3/projects => generated 397 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:27.244428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06410db1-9ecc-4c00-9bb7-4ef64c4f6fbc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:27.318957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:27.319582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:27.319898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:27.320154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:27.320762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:27.322758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:27.372487 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2016/4037] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:27 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:27.377106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:27.393240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8937c23-db44-45a7-90a9-c37ebb087432 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:27.404018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:27.406668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:27.407175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:27.407631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:27.408459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:27.411451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:27.450441 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2022/4038] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:27.455190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:27.484261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90755480-9af8-4787-a532-9cc763422aa1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:27.490554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d648b74-dc4c-4624-b646-c27fb9f93cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:27.491560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648b74-dc4c-4624-b646-c27fb9f93cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:27.492055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648b74-dc4c-4624-b646-c27fb9f93cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:27.493138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648b74-dc4c-4624-b646-c27fb9f93cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:27.494024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648b74-dc4c-4624-b646-c27fb9f93cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:27.496423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648b74-dc4c-4624-b646-c27fb9f93cd4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:27.514218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2017/4039] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:27.518540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:27.531094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b791b-6a35-4254-95f3-114477a070e8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:27.540789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:27.541481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:27.541765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:27.542077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] PATH_INFO: `/v3/projects/daa76318457c400b80564b1035b25633/users/edef2887c6b741108da9b88745486339/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:27.581633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa76318457c400b80564b1035b25633, user_id=edef2887c6b741108da9b88745486339, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:27.584453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:27.631198 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2023/4040] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:27 2022] PUT /identity/v3/projects/daa76318457c400b80564b1035b25633/users/edef2887c6b741108da9b88745486339/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:27.636698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:27.636698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:27.637021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:27.681389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:27.683831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:27.820305 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2018/4041] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:27.864537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:27.866846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:27.923783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:28.107161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3b8f3-22b6-4c51-8a1b-5b328c894e58 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:28.113729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:28.115169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:28.115599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:28.115806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:28.133219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:28.134753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:28.149273 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2024/4042] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:32.328186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:32.330279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:32.348185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49de29-6f34-41ef-968b-16968daaff19 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:32.364011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:32.365454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:32.365957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:32.366425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] PATH_INFO: `/v3/users/2bed1a15bbff41c6928afd3c057e7dfc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:32.370094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bed1a15bbff41c6928afd3c057e7dfc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:32.372851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:32.433421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:32.480846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bed1a15bbff41c6928afd3c057e7dfc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:32.504858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bed1a15bbff41c6928afd3c057e7dfc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:32.505707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bed1a15bbff41c6928afd3c057e7dfc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:32.521157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bed1a15bbff41c6928afd3c057e7dfc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:32.522091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1ec59a28-44c8-49d0-9190-d8bf04c20b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bed1a15bbff41c6928afd3c057e7dfc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:32.554032 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2019/4043] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:32 2022] DELETE /identity/v3/users/2bed1a15bbff41c6928afd3c057e7dfc => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:33.778986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:33.796042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024b83c-c927-4d98-89b0-159a612a99f1 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:33.899829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:33.900774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:33.901206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:33.901673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] PATH_INFO: `/v3/projects/d640711fd8c64403adc4d7e7235a8fad` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:33.909689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d640711fd8c64403adc4d7e7235a8fad)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:33.912117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:34.008791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd640711fd8c64403adc4d7e7235a8fad', 'request_id': 'req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:34.023729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd640711fd8c64403adc4d7e7235a8fad', 'request_id': 'req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:34.037767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd640711fd8c64403adc4d7e7235a8fad', 'request_id': 'req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:34.059726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] The token cache is being invalidate because project d640711fd8c64403adc4d7e7235a8fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:34.060022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:34.060992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70e91d0c-edf9-4b79-94a9-30b5ab6280ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:34.062660 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2025/4044] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:33 2022] DELETE /identity/v3/projects/d640711fd8c64403adc4d7e7235a8fad => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:34.068731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19204b90-52ad-4863-9485-b0a0ce67ce98 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.069213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19204b90-52ad-4863-9485-b0a0ce67ce98 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.069336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19204b90-52ad-4863-9485-b0a0ce67ce98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.142949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-19204b90-52ad-4863-9485-b0a0ce67ce98 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:34.144256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-19204b90-52ad-4863-9485-b0a0ce67ce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:34.372759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2020/4045] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:34.374706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2629f-e574-4a2d-98d9-cf1ed14b8827 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.374995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2629f-e574-4a2d-98d9-cf1ed14b8827 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.375271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2629f-e574-4a2d-98d9-cf1ed14b8827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.414193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-11d2629f-e574-4a2d-98d9-cf1ed14b8827 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:34.415563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-11d2629f-e574-4a2d-98d9-cf1ed14b8827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:34.436709 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2026/4046] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:34.441063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19204b90-52ad-4863-9485-b0a0ce67ce98 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:34.453222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19204b90-52ad-4863-9485-b0a0ce67ce98 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:34.463317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:34.464158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.464570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.464875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.468683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:34.468683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:34.468683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:34.494975 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2021/4047] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:34.495891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2629f-e574-4a2d-98d9-cf1ed14b8827 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:34.509084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2629f-e574-4a2d-98d9-cf1ed14b8827 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:34.515944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:34.517261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.517771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.518358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.519288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:34.522119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:34.528986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:34.580715 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2027/4048] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:34 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:34.584525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:34.597620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9173d4-eb98-46a4-b650-ad4b09a65a79 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:34.786995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:34.787840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.788367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.789040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.789806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:34.792105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:34.852231 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2022/4049] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:34 2022] POST /identity/v3/users => generated 283 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:34.859596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:34.880499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb5b39-1b1e-476b-94d1-09baa7f6c816 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:34.905122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:34.906002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.906433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.906811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.907571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:34.915779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:34.945588 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2028/4050] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:34.949848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:34.968068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bd2ad-0af1-46c5-abe5-390569992917 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:34.991274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53c9b3-b0fb-4713-bc25-78a9a799d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:34.992373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53c9b3-b0fb-4713-bc25-78a9a799d20d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:34.993042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53c9b3-b0fb-4713-bc25-78a9a799d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:34.993540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53c9b3-b0fb-4713-bc25-78a9a799d20d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:34.994472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53c9b3-b0fb-4713-bc25-78a9a799d20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:34.996363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53c9b3-b0fb-4713-bc25-78a9a799d20d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:35.035541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:35.050573 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2023/4051] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:35.051349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6672341-2b78-4fce-98d1-d1f377afc853 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:35.058138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:35.059235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:35.059739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:35.060240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] PATH_INFO: `/v3/projects/41b446a06aed49938ae24e5c69c5741a/users/e3c9cfc67bb6497fa6ee77c1c7d98fbd/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:35.090267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b446a06aed49938ae24e5c69c5741a, user_id=e3c9cfc67bb6497fa6ee77c1c7d98fbd, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:35.095356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:35.136654 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2029/4052] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:35 2022] PUT /identity/v3/projects/41b446a06aed49938ae24e5c69c5741a/users/e3c9cfc67bb6497fa6ee77c1c7d98fbd/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:35.147323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:35.148294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:35.148816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:35.161238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:35.199430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] MFA Rules not processed for user `e3c9cfc67bb6497fa6ee77c1c7d98fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:35.201281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:35.389770 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2024/4053] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:35.632413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:35.632642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:35.635220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:35.636007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:35.803805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51cb79-55d8-4090-88db-6ab5db1de258 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:35.819837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '933b9d039f1a4fd3b3ffc01099887f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1c3701208984be8ad70e79602180ab2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:35.820642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:35.820944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:35.821231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:35.821943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:35.822314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:35.824378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:35.855067 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2025/4054] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:35.859562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:35.874500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c3e98-d20f-4200-ba07-11b55f0aa779 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:35.877560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205a50d-d9f6-40e8-990e-bd861f4cb704 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:35.891310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '933b9d039f1a4fd3b3ffc01099887f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1c3701208984be8ad70e79602180ab2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:35.891948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:35.892285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:35.892519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:35.893214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:35.895227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:35.895384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b193f34-8e79-4790-a018-7154584a0123 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:35.895926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b193f34-8e79-4790-a018-7154584a0123 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:35.900420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b193f34-8e79-4790-a018-7154584a0123 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:35.902144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b193f34-8e79-4790-a018-7154584a0123 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:35.927437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b193f34-8e79-4790-a018-7154584a0123 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:35.932998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b193f34-8e79-4790-a018-7154584a0123 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:35.959319 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2026/4055] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:35 2022] POST /identity/v3/projects => generated 347 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:35.965009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:35.998478 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2030/4056] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:36.013629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2ddf-2470-4032-9f62-fa1568ff0055 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.014569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2ddf-2470-4032-9f62-fa1568ff0055 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.014802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2ddf-2470-4032-9f62-fa1568ff0055 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.016163 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2031/4057] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 21:02:36.022815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a2ddf-2470-4032-9f62-fa1568ff0055 None None] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 21:02:36.152295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dbf09-d1d3-4a9a-8179-a888ff867647 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.169522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb16cfd-5425-44e5-a801-0973cb77caa9 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '933b9d039f1a4fd3b3ffc01099887f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1c3701208984be8ad70e79602180ab2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.170354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16cfd-5425-44e5-a801-0973cb77caa9 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.170819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16cfd-5425-44e5-a801-0973cb77caa9 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.171110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16cfd-5425-44e5-a801-0973cb77caa9 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.171782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb16cfd-5425-44e5-a801-0973cb77caa9 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:36.172099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb16cfd-5425-44e5-a801-0973cb77caa9 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.173729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb16cfd-5425-44e5-a801-0973cb77caa9 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.193997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:36.200107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a2ddf-2470-4032-9f62-fa1568ff0055 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.215832 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2027/4058] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:36.252560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:36.310931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a2ddf-2470-4032-9f62-fa1568ff0055 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.321537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41b446a06aed49938ae24e5c69c5741a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.322457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.322596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.322935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] PATH_INFO: `/v3/projects/41b446a06aed49938ae24e5c69c5741a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.330793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorizing `identity:get_project(project_id=41b446a06aed49938ae24e5c69c5741a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.332642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.346870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2032/4059] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 21:02:36 2022] GET /identity/v3/projects/41b446a06aed49938ae24e5c69c5741a => generated 365 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:36.381860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51096663-127a-4606-aff8-9af278b4d07e None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.382587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51096663-127a-4606-aff8-9af278b4d07e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.383077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51096663-127a-4606-aff8-9af278b4d07e None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.384630 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2028/4060] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:02:36.391570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 21:02:36.394641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:36.430580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.468008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f81004-9503-461a-a63a-b8489c445095 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.488179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41b446a06aed49938ae24e5c69c5741a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.489273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.489799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.490420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] PATH_INFO: `/v3/projects/41b446a06aed49938ae24e5c69c5741a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.492091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorizing `identity:get_project(project_id=41b446a06aed49938ae24e5c69c5741a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.498755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.503817 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2033/4061] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 21:02:36 2022] GET /identity/v3/projects/41b446a06aed49938ae24e5c69c5741a => generated 365 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:36.568101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fffd05-1206-4f42-899c-355b7540ddce None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.568749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fffd05-1206-4f42-899c-355b7540ddce None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.569243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fffd05-1206-4f42-899c-355b7540ddce None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.571347 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2029/4062] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:02:36.582983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 21:02:36.611398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.628731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ac18a-33df-425d-bbd9-7631fded9499 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.638816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41b446a06aed49938ae24e5c69c5741a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.639536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.639888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.640289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] PATH_INFO: `/v3/projects/41b446a06aed49938ae24e5c69c5741a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.642100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorizing `identity:get_project(project_id=41b446a06aed49938ae24e5c69c5741a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.646270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.658715 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2034/4063] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 21:02:36 2022] GET /identity/v3/projects/41b446a06aed49938ae24e5c69c5741a => generated 365 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:36.690525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80b2ee-aa83-477a-a8b0-c4ffb5a6f11a None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.691030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80b2ee-aa83-477a-a8b0-c4ffb5a6f11a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.691490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80b2ee-aa83-477a-a8b0-c4ffb5a6f11a None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.693439 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2030/4064] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:02:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:02:36.701940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 21:02:36.722197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.734289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cd873-93e7-4edc-912d-7234c198bb85 tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.740390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41b446a06aed49938ae24e5c69c5741a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.741271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.742024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.742493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] PATH_INFO: `/v3/projects/41b446a06aed49938ae24e5c69c5741a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.744999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorizing `identity:get_project(project_id=41b446a06aed49938ae24e5c69c5741a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.747476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.763123 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2035/4065] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 21:02:36 2022] GET /identity/v3/projects/41b446a06aed49938ae24e5c69c5741a => generated 365 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:36.802983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1903edd-7dbe-43ba-91e9-ff4a0bbc90f8 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.803599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1903edd-7dbe-43ba-91e9-ff4a0bbc90f8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.804114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1903edd-7dbe-43ba-91e9-ff4a0bbc90f8 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.806994 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2031/4066] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:02:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:02:36.814517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 21:02:36.839060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:36.856384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a449f22-52fa-4d03-84cc-df18b657f49e tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:36.864033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd978aa-0235-4574-8613-2bf60fd1d0df tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41b446a06aed49938ae24e5c69c5741a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:36.865004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd978aa-0235-4574-8613-2bf60fd1d0df tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:36.865452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd978aa-0235-4574-8613-2bf60fd1d0df tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:36.866628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd978aa-0235-4574-8613-2bf60fd1d0df tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] PATH_INFO: `/v3/projects/41b446a06aed49938ae24e5c69c5741a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:36.868593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd978aa-0235-4574-8613-2bf60fd1d0df tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorizing `identity:get_project(project_id=41b446a06aed49938ae24e5c69c5741a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:36.871492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd978aa-0235-4574-8613-2bf60fd1d0df tempest-QuotasTestJSON-1042514520 tempest-QuotasTestJSON-1042514520-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:36.889811 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2036/4067] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 21:02:36 2022] GET /identity/v3/projects/41b446a06aed49938ae24e5c69c5741a => generated 365 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:36.926335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1903edd-7dbe-43ba-91e9-ff4a0bbc90f8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:37.109439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1903edd-7dbe-43ba-91e9-ff4a0bbc90f8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:37.114827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:37.115476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:37.115808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:37.116063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] PATH_INFO: `/v3/users/e3c9cfc67bb6497fa6ee77c1c7d98fbd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:37.120152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c9cfc67bb6497fa6ee77c1c7d98fbd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:37.123042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:37.249925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.265570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.269110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.283785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.285726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c9cfc67bb6497fa6ee77c1c7d98fbd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:37.331551 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2032/4068] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:36 2022] DELETE /identity/v3/users/e3c9cfc67bb6497fa6ee77c1c7d98fbd => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:37.336639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:37.336885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:37.338355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:37.369701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:37.373486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:37.504078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2037/4069] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:37.557024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:37.558930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:37.700372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50e9e7-c602-4c14-b60f-10b2f8fc19ca admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:37.706804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:37.707313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:37.707624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:37.708001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:37.724901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:37.728697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:37.746636 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2033/4070] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:38.528685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:38.530524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:38.692380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:38.695228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:38.723624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c1d7-315e-4140-962b-721e1ac26da3 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:38.730075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:38.735000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.735589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.736096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] PATH_INFO: `/v3/projects/41b446a06aed49938ae24e5c69c5741a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.738433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b446a06aed49938ae24e5c69c5741a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:38.741389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7252b-39ca-47ba-a14b-e99f03532468 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:38.741695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:38.760203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '933b9d039f1a4fd3b3ffc01099887f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1c3701208984be8ad70e79602180ab2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:38.760862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.761145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.761414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] PATH_INFO: `/v3/projects/10cbc7b689864aabb803baebd06759dd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:38.769858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorizing `identity:delete_project(project_id=10cbc7b689864aabb803baebd06759dd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:38.772974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:38.780445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:38.865693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b446a06aed49938ae24e5c69c5741a', 'request_id': 'req-59ac5753-4ab9-418b-8928-56dad50dfc60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.893028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b446a06aed49938ae24e5c69c5741a', 'request_id': 'req-59ac5753-4ab9-418b-8928-56dad50dfc60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.915792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b446a06aed49938ae24e5c69c5741a', 'request_id': 'req-59ac5753-4ab9-418b-8928-56dad50dfc60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.936838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:38.943221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10cbc7b689864aabb803baebd06759dd', 'request_id': 'req-c937e9a9-9496-4f6b-a513-8ba9f23360b8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.951683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] The token cache is being invalidate because project 41b446a06aed49938ae24e5c69c5741a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:38.951901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.954921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-59ac5753-4ab9-418b-8928-56dad50dfc60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.959004 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2034/4071] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:38 2022] DELETE /identity/v3/projects/41b446a06aed49938ae24e5c69c5741a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:38.959578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10cbc7b689864aabb803baebd06759dd', 'request_id': 'req-c937e9a9-9496-4f6b-a513-8ba9f23360b8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.969802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10cbc7b689864aabb803baebd06759dd', 'request_id': 'req-c937e9a9-9496-4f6b-a513-8ba9f23360b8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:38.972021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4ea3-d31e-4679-85ad-82847ef0f443 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:38.972465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4ea3-d31e-4679-85ad-82847ef0f443 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:38.972736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef4ea3-d31e-4679-85ad-82847ef0f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.024526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] The token cache is being invalidate because project 10cbc7b689864aabb803baebd06759dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:39.024822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:39.027259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:39.033768 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2038/4072] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:38 2022] DELETE /identity/v3/projects/10cbc7b689864aabb803baebd06759dd => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:39.034999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8aef4ea3-d31e-4679-85ad-82847ef0f443 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:39.036433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8aef4ea3-d31e-4679-85ad-82847ef0f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:39.071046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:39.074029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:39.182436 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2035/4073] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:39.191241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dae8-cb3e-4438-9613-391e220241cf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.191740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dae8-cb3e-4438-9613-391e220241cf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.192667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dae8-cb3e-4438-9613-391e220241cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.198865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937e9a9-9496-4f6b-a513-8ba9f23360b8 tempest-RoutersAdminTest-2058345133 tempest-RoutersAdminTest-2058345133-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:39.209472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:39.210978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.213127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.214668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.218469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:39.254121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-53b5dae8-cb3e-4438-9613-391e220241cf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:39.257303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-53b5dae8-cb3e-4438-9613-391e220241cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:39.287380 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2036/4074] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:39.292227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dae8-cb3e-4438-9613-391e220241cf None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:39.311272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dae8-cb3e-4438-9613-391e220241cf None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:39.329426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:39.330091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.330356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.330629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.331603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:39.331957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:39.333786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:39.360512 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2037/4075] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:02:39.365087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:39.403211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:39.403692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9f92c-b6be-4edf-85a5-79466a032a3c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:39.405341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:39.416270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:39.418581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.418581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.418581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.421629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:39.424114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:39.426532 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2039/4076] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:02:39.468194 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2038/4077] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:39 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:39.472872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:39.487118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2bd62-7085-45dc-aae7-9abb3a6fbf67 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:39.599369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:39.600202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.600416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.601380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.602133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:39.605182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:39.654179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2040/4078] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:39 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:39.658812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:39.673427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbe2db-43d0-4c76-a939-05303179c0af admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:39.686205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:39.687937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.688480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.689653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.691594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:39.694304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:39.743586 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2039/4079] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:39.748313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:39.761158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ad0de-0a7f-4a1e-ba28-b98b8176d72b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:39.781634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041c204-2552-4252-a45a-fea7bd51ba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:39.782752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c204-2552-4252-a45a-fea7bd51ba57 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.784194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c204-2552-4252-a45a-fea7bd51ba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.784194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c204-2552-4252-a45a-fea7bd51ba57 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.784194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041c204-2552-4252-a45a-fea7bd51ba57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:39.788111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041c204-2552-4252-a45a-fea7bd51ba57 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:39.817876 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2041/4080] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:02:39.822021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:39.835934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67958375-b6bf-476b-8c68-235184b7a268 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:39.843001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb04279-1bae-4062-a9ee-3bef5e04adca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:39.843868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04279-1bae-4062-a9ee-3bef5e04adca admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.844159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04279-1bae-4062-a9ee-3bef5e04adca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.844431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04279-1bae-4062-a9ee-3bef5e04adca admin admin] PATH_INFO: `/v3/projects/ff1d75a125014e7ea9fe4b62cbbdfa2a/users/29fdd87326be407b8840931f9f7a8bde/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.867598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb04279-1bae-4062-a9ee-3bef5e04adca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1d75a125014e7ea9fe4b62cbbdfa2a, user_id=29fdd87326be407b8840931f9f7a8bde, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:39.870703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb04279-1bae-4062-a9ee-3bef5e04adca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:39.921613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:39.921993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:39.922377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:39.923359 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2040/4081] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:39 2022] PUT /identity/v3/projects/ff1d75a125014e7ea9fe4b62cbbdfa2a/users/29fdd87326be407b8840931f9f7a8bde/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:39.939439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:39.958325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] MFA Rules not processed for user `29fdd87326be407b8840931f9f7a8bde`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:39.960041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:40.092009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:40.092438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:40.092905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:40.169136 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2042/4082] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:40.179964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] MFA Rules not processed for user `96372af27d29482491d613b5a783a0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:40.181265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:40.183880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:40.186437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:40.214454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:40.401992 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2041/4083] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:40.405995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:40.408639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:40.429957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cac78-4b87-4c6d-ae7d-ef00794ca7f3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:40.462623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:40.463608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:40.464117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:40.465226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:40.480538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:40.482760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:40.513935 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2043/4084] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:40.586959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cb32e-d11c-426a-8d57-9e19585c59df None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:40.603013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:40.603820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:40.604284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:40.604491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:40.605143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:40.610468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:40.663181 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2042/4085] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:40 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:40.665720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:40.691293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b924f0f-0b83-47f2-be0e-8027ad0622b7 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:40.801772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:40.802642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:40.803029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:40.803629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:40.804454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:40.807209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:40.866408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2044/4086] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:40 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:40.869228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:40.890737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120daaa8-7ad1-4d2b-a7ff-49a38ae7eb25 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:40.906537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:40.907289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:40.907669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:40.908039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:40.908792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:40.910816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:40.939780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:40.939970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2043/4087] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:40.966212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d1a15-b908-47eb-82c7-7fb85674d420 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:40.972252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:40.973428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:40.974421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:40.975094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] PATH_INFO: `/v3/projects/57907b81d51d40488a5c5a6129a702ff/users/52973c98d6374eec88f4ebd5077c74bd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:40.988612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:41.019894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57907b81d51d40488a5c5a6129a702ff, user_id=52973c98d6374eec88f4ebd5077c74bd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:41.039609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:41.073015 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2045/4088] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:40 2022] PUT /identity/v3/projects/57907b81d51d40488a5c5a6129a702ff/users/52973c98d6374eec88f4ebd5077c74bd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:41.077682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:41.094254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab16d5-2faf-42d5-8438-5b1092e29f32 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:41.222520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8f9a1-206f-412b-93fe-33fc4be807a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:41.223452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8f9a1-206f-412b-93fe-33fc4be807a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:41.224410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8f9a1-206f-412b-93fe-33fc4be807a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:41.225027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8f9a1-206f-412b-93fe-33fc4be807a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:41.225926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8f9a1-206f-412b-93fe-33fc4be807a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:41.228520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8f9a1-206f-412b-93fe-33fc4be807a2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:41.253132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2044/4089] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:41.257782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:41.274211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10828730-70ed-41eb-9130-3aaa74ce07a5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:41.282311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3686ce-4cfe-496b-b1dd-67ce46f12bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:41.283190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3686ce-4cfe-496b-b1dd-67ce46f12bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:41.284369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3686ce-4cfe-496b-b1dd-67ce46f12bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:41.284810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3686ce-4cfe-496b-b1dd-67ce46f12bdf admin admin] PATH_INFO: `/v3/projects/57907b81d51d40488a5c5a6129a702ff/users/52973c98d6374eec88f4ebd5077c74bd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:41.290116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3686ce-4cfe-496b-b1dd-67ce46f12bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=57907b81d51d40488a5c5a6129a702ff, user_id=52973c98d6374eec88f4ebd5077c74bd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:41.294382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3686ce-4cfe-496b-b1dd-67ce46f12bdf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:41.332997 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2046/4090] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:41 2022] PUT /identity/v3/projects/57907b81d51d40488a5c5a6129a702ff/users/52973c98d6374eec88f4ebd5077c74bd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:43.629435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202797e-709c-4e5f-9556-b57126e6b8a8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:43.630332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202797e-709c-4e5f-9556-b57126e6b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:43.630936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202797e-709c-4e5f-9556-b57126e6b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:43.754105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4202797e-709c-4e5f-9556-b57126e6b8a8 None None] MFA Rules not processed for user `edef2887c6b741108da9b88745486339`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:43.757545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4202797e-709c-4e5f-9556-b57126e6b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:43.793531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:43.896494 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2045/4091] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:43.907649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16101-b79d-4a5a-90f7-43c3f2ce3c33 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:43.907649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16101-b79d-4a5a-90f7-43c3f2ce3c33 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:43.907649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16101-b79d-4a5a-90f7-43c3f2ce3c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:43.951754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-16e16101-b79d-4a5a-90f7-43c3f2ce3c33 None None] MFA Rules not processed for user `edef2887c6b741108da9b88745486339`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:43.954375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-16e16101-b79d-4a5a-90f7-43c3f2ce3c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:43.994206 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2047/4092] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:44.007570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202797e-709c-4e5f-9556-b57126e6b8a8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:44.145558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202797e-709c-4e5f-9556-b57126e6b8a8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:44.171527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:44.172179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:44.172460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:44.172736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:44.183489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:44.185142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:44.208340 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2046/4093] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:46.644812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e34f-9c53-4c62-941f-86c6423c6423 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:46.648171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e34f-9c53-4c62-941f-86c6423c6423 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:46.649761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e34f-9c53-4c62-941f-86c6423c6423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:46.676573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ce81e34f-9c53-4c62-941f-86c6423c6423 None None] MFA Rules not processed for user `6f50ca756d594703846f1dade2d38eac`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:46.678001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ce81e34f-9c53-4c62-941f-86c6423c6423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:46.808784 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2048/4094] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:50.018390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:50.195376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a0979-2dd9-4c13-a22e-5d43b9f26eb6 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:50.204942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:50.207767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:50.208290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:50.210241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] PATH_INFO: `/v3/users/29fdd87326be407b8840931f9f7a8bde` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:50.212186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29fdd87326be407b8840931f9f7a8bde)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:50.216718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:50.417580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:50.424110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29fdd87326be407b8840931f9f7a8bde'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:50.460598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29fdd87326be407b8840931f9f7a8bde'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:50.465917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29fdd87326be407b8840931f9f7a8bde'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:50.493817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29fdd87326be407b8840931f9f7a8bde'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:50.510914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29fdd87326be407b8840931f9f7a8bde'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:50.623567 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2047/4095] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:50 2022] DELETE /identity/v3/users/29fdd87326be407b8840931f9f7a8bde => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:50.628364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:50.629274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:50.629274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:50.704812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:50.709693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:50.933036 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2049/4096] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:51.056155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:51.058937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:51.206533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542626-81d8-4bdf-ab42-922529295e82 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:51.224256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac8429-6eec-4d8e-bce5-0a3858be0922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:51.224905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac8429-6eec-4d8e-bce5-0a3858be0922 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:51.226080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac8429-6eec-4d8e-bce5-0a3858be0922 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:51.226080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac8429-6eec-4d8e-bce5-0a3858be0922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:51.244278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac8429-6eec-4d8e-bce5-0a3858be0922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:51.258483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac8429-6eec-4d8e-bce5-0a3858be0922 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:51.300892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2048/4097] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:52.240330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:52.242265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:52.274923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466ee2a-fa18-40d4-b51a-d6efef3df572 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:52.293493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:52.297863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:52.300577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:52.301095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] PATH_INFO: `/v3/projects/ff1d75a125014e7ea9fe4b62cbbdfa2a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:52.304731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1d75a125014e7ea9fe4b62cbbdfa2a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:52.310878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:52.443026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1d75a125014e7ea9fe4b62cbbdfa2a', 'request_id': 'req-cbd1db55-0037-487e-985c-c96c9e07da52'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:52.474315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1d75a125014e7ea9fe4b62cbbdfa2a', 'request_id': 'req-cbd1db55-0037-487e-985c-c96c9e07da52'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:52.502855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1d75a125014e7ea9fe4b62cbbdfa2a', 'request_id': 'req-cbd1db55-0037-487e-985c-c96c9e07da52'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:52.515473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:52.542655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] The token cache is being invalidate because project ff1d75a125014e7ea9fe4b62cbbdfa2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:02:52.543034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:52.546562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cbd1db55-0037-487e-985c-c96c9e07da52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:52.551789 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2050/4098] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:02:52 2022] DELETE /identity/v3/projects/ff1d75a125014e7ea9fe4b62cbbdfa2a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:52.553709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5bc8-705c-47e6-afe8-c9aaf3a45dbf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:52.555737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5bc8-705c-47e6-afe8-c9aaf3a45dbf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:52.555737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5bc8-705c-47e6-afe8-c9aaf3a45dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:52.597844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f45d5bc8-705c-47e6-afe8-c9aaf3a45dbf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:52.599696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f45d5bc8-705c-47e6-afe8-c9aaf3a45dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:52.876659 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2049/4099] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:52.880205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43654c7-e69b-4bbe-9e39-ea7cce29d39d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:52.880815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43654c7-e69b-4bbe-9e39-ea7cce29d39d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:52.880815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43654c7-e69b-4bbe-9e39-ea7cce29d39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:52.942834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f43654c7-e69b-4bbe-9e39-ea7cce29d39d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:52.945381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f43654c7-e69b-4bbe-9e39-ea7cce29d39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:52.975740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5bc8-705c-47e6-afe8-c9aaf3a45dbf None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:52.976362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2051/4100] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:52.992674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5bc8-705c-47e6-afe8-c9aaf3a45dbf None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:53.012823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:53.017063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:53.017434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:53.017610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:53.018317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:53.032862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:53.038201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:53.092173 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2050/4101] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:53.097448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43654c7-e69b-4bbe-9e39-ea7cce29d39d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:53.141182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43654c7-e69b-4bbe-9e39-ea7cce29d39d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:53.171989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:53.172633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:53.172913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:53.174458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:53.175408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:53.178552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:53.271930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:53.275107 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2052/4102] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:53 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:53.288713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6efa4-1799-4bb5-b883-b5cc15a3029f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:53.427505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:53.462462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:53.463194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:53.463570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:53.464705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:53.465653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:53.469097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:53.533491 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2051/4103] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:53 2022] POST /identity/v3/users => generated 284 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:53.538630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:53.563305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfd592-efba-4216-b6cb-c8b776b0ef11 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:53.598521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:53.602727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:53.604265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:53.605518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:53.607285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:53.613154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:53.654704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:53.657653 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2053/4104] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:53.702883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35523b6-a650-45ba-ade7-4539195a1dac admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:53.725885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884b6a6-6ea9-4730-bd98-2b780080a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:53.726596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884b6a6-6ea9-4730-bd98-2b780080a144 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:53.726993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884b6a6-6ea9-4730-bd98-2b780080a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:53.727363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884b6a6-6ea9-4730-bd98-2b780080a144 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:53.728215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884b6a6-6ea9-4730-bd98-2b780080a144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:53.731637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884b6a6-6ea9-4730-bd98-2b780080a144 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:53.767693 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2052/4105] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:53.771531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:53.786740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191dac4-8670-42b0-9805-caf16509a8b6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:53.799195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:53.799895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:53.800184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:53.800450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] PATH_INFO: `/v3/projects/a11948dbe3a04861bc6cb57e97c8ed58/users/c721faf607014105ac3a6ce2834f8311/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:53.868788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11948dbe3a04861bc6cb57e97c8ed58, user_id=c721faf607014105ac3a6ce2834f8311, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:53.877529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:53.907284 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2054/4106] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:53 2022] PUT /identity/v3/projects/a11948dbe3a04861bc6cb57e97c8ed58/users/c721faf607014105ac3a6ce2834f8311/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:53.921171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014938-1eba-441a-ae93-38d54b7f68b0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:53.921463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014938-1eba-441a-ae93-38d54b7f68b0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:53.921742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014938-1eba-441a-ae93-38d54b7f68b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:54.007101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f5014938-1eba-441a-ae93-38d54b7f68b0 None None] MFA Rules not processed for user `c721faf607014105ac3a6ce2834f8311`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:54.008316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f5014938-1eba-441a-ae93-38d54b7f68b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:54.209216 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2053/4107] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:54.213917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:54.376039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce39c81-e077-4513-8a36-af5159d66a3a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:54.391515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:54.392461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:54.392923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:54.393397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:54.394245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:54.397238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:54.577704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2055/4108] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:02:54 2022] POST /identity/v3/projects => generated 365 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:54.581706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014938-1eba-441a-ae93-38d54b7f68b0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:54.622319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014938-1eba-441a-ae93-38d54b7f68b0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:54.887080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:54.897201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:54.897656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:54.898024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:54.902617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:54.920677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:54.996745 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2054/4109] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:02:54 2022] POST /identity/v3/users => generated 289 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:55.001475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.022983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f2a6a-0b65-4934-8e58-04ff57ce000e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:55.031929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:55.034119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.035591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.036176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.038713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:55.043340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:55.076148 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2056/4110] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:55.080772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.096460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ff9c9-1717-4de6-9125-46a236f197cb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:55.111120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:55.112196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.115635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.116146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] PATH_INFO: `/v3/projects/46582b8d0aaf4b28a63c17c4fa60518d/users/9cb834b4aa074d59902feb7cbe22588d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.172365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46582b8d0aaf4b28a63c17c4fa60518d, user_id=9cb834b4aa074d59902feb7cbe22588d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:55.185927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:55.212838 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2055/4111] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:55 2022] PUT /identity/v3/projects/46582b8d0aaf4b28a63c17c4fa60518d/users/9cb834b4aa074d59902feb7cbe22588d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:55.217943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.233239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f09c-cd51-4b2a-a74c-691323c37628 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:55.330757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8b0d-22db-4e6f-841c-6ad47972e1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:55.331501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8b0d-22db-4e6f-841c-6ad47972e1ae admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.331856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8b0d-22db-4e6f-841c-6ad47972e1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.332457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8b0d-22db-4e6f-841c-6ad47972e1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.333341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8b0d-22db-4e6f-841c-6ad47972e1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:55.335648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8b0d-22db-4e6f-841c-6ad47972e1ae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:55.355417 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2057/4112] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:02:55.356507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.371028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3960d0-715a-4967-90d1-65273bb78954 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:55.379778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:55.380882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.381552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.382060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] PATH_INFO: `/v3/projects/46582b8d0aaf4b28a63c17c4fa60518d/users/9cb834b4aa074d59902feb7cbe22588d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.386888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46582b8d0aaf4b28a63c17c4fa60518d, user_id=9cb834b4aa074d59902feb7cbe22588d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:55.394188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:55.422647 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2056/4113] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:02:55 2022] PUT /identity/v3/projects/46582b8d0aaf4b28a63c17c4fa60518d/users/9cb834b4aa074d59902feb7cbe22588d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:55.428093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1d87-f506-4fdb-b6ff-b51a6fa7ed83 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.428617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1d87-f506-4fdb-b6ff-b51a6fa7ed83 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.429813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1d87-f506-4fdb-b6ff-b51a6fa7ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.441373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:55.477336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3efc1d87-f506-4fdb-b6ff-b51a6fa7ed83 None None] MFA Rules not processed for user `9cb834b4aa074d59902feb7cbe22588d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:55.479572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3efc1d87-f506-4fdb-b6ff-b51a6fa7ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:55.652590 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2058/4114] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:55.658860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.678109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01c8c6-9c0f-4ec7-8534-58aef775eda6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:55.686532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:55.687625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.688654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.689347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.690398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:55.692054 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:02:55.692054 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:02:55.692054 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:02:55.693599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:55.727236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:55.750424 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2057/4115] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:02:55 2022] POST /identity/v3/regions => generated 224 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:02:55.750669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc1d87-f506-4fdb-b6ff-b51a6fa7ed83 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.767257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc1d87-f506-4fdb-b6ff-b51a6fa7ed83 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:55.810954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:55.814017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.814717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.814717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.816051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:55.819736 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:02:55.819736 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:02:55.819736 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:02:55.820974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:55.863009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.877394 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2059/4116] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:02:55 2022] POST /identity/v3/regions => generated 223 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:55.883216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a013e-328f-4970-b7f6-bb39e6e196d7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:55.916321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:55.918756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:55.919212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:55.920294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/4d1b2a55-76a8-4ae6-a563-fcc0cd170bd4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:55.921987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:create_region(region_id=4d1b2a55-76a8-4ae6-a563-fcc0cd170bd4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:55.923879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:55.948785 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2058/4117] 10.222.0.33 () {60 vars in 1259 bytes} [Sat Sep 10 21:02:55 2022] PUT /identity/v3/regions/4d1b2a55-76a8-4ae6-a563-fcc0cd170bd4 => generated 231 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:55.955998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:55.984946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198895f-29bb-474a-a22f-278ff196f03d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.007504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.008073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.008453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.008923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/4d1b2a55-76a8-4ae6-a563-fcc0cd170bd4` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.010017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:delete_region(region_id=4d1b2a55-76a8-4ae6-a563-fcc0cd170bd4)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.010985 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:02:56.010985 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:02:56.010985 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:02:56.012286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.048716 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2060/4118] 10.222.0.33 () {58 vars in 1242 bytes} [Sat Sep 10 21:02:55 2022] DELETE /identity/v3/regions/4d1b2a55-76a8-4ae6-a563-fcc0cd170bd4 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:56.055252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.074778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17498a3-07e2-4cbc-9be9-c0257f29694e tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.117347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.121721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.121721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.121968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.122990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.126402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.169146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2059/4119] 10.222.0.33 () {60 vars in 1150 bytes} [Sat Sep 10 21:02:56 2022] POST /identity/v3/regions => generated 253 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:56.172158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.195650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90344c-7f9b-42c3-9e65-288000abe4fb tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.242110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.242943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.243408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.243844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/9469181568cf4bca9205fc933797829b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.244655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:update_region(region_id=9469181568cf4bca9205fc933797829b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.245844 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:02:56.245844 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:02:56.245844 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:02:56.247136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.278323 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2061/4120] 10.222.0.33 () {60 vars in 1250 bytes} [Sat Sep 10 21:02:56 2022] PATCH /identity/v3/regions/9469181568cf4bca9205fc933797829b => generated 253 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:56.283576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.296488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d15ee9-f9e6-4a6e-8047-67b64b5e1ac7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.323339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.324053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.324491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.324960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/9469181568cf4bca9205fc933797829b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.325880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:get_region(region_id=9469181568cf4bca9205fc933797829b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.327855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.339935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2060/4121] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:02:56 2022] GET /identity/v3/regions/9469181568cf4bca9205fc933797829b => generated 253 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:56.344669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.364756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae957e9-e423-4c27-a4c7-d0b53b2ae545 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.379003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.380006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.380512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.381014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/9469181568cf4bca9205fc933797829b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.381812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:delete_region(region_id=9469181568cf4bca9205fc933797829b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.383696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.422587 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2062/4122] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:02:56 2022] DELETE /identity/v3/regions/9469181568cf4bca9205fc933797829b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:56.426517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.439364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebc6a4-8a9f-425f-8ff2-1362b3c3a2a7 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.463434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.464726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.464726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.465125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.466285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.468043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.477076 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2061/4123] 10.222.0.33 () {58 vars in 1128 bytes} [Sat Sep 10 21:02:56 2022] GET /identity/v3/regions => generated 669 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:56.482153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.498933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94b5ba-3e26-4cd2-b567-da6ea959261d tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.507104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.507872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.509329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.511844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/9469181568cf4bca9205fc933797829b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.514593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:delete_region(region_id=9469181568cf4bca9205fc933797829b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.522692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Could not find region: 9469181568cf4bca9205fc933797829b.: keystone.exception.RegionNotFound: Could not find region: 9469181568cf4bca9205fc933797829b. Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 9469181568cf4bca9205fc933797829b. Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:02:56.552088 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 9469181568cf4bca9205fc933797829b. Sep 10 21:02:56.554819 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:02:56.556741 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2063/4124] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:02:56 2022] DELETE /identity/v3/regions/9469181568cf4bca9205fc933797829b => generated 112 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:02:56.561643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.576980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466841f-6916-41e2-8abd-ca523e4f8098 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.584036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.584792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.585783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.586152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.586878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.588163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.597166 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2062/4125] 10.222.0.33 () {58 vars in 1128 bytes} [Sat Sep 10 21:02:56 2022] GET /identity/v3/regions => generated 669 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:56.604869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.658505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738e0eb-99d0-4fe1-b35d-d14a29002a5a tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.670106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.671443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.672110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.698907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.702951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.709362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.770823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.771355 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2064/4126] 10.222.0.33 () {60 vars in 1150 bytes} [Sat Sep 10 21:02:56 2022] POST /identity/v3/regions => generated 254 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:02:56.784764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2197-badb-4508-badf-848b21cc1e0f tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.836647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f95e3-afd9-4d25-a8cd-99e74d29b6db tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.837283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f95e3-afd9-4d25-a8cd-99e74d29b6db tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.837579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f95e3-afd9-4d25-a8cd-99e74d29b6db tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.838062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f95e3-afd9-4d25-a8cd-99e74d29b6db tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.839363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f95e3-afd9-4d25-a8cd-99e74d29b6db tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Adding query filter params (parent_region_id=3fdabef47e34469d8cedf568db93b257) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:02:56.839363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f95e3-afd9-4d25-a8cd-99e74d29b6db tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.840296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f95e3-afd9-4d25-a8cd-99e74d29b6db tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.860479 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2063/4127] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:02:56 2022] GET /identity/v3/regions?parent_region_id=3fdabef47e34469d8cedf568db93b257 => generated 400 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:02:56.861389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:56.881715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2aaaf-9ad0-4360-82e5-0c8d682a1d9c tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:56.898378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547788c4-a325-4321-99d6-b2030f9fc4b6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.899028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547788c4-a325-4321-99d6-b2030f9fc4b6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.899527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547788c4-a325-4321-99d6-b2030f9fc4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.912159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:56.917378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:56.920052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:56.921531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/1eee0f3217a84c4fad590295dac0163c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:56.923209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:delete_region(region_id=1eee0f3217a84c4fad590295dac0163c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:56.926128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:56.934930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=2,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:56.956100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-547788c4-a325-4321-99d6-b2030f9fc4b6 None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:56.958402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-547788c4-a325-4321-99d6-b2030f9fc4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:56.993991 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2065/4128] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:02:56 2022] DELETE /identity/v3/regions/1eee0f3217a84c4fad590295dac0163c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:56.995294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:57.088166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8aba16-501f-431b-99ff-dd7699d69856 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:57.103691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:57.106221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:57.106840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:57.110115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/9c009010f54b461d92c76cb61de47b81` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:57.112154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:delete_region(region_id=9c009010f54b461d92c76cb61de47b81)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:57.115166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:57.133339 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2064/4129] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:57.173370 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2066/4130] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:02:56 2022] DELETE /identity/v3/regions/9c009010f54b461d92c76cb61de47b81 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:57.177162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547788c4-a325-4321-99d6-b2030f9fc4b6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:57.205132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547788c4-a325-4321-99d6-b2030f9fc4b6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:57.268205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2cd45-0f06-4c6f-bc85-3efe0ab71d23 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb834b4aa074d59902feb7cbe22588d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46582b8d0aaf4b28a63c17c4fa60518d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:57.268817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2cd45-0f06-4c6f-bc85-3efe0ab71d23 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:57.269094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2cd45-0f06-4c6f-bc85-3efe0ab71d23 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:57.269375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2cd45-0f06-4c6f-bc85-3efe0ab71d23 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] PATH_INFO: `/v3/regions/3fdabef47e34469d8cedf568db93b257` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:57.269996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2cd45-0f06-4c6f-bc85-3efe0ab71d23 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorizing `identity:delete_region(region_id=3fdabef47e34469d8cedf568db93b257)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:57.271025 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-40a2cd45-0f06-4c6f-bc85-3efe0ab71d23 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:02:57.271025 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:02:57.271025 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:02:57.271950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2cd45-0f06-4c6f-bc85-3efe0ab71d23 tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:57.338171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2065/4131] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:02:57 2022] DELETE /identity/v3/regions/3fdabef47e34469d8cedf568db93b257 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:02:57.339208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:57.481266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7afce-3c0e-4328-9652-b66663665eec tempest-RegionsTestJSON-818196276 tempest-RegionsTestJSON-818196276-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:57.517367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:57.520965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:57.521544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:57.522222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] PATH_INFO: `/v3/users/c721faf607014105ac3a6ce2834f8311` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:57.523851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c721faf607014105ac3a6ce2834f8311)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:57.526745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:57.664115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:57.664655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:57.665252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:57.675605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:57.680026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c721faf607014105ac3a6ce2834f8311'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:57.732928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c721faf607014105ac3a6ce2834f8311'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:57.755481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] MFA Rules not processed for user `52973c98d6374eec88f4ebd5077c74bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:57.769773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c721faf607014105ac3a6ce2834f8311'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:57.777186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:57.804062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c721faf607014105ac3a6ce2834f8311'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:57.804933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b3258c9c-ea44-485b-a95c-f1afe6ad5071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c721faf607014105ac3a6ce2834f8311'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:02:57.827248 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2067/4132] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:02:57 2022] DELETE /identity/v3/users/c721faf607014105ac3a6ce2834f8311 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:02:57.829106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191d02-ea05-476f-a517-0910735db257 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:57.829577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191d02-ea05-476f-a517-0910735db257 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:57.831204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191d02-ea05-476f-a517-0910735db257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:57.892209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-44191d02-ea05-476f-a517-0910735db257 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:02:57.895634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-44191d02-ea05-476f-a517-0910735db257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:57.957182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:58.040312 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2066/4133] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:02:58.089470 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2068/4134] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:02:58.095330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:58.104109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:58.126699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191d02-ea05-476f-a517-0910735db257 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:02:58.133584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-44191d02-ea05-476f-a517-0910735db257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:02:58.389060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191d02-ea05-476f-a517-0910735db257 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:58.390385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802d681-d293-47b1-8960-9323413c9f01 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:02:58.396021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:58.396656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:58.396936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:58.397380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:58.420766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:58.423409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:58.440759 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2069/4135] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:02:58.495025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:02:58.495738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:02:58.496039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:02:58.496370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:02:58.573509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:02:58.599764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:02:58.601367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:02:58.617299 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2067/4136] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:02:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:01.254415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:01.258237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:01.431958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977101-47d5-41d4-8d25-46f309c4e63a service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:01.459125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:01.462824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:01.465014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:01.466918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] PATH_INFO: `/v3/projects/a11948dbe3a04861bc6cb57e97c8ed58` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:01.471658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11948dbe3a04861bc6cb57e97c8ed58)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:01.476402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:01.803608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11948dbe3a04861bc6cb57e97c8ed58', 'request_id': 'req-235ad8ca-1578-4c33-b4c0-82c03b6377c6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:01.834460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11948dbe3a04861bc6cb57e97c8ed58', 'request_id': 'req-235ad8ca-1578-4c33-b4c0-82c03b6377c6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:01.877024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11948dbe3a04861bc6cb57e97c8ed58', 'request_id': 'req-235ad8ca-1578-4c33-b4c0-82c03b6377c6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:01.946708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] The token cache is being invalidate because project a11948dbe3a04861bc6cb57e97c8ed58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:01.959432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:01.964243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:01.971291 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2070/4137] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:01 2022] DELETE /identity/v3/projects/a11948dbe3a04861bc6cb57e97c8ed58 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:01.972074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:02.173459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a94e1e-452e-4e5a-8f5c-cf2669acfa9f service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:02.548965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:02.555885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:02.556253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:02.559174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] PATH_INFO: `/v3/users/9cb834b4aa074d59902feb7cbe22588d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:02.570323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb834b4aa074d59902feb7cbe22588d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:02.594362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:02.797910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb834b4aa074d59902feb7cbe22588d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:02.819765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb834b4aa074d59902feb7cbe22588d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:02.821247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb834b4aa074d59902feb7cbe22588d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:02.846689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb834b4aa074d59902feb7cbe22588d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:02.848383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7269616e-4d68-44b9-bad1-e91c5b599421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb834b4aa074d59902feb7cbe22588d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:02.958922 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2068/4138] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:01 2022] DELETE /identity/v3/users/9cb834b4aa074d59902feb7cbe22588d => generated 0 bytes in 984 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:05.543180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:05.545598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:05.576450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad8ca-1578-4c33-b4c0-82c03b6377c6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:05.666160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:05.711384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:05.712124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:05.712423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:05.712709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] PATH_INFO: `/v3/projects/46582b8d0aaf4b28a63c17c4fa60518d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:05.714551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] RBAC: Authorizing `identity:delete_project(project_id=46582b8d0aaf4b28a63c17c4fa60518d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:05.717030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:05.829444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46582b8d0aaf4b28a63c17c4fa60518d', 'request_id': 'req-15c5fff6-525a-491e-927f-1126658d744b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:05.838286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46582b8d0aaf4b28a63c17c4fa60518d', 'request_id': 'req-15c5fff6-525a-491e-927f-1126658d744b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:05.847441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46582b8d0aaf4b28a63c17c4fa60518d', 'request_id': 'req-15c5fff6-525a-491e-927f-1126658d744b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:05.868940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] The token cache is being invalidate because project 46582b8d0aaf4b28a63c17c4fa60518d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:05.869273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:05.869909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15c5fff6-525a-491e-927f-1126658d744b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:05.873075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2071/4139] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:05 2022] DELETE /identity/v3/projects/46582b8d0aaf4b28a63c17c4fa60518d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:05.877679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5baae-9f5e-4570-9824-9f9c139dea6b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:05.878269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5baae-9f5e-4570-9824-9f9c139dea6b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:05.878269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5baae-9f5e-4570-9824-9f9c139dea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:05.905447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-33b5baae-9f5e-4570-9824-9f9c139dea6b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:05.907783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-33b5baae-9f5e-4570-9824-9f9c139dea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:06.038596 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2069/4140] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:06.039752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1eb8c-950f-4ee5-bd33-00152b6f2320 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.040185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1eb8c-950f-4ee5-bd33-00152b6f2320 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.040470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1eb8c-950f-4ee5-bd33-00152b6f2320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.079792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e0b1eb8c-950f-4ee5-bd33-00152b6f2320 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:06.083863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e0b1eb8c-950f-4ee5-bd33-00152b6f2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:06.101874 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2072/4141] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:06.102577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5baae-9f5e-4570-9824-9f9c139dea6b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:06.112299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:06.120414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5baae-9f5e-4570-9824-9f9c139dea6b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:06.130712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:06.137459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.138739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.139255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.140465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:06.141423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:06.145009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:06.181040 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2070/4142] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:06.186738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1eb8c-950f-4ee5-bd33-00152b6f2320 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:06.205184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1eb8c-950f-4ee5-bd33-00152b6f2320 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:06.213245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:06.213982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.214198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.214449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.215215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:06.217685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:06.274357 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2073/4143] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:06 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:06.277505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:06.290711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c862e-5cf9-4697-aec2-731f3f4455f3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:06.379449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:06.380518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.381043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.381549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.382579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:06.384824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:06.426905 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2071/4144] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:06 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:06.431538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:06.447646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07af1d-685d-4c44-9d81-4f044f45557a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:06.456396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:06.457049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.457379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.457653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.458519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:06.460167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:06.499995 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2074/4145] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:06.504664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:06.538143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6794e9c-318f-4183-81c6-9e92057ff3a4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:06.559975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1748d61-36a7-4935-8a77-2c1b588bb499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:06.561972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1748d61-36a7-4935-8a77-2c1b588bb499 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.561972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1748d61-36a7-4935-8a77-2c1b588bb499 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.564205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1748d61-36a7-4935-8a77-2c1b588bb499 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.565130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1748d61-36a7-4935-8a77-2c1b588bb499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:06.571726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1748d61-36a7-4935-8a77-2c1b588bb499 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:06.596765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2072/4146] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:06.600987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:06.614631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686c715-6c49-4437-99c3-0d5a3f168919 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:06.626540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:06.627364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.627745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.628282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] PATH_INFO: `/v3/projects/4807914d1c604dd7bf9e85a3c1a68234/users/c85ff3d135eb4e55b3f64a5c3e5326ea/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.711536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4807914d1c604dd7bf9e85a3c1a68234, user_id=c85ff3d135eb4e55b3f64a5c3e5326ea, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:06.719991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:06.765027 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2075/4147] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:06 2022] PUT /identity/v3/projects/4807914d1c604dd7bf9e85a3c1a68234/users/c85ff3d135eb4e55b3f64a5c3e5326ea/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:06.775636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786712-7763-43a6-8d14-6e380e85be34 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:06.776555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786712-7763-43a6-8d14-6e380e85be34 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:06.777064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786712-7763-43a6-8d14-6e380e85be34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:06.846832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3a786712-7763-43a6-8d14-6e380e85be34 None None] MFA Rules not processed for user `c85ff3d135eb4e55b3f64a5c3e5326ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:06.848315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3a786712-7763-43a6-8d14-6e380e85be34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:07.145188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2073/4148] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:07.147474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:07.166217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7405d-ba3e-4fb9-8316-a45daf562151 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:07.187426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ceeb1-d945-404c-8245-e0b4d810a62e tempest-IdentityCatalogTest-1280469266 tempest-IdentityCatalogTest-1280469266-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c85ff3d135eb4e55b3f64a5c3e5326ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4807914d1c604dd7bf9e85a3c1a68234', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:07.188274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ceeb1-d945-404c-8245-e0b4d810a62e tempest-IdentityCatalogTest-1280469266 tempest-IdentityCatalogTest-1280469266-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:07.189431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ceeb1-d945-404c-8245-e0b4d810a62e tempest-IdentityCatalogTest-1280469266 tempest-IdentityCatalogTest-1280469266-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:07.190700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ceeb1-d945-404c-8245-e0b4d810a62e tempest-IdentityCatalogTest-1280469266 tempest-IdentityCatalogTest-1280469266-project] PATH_INFO: `/v3/auth/catalog` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:07.193161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ceeb1-d945-404c-8245-e0b4d810a62e tempest-IdentityCatalogTest-1280469266 tempest-IdentityCatalogTest-1280469266-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:07.198557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ceeb1-d945-404c-8245-e0b4d810a62e tempest-IdentityCatalogTest-1280469266 tempest-IdentityCatalogTest-1280469266-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:07.207752 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2076/4149] 10.222.0.33 () {58 vars in 1143 bytes} [Sat Sep 10 21:03:07 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:07.209245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786712-7763-43a6-8d14-6e380e85be34 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:07.404999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786712-7763-43a6-8d14-6e380e85be34 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:07.417921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:07.421311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:07.421608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:07.422407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] PATH_INFO: `/v3/users/c85ff3d135eb4e55b3f64a5c3e5326ea` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:07.429098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85ff3d135eb4e55b3f64a5c3e5326ea)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:07.431501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:07.436281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:07.603285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85ff3d135eb4e55b3f64a5c3e5326ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:07.618705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85ff3d135eb4e55b3f64a5c3e5326ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:07.622615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85ff3d135eb4e55b3f64a5c3e5326ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:07.656505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85ff3d135eb4e55b3f64a5c3e5326ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:07.658098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85ff3d135eb4e55b3f64a5c3e5326ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:07.695026 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2074/4150] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:07 2022] DELETE /identity/v3/users/c85ff3d135eb4e55b3f64a5c3e5326ea => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:07.699453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:07.704512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:07.705088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:07.758981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:07.761652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:07.832293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:07.867743 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2077/4151] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:07.922542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:07.924571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:08.068420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf5d56-421d-43b2-b03c-9cc6089e26f8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:08.077913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:08.078543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:08.078820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:08.079099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:08.095858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:08.097214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:08.130678 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2075/4152] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:09.648332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:09.670203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:09.692362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd892b-8299-4ac7-a883-9fd01c09657d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:09.705545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:09.706257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:09.706554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:09.708479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:09.714761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:09.932476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:09.934142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:10.021972 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2078/4153] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:11.538166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:11.541081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:11.583927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a366f-d12d-4f9b-8179-9da4f584d5d9 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:11.601051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:11.602545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:11.603100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:11.603637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] PATH_INFO: `/v3/projects/4807914d1c604dd7bf9e85a3c1a68234` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:11.607128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4807914d1c604dd7bf9e85a3c1a68234)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:11.630938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:11.748468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:11.762233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae46cd6-54ef-41d3-a5c4-ad08987f6071 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:11.774559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c566f6f-dc1c-4b8e-b1e5-2f8c1de5afd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:11.776427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c566f6f-dc1c-4b8e-b1e5-2f8c1de5afd7 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:11.779359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c566f6f-dc1c-4b8e-b1e5-2f8c1de5afd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:11.780687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c566f6f-dc1c-4b8e-b1e5-2f8c1de5afd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:11.784965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c566f6f-dc1c-4b8e-b1e5-2f8c1de5afd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:11.827081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c566f6f-dc1c-4b8e-b1e5-2f8c1de5afd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:11.829469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c566f6f-dc1c-4b8e-b1e5-2f8c1de5afd7 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:11.834821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4807914d1c604dd7bf9e85a3c1a68234', 'request_id': 'req-e5a83847-faae-4de3-96e7-499fcb24e82e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:11.861130 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2079/4154] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:11.874793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4807914d1c604dd7bf9e85a3c1a68234', 'request_id': 'req-e5a83847-faae-4de3-96e7-499fcb24e82e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:11.892116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4807914d1c604dd7bf9e85a3c1a68234', 'request_id': 'req-e5a83847-faae-4de3-96e7-499fcb24e82e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:11.926626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] The token cache is being invalidate because project 4807914d1c604dd7bf9e85a3c1a68234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:11.927285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:11.928254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e5a83847-faae-4de3-96e7-499fcb24e82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:11.937787 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2076/4155] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:11 2022] DELETE /identity/v3/projects/4807914d1c604dd7bf9e85a3c1a68234 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:11.939507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831f0c6-f777-4762-899c-4f7e59dc0b53 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:11.940104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831f0c6-f777-4762-899c-4f7e59dc0b53 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:11.940705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831f0c6-f777-4762-899c-4f7e59dc0b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:12.000407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d831f0c6-f777-4762-899c-4f7e59dc0b53 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:12.012536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d831f0c6-f777-4762-899c-4f7e59dc0b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:12.129897 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2080/4156] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:12.137865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55911726-665b-49e9-be65-e3f3f43cb75d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:12.139161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55911726-665b-49e9-be65-e3f3f43cb75d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:12.139562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55911726-665b-49e9-be65-e3f3f43cb75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:12.152006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:12.189861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-55911726-665b-49e9-be65-e3f3f43cb75d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:12.191587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55911726-665b-49e9-be65-e3f3f43cb75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:12.221966 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2077/4157] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:12.226024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831f0c6-f777-4762-899c-4f7e59dc0b53 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:12.240506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831f0c6-f777-4762-899c-4f7e59dc0b53 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:12.253910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:12.255301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:12.255951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:12.256446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:12.257355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:12.266601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:12.279781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:12.328704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55911726-665b-49e9-be65-e3f3f43cb75d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:12.329310 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2081/4158] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:12.345788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55911726-665b-49e9-be65-e3f3f43cb75d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:12.373671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:12.373671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:12.373671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:12.373671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:12.374395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:12.374395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:12.512175 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2078/4159] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:12 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:12.515323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:12.532536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ab5e2-4a94-4dff-9a7a-234b03591c0e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:12.826448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:12.848532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:12.850566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:12.850960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:12.851974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:12.853167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:12.858119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:12.916864 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2082/4160] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:12 2022] POST /identity/v3/users => generated 286 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:12.927112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:12.951689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e25ed-6522-4c35-bd9f-039366f02108 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:12.963916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:12.967072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:12.967538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:12.967820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:12.969118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:12.973696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:13.018904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2079/4161] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:13.022305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.043396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e88ef-44ef-452e-8e77-a31459bcd098 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.070660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe250f-eba0-48cc-b65e-e146d4743fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:13.071579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe250f-eba0-48cc-b65e-e146d4743fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.072022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe250f-eba0-48cc-b65e-e146d4743fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.072433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe250f-eba0-48cc-b65e-e146d4743fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.074025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe250f-eba0-48cc-b65e-e146d4743fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:13.081170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe250f-eba0-48cc-b65e-e146d4743fc4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:13.146118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.151371 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2083/4162] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:13.167400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c153d4-11dc-4b65-9b79-5c8040292d83 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.237421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:13.237421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.237421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.237421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] PATH_INFO: `/v3/projects/cfbe90aeef5e45c8849e5641d43c7f74/users/ef782b6983d54db6ad8c8d56464ad8f0/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.257654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfbe90aeef5e45c8849e5641d43c7f74, user_id=ef782b6983d54db6ad8c8d56464ad8f0, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:13.271277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:13.328933 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2080/4163] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:13 2022] PUT /identity/v3/projects/cfbe90aeef5e45c8849e5641d43c7f74/users/ef782b6983d54db6ad8c8d56464ad8f0/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:13.333053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82950b92-ac33-4ec9-bbe1-d21710cafd06 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.333453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82950b92-ac33-4ec9-bbe1-d21710cafd06 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.333898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82950b92-ac33-4ec9-bbe1-d21710cafd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.400042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-82950b92-ac33-4ec9-bbe1-d21710cafd06 None None] MFA Rules not processed for user `ef782b6983d54db6ad8c8d56464ad8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:13.413671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-82950b92-ac33-4ec9-bbe1-d21710cafd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:13.590920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.598854 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2084/4164] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:13.603392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221c7d1-c1e6-4faf-b0cd-131a7b7e7807 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.610223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:13.611309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.612188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.612680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.613565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:13.619152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:13.673700 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2081/4165] 10.222.0.33 () {60 vars in 1299 bytes} [Sat Sep 10 21:03:13 2022] POST /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2 => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:13.675160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82950b92-ac33-4ec9-bbe1-d21710cafd06 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.696210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82950b92-ac33-4ec9-bbe1-d21710cafd06 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.696210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:13.696210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.696548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.696548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/ad3a5364f5a6439482a9124bda7bd8da` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.696548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0, credential_id=ad3a5364f5a6439482a9124bda7bd8da)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:13.697426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:13.729743 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2085/4166] 10.222.0.33 () {58 vars in 1380 bytes} [Sat Sep 10 21:03:13 2022] DELETE /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/ad3a5364f5a6439482a9124bda7bd8da => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:13.735368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.748576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a20259-8670-4c07-b505-263d567ab454 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.761498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:13.762252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.763278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.763278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.763752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:13.765269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:13.791667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2082/4167] 10.222.0.33 () {60 vars in 1299 bytes} [Sat Sep 10 21:03:13 2022] POST /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2 => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:13.798123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.807975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37afbc-7ec2-470a-9407-f44ea575c95f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.813792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a09d6-603a-46b4-a9ce-7d3bd30a8e62 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:13.814566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a09d6-603a-46b4-a9ce-7d3bd30a8e62 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.814854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a09d6-603a-46b4-a9ce-7d3bd30a8e62 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.815399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a09d6-603a-46b4-a9ce-7d3bd30a8e62 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/b5a90abc20b440b58e27426819aeb712` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.818939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a09d6-603a-46b4-a9ce-7d3bd30a8e62 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0, credential_id=b5a90abc20b440b58e27426819aeb712)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:13.821144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a09d6-603a-46b4-a9ce-7d3bd30a8e62 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:13.867525 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2086/4168] 10.222.0.33 () {58 vars in 1380 bytes} [Sat Sep 10 21:03:13 2022] DELETE /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/b5a90abc20b440b58e27426819aeb712 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:13.871550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.893701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaad22-7a45-4aa9-a423-c2b8d2c5f19a tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.901383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06d2c-b17d-4b90-b20a-dcb8dfc33aa9 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:13.902469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06d2c-b17d-4b90-b20a-dcb8dfc33aa9 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.903059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06d2c-b17d-4b90-b20a-dcb8dfc33aa9 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.904066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06d2c-b17d-4b90-b20a-dcb8dfc33aa9 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/b5a90abc20b440b58e27426819aeb712` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.930476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae80bdc-1395-4341-a931-950f98c7645d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:13.930890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae80bdc-1395-4341-a931-950f98c7645d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:13.931236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae80bdc-1395-4341-a931-950f98c7645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-1bd06d2c-b17d-4b90-b20a-dcb8dfc33aa9 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Could not find credential: 4318a479fd1ec5bdf05dc680d65675543e463ca526ad086ef03ab1280fe2e88a.: keystone.exception.CredentialNotFound: Could not find credential: 4318a479fd1ec5bdf05dc680d65675543e463ca526ad086ef03ab1280fe2e88a. Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Sep 10 21:03:13.963827 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:03:13.966995 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: 4318a479fd1ec5bdf05dc680d65675543e463ca526ad086ef03ab1280fe2e88a. Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:03:13.968515 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2083/4169] 10.222.0.33 () {58 vars in 1377 bytes} [Sat Sep 10 21:03:13 2022] GET /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/b5a90abc20b440b58e27426819aeb712 => generated 148 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:03:13.971957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bae80bdc-1395-4341-a931-950f98c7645d None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:13.973327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06d2c-b17d-4b90-b20a-dcb8dfc33aa9 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:13.975794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bae80bdc-1395-4341-a931-950f98c7645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:13.992291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06d2c-b17d-4b90-b20a-dcb8dfc33aa9 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:13.999550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.001233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.001948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.002505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.003244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.005000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.025736 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2084/4170] 10.222.0.33 () {60 vars in 1299 bytes} [Sat Sep 10 21:03:13 2022] POST /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2 => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:14.033360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.049987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339d19d-a0f0-49f2-ab88-8f63da58380e tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.057008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.057679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.058127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.058414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.059040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.061313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.073869 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2087/4171] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:14.092474 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2085/4172] 10.222.0.33 () {60 vars in 1299 bytes} [Sat Sep 10 21:03:14 2022] POST /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2 => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:14.092734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae80bdc-1395-4341-a931-950f98c7645d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.104737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae80bdc-1395-4341-a931-950f98c7645d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.111052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.111886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.112241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.112589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.113252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=ef782b6983d54db6ad8c8d56464ad8f0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.115866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.133228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2088/4173] 10.222.0.33 () {58 vars in 1278 bytes} [Sat Sep 10 21:03:14 2022] GET /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2 => generated 865 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:14.134967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.146179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.148646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:14.150288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad457cb-918f-4c3d-a32e-0e537aa8f484 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.159083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.159607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.160203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.160333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/0745883e166049d0b2cf813e398890ca` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.162502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0, credential_id=0745883e166049d0b2cf813e398890ca)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.163495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:14.164033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.188025 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2086/4174] 10.222.0.33 () {58 vars in 1380 bytes} [Sat Sep 10 21:03:14 2022] DELETE /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/0745883e166049d0b2cf813e398890ca => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:14.192135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.216831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a230c-a00b-448b-b646-689ed3e2998f tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.224408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.225539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.225960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.226340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/e7c31633b835481b8277f8d9f82a1fc1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.229736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0, credential_id=e7c31633b835481b8277f8d9f82a1fc1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.231625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.253504 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2087/4175] 10.222.0.33 () {58 vars in 1380 bytes} [Sat Sep 10 21:03:14 2022] DELETE /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/e7c31633b835481b8277f8d9f82a1fc1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:14.259216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.271901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d6c0-f4f7-4e41-81b5-4e1f7009df10 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.272980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046c870-660a-4a6c-b2f2-baa6fe83f356 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.276706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.277655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.279025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.279384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.279702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.280135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.280456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.280810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.281155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.281996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.293084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.295745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.308066 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2088/4176] 10.222.0.33 () {60 vars in 1299 bytes} [Sat Sep 10 21:03:14 2022] POST /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2 => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:14.314411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.317108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2089/4177] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:14.347434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb826b20-560d-4e9b-98d2-096cfbf03b90 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.370220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.371223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.371510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.372357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/e2eb99ebd14044e5bfd58462b6cd2a61` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.379187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0, credential_id=e2eb99ebd14044e5bfd58462b6cd2a61)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.380562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.389683 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2089/4178] 10.222.0.33 () {58 vars in 1377 bytes} [Sat Sep 10 21:03:14 2022] GET /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/e2eb99ebd14044e5bfd58462b6cd2a61 => generated 367 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:14.390925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.401930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab91c1d-54ad-4344-9335-2fe6e7ab87a3 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.407881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc4adb-2aec-4299-afe9-678e850cb7f7 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef782b6983d54db6ad8c8d56464ad8f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfbe90aeef5e45c8849e5641d43c7f74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.409531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc4adb-2aec-4299-afe9-678e850cb7f7 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.410085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc4adb-2aec-4299-afe9-678e850cb7f7 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.410674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc4adb-2aec-4299-afe9-678e850cb7f7 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/e2eb99ebd14044e5bfd58462b6cd2a61` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.413472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc4adb-2aec-4299-afe9-678e850cb7f7 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=ef782b6983d54db6ad8c8d56464ad8f0, credential_id=e2eb99ebd14044e5bfd58462b6cd2a61)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.416031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc4adb-2aec-4299-afe9-678e850cb7f7 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.471648 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2090/4179] 10.222.0.33 () {58 vars in 1380 bytes} [Sat Sep 10 21:03:14 2022] DELETE /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0/credentials/OS-EC2/e2eb99ebd14044e5bfd58462b6cd2a61 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:14.476437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:14.554509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=7,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:14.575562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9f521-b7b9-4be3-a1d9-5863fb89f826 tempest-EC2CredentialsTest-228684903 tempest-EC2CredentialsTest-228684903-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:14.581212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:14.581873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.585072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.585406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] PATH_INFO: `/v3/users/ef782b6983d54db6ad8c8d56464ad8f0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.587766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef782b6983d54db6ad8c8d56464ad8f0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:14.589843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:14.670672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef782b6983d54db6ad8c8d56464ad8f0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:14.684770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef782b6983d54db6ad8c8d56464ad8f0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:14.688181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef782b6983d54db6ad8c8d56464ad8f0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:14.699404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef782b6983d54db6ad8c8d56464ad8f0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:14.700844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef782b6983d54db6ad8c8d56464ad8f0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:14.730318 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2090/4180] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:14 2022] DELETE /identity/v3/users/ef782b6983d54db6ad8c8d56464ad8f0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:14.732572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:14.732751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:14.733876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:14.814975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:14.817687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:15.006159 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2091/4181] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:15.097400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:15.100652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:15.326481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c0782-0f47-40cf-9943-2e0397aabbe5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:15.336378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1f319-3a9d-4deb-9b35-acb2c7d13a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:15.337519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f319-3a9d-4deb-9b35-acb2c7d13a07 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:15.338493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f319-3a9d-4deb-9b35-acb2c7d13a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:15.338996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f319-3a9d-4deb-9b35-acb2c7d13a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:15.386189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1f319-3a9d-4deb-9b35-acb2c7d13a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:15.393167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1f319-3a9d-4deb-9b35-acb2c7d13a07 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:15.455718 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2091/4182] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:17.545818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:17.547518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:17.585023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883201b-9de0-407d-909a-c4dba78fb4a8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:17.593689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:17.594903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:17.595377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:17.595841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] PATH_INFO: `/v3/projects/cfbe90aeef5e45c8849e5641d43c7f74` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:17.597215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfbe90aeef5e45c8849e5641d43c7f74)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:17.600040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:17.745166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfbe90aeef5e45c8849e5641d43c7f74', 'request_id': 'req-89b261d5-ba27-48df-9016-5971f0d56929'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:17.767082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfbe90aeef5e45c8849e5641d43c7f74', 'request_id': 'req-89b261d5-ba27-48df-9016-5971f0d56929'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:17.800097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfbe90aeef5e45c8849e5641d43c7f74', 'request_id': 'req-89b261d5-ba27-48df-9016-5971f0d56929'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:17.823539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:17.848523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] The token cache is being invalidate because project cfbe90aeef5e45c8849e5641d43c7f74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:17.849561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:17.851030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b261d5-ba27-48df-9016-5971f0d56929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:17.859625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2092/4183] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:17 2022] DELETE /identity/v3/projects/cfbe90aeef5e45c8849e5641d43c7f74 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:17.860600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6380-3a09-47d7-9636-ce473c759879 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:17.861058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6380-3a09-47d7-9636-ce473c759879 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:17.861164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c6380-3a09-47d7-9636-ce473c759879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:17.898740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ed8c6380-3a09-47d7-9636-ce473c759879 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:17.901807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed8c6380-3a09-47d7-9636-ce473c759879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:18.099685 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2092/4184] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:18.110283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4df19-21f5-45d6-ac8c-4fa61e86f27a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:18.110602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4df19-21f5-45d6-ac8c-4fa61e86f27a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:18.110774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4df19-21f5-45d6-ac8c-4fa61e86f27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:18.142308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a8d4df19-21f5-45d6-ac8c-4fa61e86f27a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:18.143571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a8d4df19-21f5-45d6-ac8c-4fa61e86f27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:18.159457 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2093/4185] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:18.162254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c6380-3a09-47d7-9636-ce473c759879 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:18.174994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c6380-3a09-47d7-9636-ce473c759879 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:18.180850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:18.181752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:18.182094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:18.182400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:18.183066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:18.183421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:18.185905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:18.197775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:18.208994 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2093/4186] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:18.210654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4df19-21f5-45d6-ac8c-4fa61e86f27a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:18.242010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4df19-21f5-45d6-ac8c-4fa61e86f27a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:18.250563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:18.251749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:18.252339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:18.252850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:18.254451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:18.259302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:18.362833 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2094/4187] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:18 2022] POST /identity/v3/projects => generated 395 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:03:18.365244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:18.415121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525042aa-e312-4d63-b932-f38af22ba06b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:18.607151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:18.609642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:18.610336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:18.610336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:18.613106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:18.615965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:18.670564 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2094/4188] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:18 2022] POST /identity/v3/users => generated 304 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:18.674944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:18.698375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba7e75-78f7-45d3-9a70-6792d4684d2b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:18.713115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:18.714416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:18.715055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:18.716105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:18.716877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:18.718683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:18.747527 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2095/4189] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:18.752371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:18.768760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd90b03-db25-44a8-83b0-8ba19eb353e5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:18.777520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:18.779682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:18.780187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:18.780491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] PATH_INFO: `/v3/projects/474b71e9dd7b45999ef7526aa9004a4e/users/7fd83fc3f0d44dae9e3bbdb9d34cb303/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:18.804597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474b71e9dd7b45999ef7526aa9004a4e, user_id=7fd83fc3f0d44dae9e3bbdb9d34cb303, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:18.808412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:18.855813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2095/4190] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:18 2022] PUT /identity/v3/projects/474b71e9dd7b45999ef7526aa9004a4e/users/7fd83fc3f0d44dae9e3bbdb9d34cb303/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:18.856439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:18.880481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47049465-69b9-470b-b507-1c9e066c1e9a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:19.028555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af78ce8-459f-4f70-944c-424cbbf3f0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:19.029265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78ce8-459f-4f70-944c-424cbbf3f0af admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:19.029597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78ce8-459f-4f70-944c-424cbbf3f0af admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:19.030704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78ce8-459f-4f70-944c-424cbbf3f0af admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:19.032006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af78ce8-459f-4f70-944c-424cbbf3f0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:19.033928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af78ce8-459f-4f70-944c-424cbbf3f0af admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:19.048969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2096/4191] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:19.052867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:19.071431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa35f31-5037-42a2-9c7d-071587c3f4d5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:19.082692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:19.084181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:19.084482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:19.084758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] PATH_INFO: `/v3/projects/474b71e9dd7b45999ef7526aa9004a4e/users/7fd83fc3f0d44dae9e3bbdb9d34cb303/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:19.089714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] RBAC: Authorizing `identity:create_grant(project_id=474b71e9dd7b45999ef7526aa9004a4e, user_id=7fd83fc3f0d44dae9e3bbdb9d34cb303, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:19.097992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:19.128322 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2096/4192] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:19 2022] PUT /identity/v3/projects/474b71e9dd7b45999ef7526aa9004a4e/users/7fd83fc3f0d44dae9e3bbdb9d34cb303/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:19.139883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e3eb-1bc6-4508-bafe-60ca9c382c93 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:19.140242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e3eb-1bc6-4508-bafe-60ca9c382c93 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:19.140443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98e3eb-1bc6-4508-bafe-60ca9c382c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:19.206417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ae98e3eb-1bc6-4508-bafe-60ca9c382c93 None None] MFA Rules not processed for user `7fd83fc3f0d44dae9e3bbdb9d34cb303`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:19.209452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ae98e3eb-1bc6-4508-bafe-60ca9c382c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:19.435346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:19.435869 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2097/4193] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:19.615929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb790b0-0622-4ee5-b5d1-81f56857345d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:19.628018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:19.631001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:19.631313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:19.631797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:19.632538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:19.635893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:19.698479 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2097/4194] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:19 2022] POST /identity/v3/projects => generated 397 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:19.704396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e3eb-1bc6-4508-bafe-60ca9c382c93 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:19.722061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98e3eb-1bc6-4508-bafe-60ca9c382c93 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:19.786640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:19.828342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:19.829337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:19.829805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:19.830356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:19.831251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:19.833660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:19.874445 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2098/4195] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:19 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:03:19.879065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:19.894011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca80de-cda2-48ef-9b82-38531790ed4e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:19.911429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:19.912797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:19.913705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:19.914376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:19.915357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:19.919687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:19.954094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:19.966188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2098/4196] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:19.980386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40927-713d-465e-81c0-2e3e919a006a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:19.986403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31736946-8230-4f2a-b72a-04acbf3b040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:19.987023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736946-8230-4f2a-b72a-04acbf3b040b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:19.987339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736946-8230-4f2a-b72a-04acbf3b040b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:19.987579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31736946-8230-4f2a-b72a-04acbf3b040b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:19.988242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31736946-8230-4f2a-b72a-04acbf3b040b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:19.989815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31736946-8230-4f2a-b72a-04acbf3b040b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:20.021050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:20.022181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2099/4197] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:20.034090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9740f-24cc-41f2-a353-9417d77c4beb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:20.045978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:20.046548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:20.047274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:20.047274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] PATH_INFO: `/v3/projects/83131dc4a20947329629a903a2251b55/users/b32a28d3e3dd4800a287a715472c9729/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:20.082010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83131dc4a20947329629a903a2251b55, user_id=b32a28d3e3dd4800a287a715472c9729, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:20.084627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:20.109393 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2099/4198] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:20 2022] PUT /identity/v3/projects/83131dc4a20947329629a903a2251b55/users/b32a28d3e3dd4800a287a715472c9729/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:20.120223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7945a50-0c74-4024-9468-294c6c30c56e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:20.120516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7945a50-0c74-4024-9468-294c6c30c56e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:20.120920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7945a50-0c74-4024-9468-294c6c30c56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:20.167191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a7945a50-0c74-4024-9468-294c6c30c56e None None] MFA Rules not processed for user `b32a28d3e3dd4800a287a715472c9729`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:20.169705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a7945a50-0c74-4024-9468-294c6c30c56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:20.298336 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2100/4199] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:20.310933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:20.313681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:20.322275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:20.427787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24e5a6-68a9-4818-8d14-38832a574707 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:20.450755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1903b5-c6a1-4645-a94f-d47e1c19ccf6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:20.452896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1903b5-c6a1-4645-a94f-d47e1c19ccf6 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:20.453727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1903b5-c6a1-4645-a94f-d47e1c19ccf6 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:20.454472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1903b5-c6a1-4645-a94f-d47e1c19ccf6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:20.594907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1903b5-c6a1-4645-a94f-d47e1c19ccf6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:20.597705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1903b5-c6a1-4645-a94f-d47e1c19ccf6 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:20.620683 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2100/4200] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:20 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:21.415663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7945a50-0c74-4024-9468-294c6c30c56e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:21.525226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7945a50-0c74-4024-9468-294c6c30c56e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:21.534198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:21.535133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:21.535588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:21.536054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] PATH_INFO: `/v3/users/7fd83fc3f0d44dae9e3bbdb9d34cb303` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:21.539324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd83fc3f0d44dae9e3bbdb9d34cb303)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:21.541921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:21.625902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd83fc3f0d44dae9e3bbdb9d34cb303'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:21.627630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:21.664104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd83fc3f0d44dae9e3bbdb9d34cb303'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:21.678537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd83fc3f0d44dae9e3bbdb9d34cb303'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:21.695330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd83fc3f0d44dae9e3bbdb9d34cb303'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:21.696136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd83fc3f0d44dae9e3bbdb9d34cb303'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:21.711060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2101/4201] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:21 2022] DELETE /identity/v3/users/7fd83fc3f0d44dae9e3bbdb9d34cb303 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:21.713085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:21.713664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:21.713921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:21.742218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:21.743574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:21.917135 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2101/4202] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:21.933597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:21.940801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:22.147039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d05955-edc5-4bf8-9fd1-5de20596c57c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:22.156098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:22.157939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:22.158240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:22.158601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:22.193867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:22.197397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:22.238020 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2102/4203] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:03:23.142598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:23.144634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:23.175935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64856576-4720-4070-8864-05b61efdd6b6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:23.206869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:23.208276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:23.208773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:23.209421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] PATH_INFO: `/v3/projects/474b71e9dd7b45999ef7526aa9004a4e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:23.211201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=474b71e9dd7b45999ef7526aa9004a4e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:23.219573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:23.364062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:23.367074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474b71e9dd7b45999ef7526aa9004a4e', 'request_id': 'req-dab019fe-1978-4bef-bef2-75db2e4c73ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:23.379450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474b71e9dd7b45999ef7526aa9004a4e', 'request_id': 'req-dab019fe-1978-4bef-bef2-75db2e4c73ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:23.398892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474b71e9dd7b45999ef7526aa9004a4e', 'request_id': 'req-dab019fe-1978-4bef-bef2-75db2e4c73ed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:23.430817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] The token cache is being invalidate because project 474b71e9dd7b45999ef7526aa9004a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:23.431084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:23.434675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:23.440167 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2102/4204] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:23 2022] DELETE /identity/v3/projects/474b71e9dd7b45999ef7526aa9004a4e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:23.445174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:23.458328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9bfc9-efbb-4fbf-966e-d08d9e960e52 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:23.691467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:23.692489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:23.693044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:23.693552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] PATH_INFO: `/v3/users/b32a28d3e3dd4800a287a715472c9729` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:23.695298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32a28d3e3dd4800a287a715472c9729)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:23.697646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:23.898194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32a28d3e3dd4800a287a715472c9729'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:23.941532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32a28d3e3dd4800a287a715472c9729'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:23.970943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:23.976937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32a28d3e3dd4800a287a715472c9729'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:24.013600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32a28d3e3dd4800a287a715472c9729'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:24.014993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-80d91358-23bc-4f07-9943-476557cca6c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32a28d3e3dd4800a287a715472c9729'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:24.046389 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2103/4205] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:23 2022] DELETE /identity/v3/users/b32a28d3e3dd4800a287a715472c9729 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:24.586257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:24.588234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:24.612313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab019fe-1978-4bef-bef2-75db2e4c73ed admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:24.753001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:24.753539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:24.754160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:24.754597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] PATH_INFO: `/v3/projects/83131dc4a20947329629a903a2251b55` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:24.760364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83131dc4a20947329629a903a2251b55)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:24.763643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:24.934393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83131dc4a20947329629a903a2251b55', 'request_id': 'req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:24.954898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83131dc4a20947329629a903a2251b55', 'request_id': 'req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:24.977516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83131dc4a20947329629a903a2251b55', 'request_id': 'req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:25.013804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] The token cache is being invalidate because project 83131dc4a20947329629a903a2251b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:25.016366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:25.019048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b8ef8ba6-c0bb-4006-a6ca-afa5e25b6687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:25.025730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2103/4206] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:24 2022] DELETE /identity/v3/projects/83131dc4a20947329629a903a2251b55 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:25.028791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fa363-9a82-48d3-8d52-6b11c4f409f9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:25.029527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fa363-9a82-48d3-8d52-6b11c4f409f9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:25.030025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fa363-9a82-48d3-8d52-6b11c4f409f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:25.085357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-451fa363-9a82-48d3-8d52-6b11c4f409f9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:25.094650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-451fa363-9a82-48d3-8d52-6b11c4f409f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:25.266042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4040c99-d1c3-4c25-8da6-9bc844b6c295 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:25.266777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4040c99-d1c3-4c25-8da6-9bc844b6c295 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:25.267091 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2104/4207] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:25.267582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4040c99-d1c3-4c25-8da6-9bc844b6c295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:25.311508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c4040c99-d1c3-4c25-8da6-9bc844b6c295 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:25.314990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c4040c99-d1c3-4c25-8da6-9bc844b6c295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:25.339529 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2104/4208] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:25.344947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fa363-9a82-48d3-8d52-6b11c4f409f9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:25.363566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fa363-9a82-48d3-8d52-6b11c4f409f9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:25.374668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:25.377099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:25.377420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:25.377697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:25.379584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:25.379901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:25.385345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:25.417923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2105/4209] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:25.421325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4040c99-d1c3-4c25-8da6-9bc844b6c295 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:25.437404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4040c99-d1c3-4c25-8da6-9bc844b6c295 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:25.460403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:25.462737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:25.463223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:25.463689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:25.464551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:25.469531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:25.534681 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2105/4210] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:25 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:25.539919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:25.564725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432790-16db-43c4-8a46-f881a4f4e8a5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:25.729960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:25.732570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:25.732856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:25.753681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:25.758545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:25.767511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:25.863499 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2106/4211] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:25 2022] POST /identity/v3/users => generated 302 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:25.865066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:25.881898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb8914-c0f2-4777-9e8b-97005f12bfd0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:25.911186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:25.914461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:25.917143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:25.918121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:25.922761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:25.930594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:25.973516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:25.983331 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2106/4212] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:25.993242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992ba9-5a48-45b9-b9b7-221729f338a7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:26.014588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:26.025875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:26.026175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:26.028966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] PATH_INFO: `/v3/projects/ebb847ae1a414fe5b2a3b49983be7224/users/9699605a94bf4f9698c9448bc442fccb/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:26.084412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb847ae1a414fe5b2a3b49983be7224, user_id=9699605a94bf4f9698c9448bc442fccb, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:26.090475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:26.145128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2107/4213] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:25 2022] PUT /identity/v3/projects/ebb847ae1a414fe5b2a3b49983be7224/users/9699605a94bf4f9698c9448bc442fccb/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:03:26.146442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:26.174110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3348-1cbc-4684-adaf-68695c41153c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:26.506249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:26.592379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267be3ca-7ac7-4721-8fc6-6bbe8a2bf2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:26.593560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267be3ca-7ac7-4721-8fc6-6bbe8a2bf2cc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:26.594446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267be3ca-7ac7-4721-8fc6-6bbe8a2bf2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:26.594978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267be3ca-7ac7-4721-8fc6-6bbe8a2bf2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:26.595789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267be3ca-7ac7-4721-8fc6-6bbe8a2bf2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:26.597702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267be3ca-7ac7-4721-8fc6-6bbe8a2bf2cc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:26.619484 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2107/4214] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:26.624614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:26.641785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb927df-d83b-4897-970d-06b6d555cb0e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:26.654021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:26.659443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:26.659744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:26.661405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] PATH_INFO: `/v3/projects/ebb847ae1a414fe5b2a3b49983be7224/users/9699605a94bf4f9698c9448bc442fccb/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:26.666910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb847ae1a414fe5b2a3b49983be7224, user_id=9699605a94bf4f9698c9448bc442fccb, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:26.675396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:26.706509 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2108/4215] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:26 2022] PUT /identity/v3/projects/ebb847ae1a414fe5b2a3b49983be7224/users/9699605a94bf4f9698c9448bc442fccb/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:26.717328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d58f0-1732-41c9-a6d3-d88495fa7061 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:26.717769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d58f0-1732-41c9-a6d3-d88495fa7061 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:26.718017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d58f0-1732-41c9-a6d3-d88495fa7061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:26.796867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d84d58f0-1732-41c9-a6d3-d88495fa7061 None None] MFA Rules not processed for user `9699605a94bf4f9698c9448bc442fccb`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:26.798568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d84d58f0-1732-41c9-a6d3-d88495fa7061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:26.991521 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2108/4216] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:26.997034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:27.069238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:27.151867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26693dfe-d348-489d-b39b-96a2a8f74a10 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:27.176399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:27.177980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.178349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.178631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:27.180472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:27.184758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:27.273533 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2109/4217] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:26 2022] POST /identity/v3/projects => generated 389 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:27.277275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d58f0-1732-41c9-a6d3-d88495fa7061 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:27.294621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d58f0-1732-41c9-a6d3-d88495fa7061 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:27.536831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:27.538740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.540084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.540373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:27.542214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:27.546321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:27.633388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:27.635779 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2109/4218] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:27 2022] POST /identity/v3/users => generated 295 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:27.650227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f349733-ac77-4729-b26f-4005eec30f02 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:27.661173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:27.661851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.662134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.662416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:27.666963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:27.668837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:27.704160 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2110/4219] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:03:27.705671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:27.737319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be72c3e-d00b-4971-8468-2a9aa031d6c9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:27.756487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49612c-9b0c-4a2c-aefa-1c1532837cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:27.760940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49612c-9b0c-4a2c-aefa-1c1532837cfb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.761231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49612c-9b0c-4a2c-aefa-1c1532837cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.761512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49612c-9b0c-4a2c-aefa-1c1532837cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:27.765778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49612c-9b0c-4a2c-aefa-1c1532837cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:27.772851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49612c-9b0c-4a2c-aefa-1c1532837cfb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:27.815986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2110/4220] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:27.817266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:27.832293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af047e2-f658-486c-89ec-e2c0272633ad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:27.844187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:27.844879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.845164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.845446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] PATH_INFO: `/v3/projects/1ad1f9c2c9b849a1825529136a774ae6/users/656c2645220b4aba868eab1a37f0d420/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:27.894811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad1f9c2c9b849a1825529136a774ae6, user_id=656c2645220b4aba868eab1a37f0d420, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:27.901075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:27.947414 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2111/4221] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:27 2022] PUT /identity/v3/projects/1ad1f9c2c9b849a1825529136a774ae6/users/656c2645220b4aba868eab1a37f0d420/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:27.949236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:27.950334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:27.950712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:28.005436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] MFA Rules not processed for user `656c2645220b4aba868eab1a37f0d420`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:28.007294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:28.036323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:28.179776 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2111/4222] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:28.198922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:28.203120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:28.335825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc94605-a828-4968-bb6e-434182d31d4a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:28.342415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:28.343958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:28.344434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:28.345458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:28.506883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:28.509850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:28.540557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2112/4223] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:03:29.180733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:29.184523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:29.376446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fe83-efdc-4744-a201-c2143dc71bb5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:29.387968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:29.388621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:29.389128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:29.389412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] PATH_INFO: `/v3/users/edef2887c6b741108da9b88745486339` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:29.391741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] RBAC: Authorizing `identity:delete_user(user_id=edef2887c6b741108da9b88745486339)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:29.396843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:29.534415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edef2887c6b741108da9b88745486339'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:29.563533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edef2887c6b741108da9b88745486339'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:29.566446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edef2887c6b741108da9b88745486339'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:29.596507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edef2887c6b741108da9b88745486339'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:29.597852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4b31d4a0-a8fe-4249-97c8-5fd3bdbc734e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edef2887c6b741108da9b88745486339'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:29.639630 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2112/4224] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:29 2022] DELETE /identity/v3/users/edef2887c6b741108da9b88745486339 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:30.688011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:30.706997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9876d8-3135-47a9-8647-88c14453b2e7 service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:30.768439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:30.842207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:30.844624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:30.847038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:30.847595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] PATH_INFO: `/v3/users/9699605a94bf4f9698c9448bc442fccb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:30.851419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9699605a94bf4f9698c9448bc442fccb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:30.853807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:31.023169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9699605a94bf4f9698c9448bc442fccb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.063216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9699605a94bf4f9698c9448bc442fccb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.079303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9699605a94bf4f9698c9448bc442fccb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.103003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9699605a94bf4f9698c9448bc442fccb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.103854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9699605a94bf4f9698c9448bc442fccb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.115246 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2113/4225] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:30 2022] DELETE /identity/v3/users/9699605a94bf4f9698c9448bc442fccb => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:31.119798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:31.120344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:31.120636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:31.162894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:31.167396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:31.289062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:31.343911 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2113/4226] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:31.365118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:31.366782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:31.418753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:31.420641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:31.460845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b119b-2ace-47c5-ac6f-064fe022ef09 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:31.474801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:31.475453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:31.475734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:31.476022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] PATH_INFO: `/v3/projects/daa76318457c400b80564b1035b25633` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:31.483923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa76318457c400b80564b1035b25633)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:31.500540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:31.543815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d4e1c-982d-4483-a8f1-a9b3679c51f1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:31.548501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77810c5f-f134-4e51-a3dc-cfb66f8a5de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:31.549130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77810c5f-f134-4e51-a3dc-cfb66f8a5de5 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:31.549415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77810c5f-f134-4e51-a3dc-cfb66f8a5de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:31.549688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77810c5f-f134-4e51-a3dc-cfb66f8a5de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:31.567520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77810c5f-f134-4e51-a3dc-cfb66f8a5de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:31.570951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77810c5f-f134-4e51-a3dc-cfb66f8a5de5 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:31.598816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2114/4227] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:31.667598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa76318457c400b80564b1035b25633', 'request_id': 'req-d36e6077-9150-406a-a209-a07832a0d464'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.701406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa76318457c400b80564b1035b25633', 'request_id': 'req-d36e6077-9150-406a-a209-a07832a0d464'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.722255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa76318457c400b80564b1035b25633', 'request_id': 'req-d36e6077-9150-406a-a209-a07832a0d464'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.743088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] The token cache is being invalidate because project daa76318457c400b80564b1035b25633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:31.743388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.745022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d36e6077-9150-406a-a209-a07832a0d464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:31.747043 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2114/4228] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:31 2022] DELETE /identity/v3/projects/daa76318457c400b80564b1035b25633 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:31.752019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317e72a-afd8-4304-b700-b48155322b77 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:31.752568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317e72a-afd8-4304-b700-b48155322b77 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:31.752858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317e72a-afd8-4304-b700-b48155322b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:31.789078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b317e72a-afd8-4304-b700-b48155322b77 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:31.791290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b317e72a-afd8-4304-b700-b48155322b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:31.878211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:31.925789 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2115/4229] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:31.933065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d2612-2872-43fc-939e-fe23669566eb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:31.933592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d2612-2872-43fc-939e-fe23669566eb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:31.934164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d2612-2872-43fc-939e-fe23669566eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:31.970380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e98d2612-2872-43fc-939e-fe23669566eb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:31.972486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e98d2612-2872-43fc-939e-fe23669566eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:31.995440 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2115/4230] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:32.003263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317e72a-afd8-4304-b700-b48155322b77 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:32.024484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317e72a-afd8-4304-b700-b48155322b77 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:32.033599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:32.034381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:32.034802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:32.035088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:32.035798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:32.036111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:32.038172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:32.066534 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2116/4231] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:32.070127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d2612-2872-43fc-939e-fe23669566eb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:32.107530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d2612-2872-43fc-939e-fe23669566eb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:32.124040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:32.125177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:32.126824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:32.129562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:32.132138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:32.138903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:32.233617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2116/4232] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:32 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:32.236996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:32.252854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dbb6a-b20a-4dbb-ab26-c1b918c12b50 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:32.386540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:32.387400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:32.388237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:32.388237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:32.388788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:32.390965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:32.436143 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2117/4233] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:32 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:32.438003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:32.459853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c85a3-0e92-4a7e-87cc-d9b1bf648f48 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:32.472449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:32.473527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:32.474116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:32.475126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:32.476022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:32.483946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:32.509630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:32.509930 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2117/4234] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:32.547146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2f59c-f9e7-4b09-811d-d2905bd4be5c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:32.562489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea8ecc-caee-4110-b4d5-695bbf3beb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:32.563297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea8ecc-caee-4110-b4d5-695bbf3beb86 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:32.563686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea8ecc-caee-4110-b4d5-695bbf3beb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:32.564137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea8ecc-caee-4110-b4d5-695bbf3beb86 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:32.564900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea8ecc-caee-4110-b4d5-695bbf3beb86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:32.566760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea8ecc-caee-4110-b4d5-695bbf3beb86 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:32.587192 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2118/4235] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:03:32.592204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:32.603923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dce75-1911-4400-b49c-42b9affad01c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:32.612302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:32.613472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:32.614035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:32.615146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] PATH_INFO: `/v3/projects/54cc07e66b1d4c55827d9b723a8df459/users/42fe36190f974a289414202fe1e4ee44/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:32.644155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54cc07e66b1d4c55827d9b723a8df459, user_id=42fe36190f974a289414202fe1e4ee44, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:32.648673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:32.706726 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2118/4236] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:32 2022] PUT /identity/v3/projects/54cc07e66b1d4c55827d9b723a8df459/users/42fe36190f974a289414202fe1e4ee44/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:32.710210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:32.710836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:32.711364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:32.744297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:32.747131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:32.875687 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2119/4237] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:32.897337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:32.910021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:33.032351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911c4cf-bff0-4b42-9c5a-d010563bfd21 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:33.038630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:33.039519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:33.040009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:33.040475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:33.062327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:33.067094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:33.080680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:33.081923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:33.083001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:33.097891 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2119/4238] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:33.115104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c90855-ec20-408b-b07a-7db8b8c7f9a6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:33.124727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:33.125788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:33.126486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:33.126879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] PATH_INFO: `/v3/projects/ebb847ae1a414fe5b2a3b49983be7224` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:33.128601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb847ae1a414fe5b2a3b49983be7224)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:33.132302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:33.286191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb847ae1a414fe5b2a3b49983be7224', 'request_id': 'req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.287808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:33.320716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb847ae1a414fe5b2a3b49983be7224', 'request_id': 'req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.339824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb847ae1a414fe5b2a3b49983be7224', 'request_id': 'req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.384396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] The token cache is being invalidate because project ebb847ae1a414fe5b2a3b49983be7224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:33.385339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.387363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.395377 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2120/4239] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:33 2022] DELETE /identity/v3/projects/ebb847ae1a414fe5b2a3b49983be7224 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:33.396549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:33.413120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df70df-ade3-48ec-a6f7-e089a740f613 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:33.556991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:33.557658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:33.557970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:33.558242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] PATH_INFO: `/v3/users/656c2645220b4aba868eab1a37f0d420` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:33.564627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=656c2645220b4aba868eab1a37f0d420)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:33.566985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:33.700756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656c2645220b4aba868eab1a37f0d420'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.713441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656c2645220b4aba868eab1a37f0d420'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.714707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656c2645220b4aba868eab1a37f0d420'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.744479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656c2645220b4aba868eab1a37f0d420'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.745942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a8d830a6-d0dd-469a-9f35-496558d2c76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656c2645220b4aba868eab1a37f0d420'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:33.824432 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2120/4240] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:33 2022] DELETE /identity/v3/users/656c2645220b4aba868eab1a37f0d420 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:03:35.309223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:35.312297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:35.331363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc0eb4-eb8d-4379-a7e4-7adfd62567fc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:35.440796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:35.442773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:35.442773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:35.443262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] PATH_INFO: `/v3/users/6f50ca756d594703846f1dade2d38eac` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:35.444701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f50ca756d594703846f1dade2d38eac)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:35.447236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:35.580001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f50ca756d594703846f1dade2d38eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:35.624044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f50ca756d594703846f1dade2d38eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:35.643625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f50ca756d594703846f1dade2d38eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:35.659618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f50ca756d594703846f1dade2d38eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:35.660507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f50ca756d594703846f1dade2d38eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:35.676792 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2121/4241] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:35 2022] DELETE /identity/v3/users/6f50ca756d594703846f1dade2d38eac => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:35.752889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:35.753341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:35.753465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:35.782369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:35.784275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:35.908632 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2121/4242] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:36.006234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.009374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:36.145668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:36.150061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ab292-0578-4cfc-b1d9-f807405473b7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.157605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:36.158423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.159155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.159155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] PATH_INFO: `/v3/projects/dd2cd4bb350d452eb8c5e2533642c0f0` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.160662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd2cd4bb350d452eb8c5e2533642c0f0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:36.165217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:36.199722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.201475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:36.223471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c640be-de49-4aa6-8ac9-319c3487162d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.244046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:36.246873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.250104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.252126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] PATH_INFO: `/v3/projects/1ad1f9c2c9b849a1825529136a774ae6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.254478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad1f9c2c9b849a1825529136a774ae6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:36.258215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:36.279911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd2cd4bb350d452eb8c5e2533642c0f0', 'request_id': 'req-199c7eb7-d010-46ec-bcc1-291422c32a3a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.290053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:36.300038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd2cd4bb350d452eb8c5e2533642c0f0', 'request_id': 'req-199c7eb7-d010-46ec-bcc1-291422c32a3a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.320767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd2cd4bb350d452eb8c5e2533642c0f0', 'request_id': 'req-199c7eb7-d010-46ec-bcc1-291422c32a3a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.360383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] The token cache is being invalidate because project dd2cd4bb350d452eb8c5e2533642c0f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:36.361239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.363917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-199c7eb7-d010-46ec-bcc1-291422c32a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.368739 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2122/4243] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:36 2022] DELETE /identity/v3/projects/dd2cd4bb350d452eb8c5e2533642c0f0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:36.376241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad1f9c2c9b849a1825529136a774ae6', 'request_id': 'req-41bf6cac-b351-4801-9f8e-aecdebbbf263'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.376481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580eab-4176-452d-ae71-185300cbb461 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.377039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580eab-4176-452d-ae71-185300cbb461 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.377469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580eab-4176-452d-ae71-185300cbb461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.385516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad1f9c2c9b849a1825529136a774ae6', 'request_id': 'req-41bf6cac-b351-4801-9f8e-aecdebbbf263'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.404359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad1f9c2c9b849a1825529136a774ae6', 'request_id': 'req-41bf6cac-b351-4801-9f8e-aecdebbbf263'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.412577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-22580eab-4176-452d-ae71-185300cbb461 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:36.414349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22580eab-4176-452d-ae71-185300cbb461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:36.431822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] The token cache is being invalidate because project 1ad1f9c2c9b849a1825529136a774ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:36.432429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.433182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41bf6cac-b351-4801-9f8e-aecdebbbf263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:36.435278 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2122/4244] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:36 2022] DELETE /identity/v3/projects/1ad1f9c2c9b849a1825529136a774ae6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:36.442061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a699-c25e-4d81-b3cd-f1ad750437e9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.443408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a699-c25e-4d81-b3cd-f1ad750437e9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.443777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a699-c25e-4d81-b3cd-f1ad750437e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.478514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8ff1a699-c25e-4d81-b3cd-f1ad750437e9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:36.479923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8ff1a699-c25e-4d81-b3cd-f1ad750437e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:36.509524 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2123/4245] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:36.518380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc2fcf-5fad-4c08-84f7-a2bc3f60dcad None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.518755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc2fcf-5fad-4c08-84f7-a2bc3f60dcad None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.519040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc2fcf-5fad-4c08-84f7-a2bc3f60dcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.545006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-82cc2fcf-5fad-4c08-84f7-a2bc3f60dcad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:36.546492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-82cc2fcf-5fad-4c08-84f7-a2bc3f60dcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:36.561769 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2124/4246] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:36.565656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc2fcf-5fad-4c08-84f7-a2bc3f60dcad None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.572727 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2123/4247] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:36.582332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dcb2-15a2-42aa-a911-867401ef9a82 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.582968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dcb2-15a2-42aa-a911-867401ef9a82 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.583469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dcb2-15a2-42aa-a911-867401ef9a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.583906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc2fcf-5fad-4c08-84f7-a2bc3f60dcad None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.591791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:36.592451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.592731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.593320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.594047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:36.594334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:36.596968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:36.619538 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2125/4248] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:36.621908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e9b8dcb2-15a2-42aa-a911-867401ef9a82 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:36.623838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e9b8dcb2-15a2-42aa-a911-867401ef9a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:36.626251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.635997 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2124/4249] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:36.640376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8dcb2-15a2-42aa-a911-867401ef9a82 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.643937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c97d3-1de8-47a1-ad83-2bb441dd6032 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.658068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8dcb2-15a2-42aa-a911-867401ef9a82 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.664639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:36.665232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:36.665431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.666028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.666380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.667727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:36.668179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:36.669949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.670233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.670514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.670842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:36.671263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:36.674210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:36.693153 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2125/4250] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:36.697549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.722766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25950100-4033-43cd-ae03-f4078c11ed37 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.729580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:36.730452 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2126/4251] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:36.734291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.734608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.735002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.735137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.735925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:36.738255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:36.808388 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2126/4252] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:36.810671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:36.813057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:36.839089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82830e7e-ec5f-4d91-a701-e4e3b9e4cac1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.961499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:36.977493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f79ca6-16d9-4022-bd99-188a911bdfa4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:36.992161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:36.994394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:36.995569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:36.996862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:36.997872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.004522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.013938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.016180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.016492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.018588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.020440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.027279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.075270 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2127/4253] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/users => generated 287 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:37.078564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.092639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2127/4254] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:36 2022] POST /identity/v3/users => generated 292 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:37.095263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.109287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175be19-8090-4cdd-8e84-861dc7b0af52 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:37.126831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023bea4-618b-4caa-a156-9bd9f4f3f618 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:37.127207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.128003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.128432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.129312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.131338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.138048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.144498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.145296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.146023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.147699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.149996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.154538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.168151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2128/4255] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:03:37.173392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.186008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2128/4256] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:37.189972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.207103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f765537-5d7f-475d-a883-d2c395836e25 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:37.226833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.232207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.233098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e2416-6345-4868-8e55-52e91651a190 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:37.234586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.236432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.237228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.247878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.253424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.255632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.256512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.257632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.262731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.271484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.298953 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2129/4257] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:37.304137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.318745 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2129/4258] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:37.324036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.342600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78a45-b8d7-4454-b31d-baaa66b45378 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:37.354406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620660f-bed3-462c-a13d-35478533530e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:37.358482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31902f46-f5cc-4e25-b050-07ef7daac3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.360266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31902f46-f5cc-4e25-b050-07ef7daac3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.360622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31902f46-f5cc-4e25-b050-07ef7daac3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.360909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31902f46-f5cc-4e25-b050-07ef7daac3a3 admin admin] PATH_INFO: `/v3/projects/def868f8a75440f4b0af981ee5bfea7c/users/da3dfed7b6b64a7c82946dfeacb82a15/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.374480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043c5e5-9c97-491a-8e89-62036bb31fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.375426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043c5e5-9c97-491a-8e89-62036bb31fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.379297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043c5e5-9c97-491a-8e89-62036bb31fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.380401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043c5e5-9c97-491a-8e89-62036bb31fa3 admin admin] PATH_INFO: `/v3/projects/7c3ba8eb9cfb41cb87be6ffad5cb3789/users/3561f4fb18f84b899f68c5fcf897ca12/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.394315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31902f46-f5cc-4e25-b050-07ef7daac3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def868f8a75440f4b0af981ee5bfea7c, user_id=da3dfed7b6b64a7c82946dfeacb82a15, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.397065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31902f46-f5cc-4e25-b050-07ef7daac3a3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.455105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043c5e5-9c97-491a-8e89-62036bb31fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3ba8eb9cfb41cb87be6ffad5cb3789, user_id=3561f4fb18f84b899f68c5fcf897ca12, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.460651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043c5e5-9c97-491a-8e89-62036bb31fa3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:37.471528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2130/4259] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:37 2022] PUT /identity/v3/projects/def868f8a75440f4b0af981ee5bfea7c/users/da3dfed7b6b64a7c82946dfeacb82a15/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:37.484504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.486478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.486854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.555097 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2130/4260] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:37 2022] PUT /identity/v3/projects/7c3ba8eb9cfb41cb87be6ffad5cb3789/users/3561f4fb18f84b899f68c5fcf897ca12/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:37.556543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.570395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.570696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.577808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] MFA Rules not processed for user `da3dfed7b6b64a7c82946dfeacb82a15`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:37.585225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:37.643050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:37.657427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:37.759912 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2131/4261] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:37.760954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.767980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:37.878178 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2131/4262] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:37.878178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:37.882149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:37.928174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f942f-0d20-4bcb-9207-29ced25e3384 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:37.942771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:37.944186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:37.944782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:37.945289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:37.948993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:37.989191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:37.993728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:37.997273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:38.020985 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2132/4263] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:38.030775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4887-c49d-4985-8105-fd165bf5e1da None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:38.034764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:38.038836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:38.039264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:38.039558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:38.040068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.040199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:38.066868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:38.068038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:38.068332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:38.086922 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2132/4264] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:38.087541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:38.096071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.238789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988a721-1da9-443b-906d-7f94d2c7b7dd service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:38.252995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:38.253840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:38.254339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:38.254564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:38.279912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:38.283980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:38.308222 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2133/4265] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:38.311875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:38.315149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.350269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59796b-b1fa-4a5e-83a4-eef9249a961c service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:38.350742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470eb19-8d3d-43ed-87f4-8d8979a11188 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:38.365924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5397dc7-d07e-487e-ac23-3c931410bb94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:38.366890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5397dc7-d07e-487e-ac23-3c931410bb94 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:38.367081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5397dc7-d07e-487e-ac23-3c931410bb94 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:38.367286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d010e-4c1c-4c8d-9975-027c25082dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:38.367434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5397dc7-d07e-487e-ac23-3c931410bb94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:38.369635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d010e-4c1c-4c8d-9975-027c25082dcc service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:38.370037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d010e-4c1c-4c8d-9975-027c25082dcc service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:38.370359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d010e-4c1c-4c8d-9975-027c25082dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:38.371729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b5397dc7-d07e-487e-ac23-3c931410bb94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.377023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4b4d010e-4c1c-4c8d-9975-027c25082dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.401525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5397dc7-d07e-487e-ac23-3c931410bb94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:38.407715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5397dc7-d07e-487e-ac23-3c931410bb94 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:38.442599 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2134/4266] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:38.509706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:38.510084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:38.510373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:38.536242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d010e-4c1c-4c8d-9975-027c25082dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:38.539298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d010e-4c1c-4c8d-9975-027c25082dcc service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:38.568134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:38.569632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.638008 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2133/4267] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:38.666877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35232-9e3f-4ee9-9af2-f895f8cdfbd0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:38.667347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35232-9e3f-4ee9-9af2-f895f8cdfbd0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:38.667681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa35232-9e3f-4ee9-9af2-f895f8cdfbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:38.714569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2fa35232-9e3f-4ee9-9af2-f895f8cdfbd0 None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:38.718232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2fa35232-9e3f-4ee9-9af2-f895f8cdfbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.725195 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2135/4268] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:38.746546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:38.749776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.893020 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2134/4269] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:38.896031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa35232-9e3f-4ee9-9af2-f895f8cdfbd0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:38.923992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa35232-9e3f-4ee9-9af2-f895f8cdfbd0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:38.931880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bacbc-73ac-410d-9445-d22f607a0d7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:38.932680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bacbc-73ac-410d-9445-d22f607a0d7c service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:38.933187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bacbc-73ac-410d-9445-d22f607a0d7c service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:38.933697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bacbc-73ac-410d-9445-d22f607a0d7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:38.937667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e65bacbc-73ac-410d-9445-d22f607a0d7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:38.956523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bacbc-73ac-410d-9445-d22f607a0d7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:38.959296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bacbc-73ac-410d-9445-d22f607a0d7c service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:38.989679 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2135/4270] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:39.037291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfb673-d652-4d74-b4c2-41d614012504 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:39.054926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:39.056923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:39.057478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:39.058257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:39.072209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:39.075481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:39.088576 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2136/4271] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:39.538433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9c862-95a9-40c7-9d0e-5cb3a538453b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:39.539341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9c862-95a9-40c7-9d0e-5cb3a538453b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:39.539641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9c862-95a9-40c7-9d0e-5cb3a538453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:39.628347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a4d9c862-95a9-40c7-9d0e-5cb3a538453b None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:39.630485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4d9c862-95a9-40c7-9d0e-5cb3a538453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:39.651535 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2136/4272] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:39.677258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:39.686537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:39.695086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9857df-25a1-4015-859d-10865879df70 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:39.711685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:39.712321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:39.712627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:39.712904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:39.733169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:39.739170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:39.756988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2137/4273] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:40.304287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549370d2-66fb-42ac-82f8-487c9fcf1f9b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:40.304987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549370d2-66fb-42ac-82f8-487c9fcf1f9b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:40.305101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549370d2-66fb-42ac-82f8-487c9fcf1f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:40.358263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-549370d2-66fb-42ac-82f8-487c9fcf1f9b None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:40.359630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-549370d2-66fb-42ac-82f8-487c9fcf1f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:40.380580 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2137/4274] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:40.400782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:40.420753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a24fdd-d030-4805-93a2-6b7e68d55d37 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:40.427962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:40.429066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:40.429566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:40.430072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:40.442318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:40.444778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:40.458684 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2138/4275] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:41.100474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3597b-2599-47d3-8d75-5512ce9d9a93 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:41.100767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3597b-2599-47d3-8d75-5512ce9d9a93 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:41.101037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3597b-2599-47d3-8d75-5512ce9d9a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:41.156845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-95c3597b-2599-47d3-8d75-5512ce9d9a93 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:41.161436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-95c3597b-2599-47d3-8d75-5512ce9d9a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:41.195268 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2138/4276] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:41.210951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:41.230248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d661e21-dc01-4bdb-9e6d-bfb893e162cf service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:41.250401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:41.253548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:41.254402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:41.255447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:41.289810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:41.302979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:41.335023 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2139/4277] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:41.994407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766b15-5757-4f03-9f93-9ec5c6a03ae4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:41.995361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766b15-5757-4f03-9f93-9ec5c6a03ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:41.995800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766b15-5757-4f03-9f93-9ec5c6a03ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:42.044700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-54766b15-5757-4f03-9f93-9ec5c6a03ae4 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:42.047143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-54766b15-5757-4f03-9f93-9ec5c6a03ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:42.063154 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2139/4278] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:42.082779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:42.106809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c47c5-7f23-4bbb-bc78-0e161df05ea1 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:42.126251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:42.128439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:42.131166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:42.131720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:42.158714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:42.165747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:42.210823 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2140/4279] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:42.725230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de66b5-309e-4cb4-a931-bb6372cf67ac None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:42.726511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de66b5-309e-4cb4-a931-bb6372cf67ac None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:42.727170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de66b5-309e-4cb4-a931-bb6372cf67ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:42.751963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:42.786195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-21de66b5-309e-4cb4-a931-bb6372cf67ac None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:42.787509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21de66b5-309e-4cb4-a931-bb6372cf67ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:42.801248 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2140/4280] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:42.820380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:42.846799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f137fc5-297e-4150-b6a0-843f42c35ab6 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:42.854396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:42.856127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:42.856849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:42.858553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:42.872246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:42.874872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:42.888783 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2141/4281] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:44.533597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02cd0-9167-43f1-8a48-996533361de6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:44.534223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02cd0-9167-43f1-8a48-996533361de6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:44.534223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02cd0-9167-43f1-8a48-996533361de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:44.578091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2df02cd0-9167-43f1-8a48-996533361de6 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:44.584428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2df02cd0-9167-43f1-8a48-996533361de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:44.610794 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2141/4282] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 10 21:03:44.627571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:44.653673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85950c63-7c76-4057-ba44-7a912f34d525 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:44.677569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:44.679636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:44.680244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:44.680732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:44.710324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:44.712787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:44.735968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2142/4283] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:45.035156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec4e2-bacc-4ab4-871f-9acea48417ed None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.035156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec4e2-bacc-4ab4-871f-9acea48417ed None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.035156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec4e2-bacc-4ab4-871f-9acea48417ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.082851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-28aec4e2-bacc-4ab4-871f-9acea48417ed None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:45.084103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-28aec4e2-bacc-4ab4-871f-9acea48417ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.104957 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2142/4284] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:45.117993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.132564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29aac6-698d-4620-8bd3-49386b6c523f service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.151794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d6eed-75cc-49bb-b09a-81a272a3cddf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.154174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d6eed-75cc-49bb-b09a-81a272a3cddf service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.154624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d6eed-75cc-49bb-b09a-81a272a3cddf service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.155063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d6eed-75cc-49bb-b09a-81a272a3cddf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.177095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d6eed-75cc-49bb-b09a-81a272a3cddf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.179214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d6eed-75cc-49bb-b09a-81a272a3cddf service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.205980 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2143/4285] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:45.458341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.458341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.458341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.461708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876047f0-c022-41a2-9025-4be78e87997c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.462103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876047f0-c022-41a2-9025-4be78e87997c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.462445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876047f0-c022-41a2-9025-4be78e87997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.515880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:45.516211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-876047f0-c022-41a2-9025-4be78e87997c None None] MFA Rules not processed for user `96372af27d29482491d613b5a783a0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:45.519582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-876047f0-c022-41a2-9025-4be78e87997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.520596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.542347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2144/4286] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:45.545811 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2143/4287] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:45.575674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876047f0-c022-41a2-9025-4be78e87997c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.617772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876047f0-c022-41a2-9025-4be78e87997c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.630559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb529f59-0057-41ed-b81c-92ea07d70628 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.633638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529f59-0057-41ed-b81c-92ea07d70628 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.633953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529f59-0057-41ed-b81c-92ea07d70628 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.634972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529f59-0057-41ed-b81c-92ea07d70628 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.682470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb529f59-0057-41ed-b81c-92ea07d70628 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.687421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb529f59-0057-41ed-b81c-92ea07d70628 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.743748 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2145/4288] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:45.792099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:45.798274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.835696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02824b-889e-426c-8c82-5dae6067614d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:45.863900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:45.864919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:45.865470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:45.866771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:45.874897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:45.896975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:45.901713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:45.971390 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2144/4289] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:46.153863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e14a8-9de5-4d60-875f-4164d7def957 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:46.154457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e14a8-9de5-4d60-875f-4164d7def957 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:46.154592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e14a8-9de5-4d60-875f-4164d7def957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:46.238961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-428e14a8-9de5-4d60-875f-4164d7def957 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:46.241925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-428e14a8-9de5-4d60-875f-4164d7def957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:46.253475 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2146/4290] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:46.273986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:46.297506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa96f5-9b96-4366-91d5-9bd90b2a963c service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:46.312417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:46.315256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:46.316482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:46.318866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:46.349951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:46.354208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:46.384041 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2145/4291] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:46.669734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151db66-5ae1-4cb7-b26b-764610f61502 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:46.670353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151db66-5ae1-4cb7-b26b-764610f61502 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:46.670481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151db66-5ae1-4cb7-b26b-764610f61502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:46.698253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8151db66-5ae1-4cb7-b26b-764610f61502 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:46.701903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8151db66-5ae1-4cb7-b26b-764610f61502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:46.723029 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2147/4292] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:46.732021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:46.760868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2a02a-9505-4dd1-a096-9a8bd795e258 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:46.786288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:46.787712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:46.788209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:46.788731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:46.817385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:46.819047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:46.837166 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2146/4293] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:47.116940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c6576-2961-4023-bce2-f753cb6314c6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:47.117234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c6576-2961-4023-bce2-f753cb6314c6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:47.117521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c6576-2961-4023-bce2-f753cb6314c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:47.148965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-831c6576-2961-4023-bce2-f753cb6314c6 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:47.150283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-831c6576-2961-4023-bce2-f753cb6314c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:47.173392 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2148/4294] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:47.181969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:47.209212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c8fe8-b5f3-43cc-a7b9-57496a8d6721 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:47.222732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd95b0-470d-4d64-9be1-79b982bd4558 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:47.223693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd95b0-470d-4d64-9be1-79b982bd4558 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:47.224154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd95b0-470d-4d64-9be1-79b982bd4558 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:47.224895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd95b0-470d-4d64-9be1-79b982bd4558 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:47.240133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd95b0-470d-4d64-9be1-79b982bd4558 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:47.242133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd95b0-470d-4d64-9be1-79b982bd4558 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:47.267611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2147/4295] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:03:47.451827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c6576-2961-4023-bce2-f753cb6314c6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:47.474902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c6576-2961-4023-bce2-f753cb6314c6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:47.495069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:47.499006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:47.499696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:47.500908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] PATH_INFO: `/v3/users/da3dfed7b6b64a7c82946dfeacb82a15` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:47.504849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3dfed7b6b64a7c82946dfeacb82a15)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:47.509744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:47.613454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3dfed7b6b64a7c82946dfeacb82a15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:47.662399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3dfed7b6b64a7c82946dfeacb82a15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:47.677127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3dfed7b6b64a7c82946dfeacb82a15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:47.695648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3dfed7b6b64a7c82946dfeacb82a15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:47.695648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3dfed7b6b64a7c82946dfeacb82a15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:47.708864 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2149/4296] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:03:47 2022] DELETE /identity/v3/users/da3dfed7b6b64a7c82946dfeacb82a15 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:47.712994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78201af-7533-4243-bff9-f42577453b3f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:47.713533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78201af-7533-4243-bff9-f42577453b3f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:47.714230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78201af-7533-4243-bff9-f42577453b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:47.749882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e78201af-7533-4243-bff9-f42577453b3f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:47.760974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e78201af-7533-4243-bff9-f42577453b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:47.945277 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2148/4297] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:48.029143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:48.031297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:48.059872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:48.189515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f4c56-711e-4e52-a152-9227dfaab5ba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:48.203447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c6bb2-f9b3-4687-9cb3-31aefbb59461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:48.204100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c6bb2-f9b3-4687-9cb3-31aefbb59461 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:48.204547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c6bb2-f9b3-4687-9cb3-31aefbb59461 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:48.204824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c6bb2-f9b3-4687-9cb3-31aefbb59461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:48.240771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c6bb2-f9b3-4687-9cb3-31aefbb59461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:48.242386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c6bb2-f9b3-4687-9cb3-31aefbb59461 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:48.270037 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2150/4298] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:48.578719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78201af-7533-4243-bff9-f42577453b3f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:48.591177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78201af-7533-4243-bff9-f42577453b3f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:48.597604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792e0d-bc9b-47d2-91a5-daa480843758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:48.598664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e0d-bc9b-47d2-91a5-daa480843758 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:48.599169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e0d-bc9b-47d2-91a5-daa480843758 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:48.599975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e0d-bc9b-47d2-91a5-daa480843758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:48.602103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23792e0d-bc9b-47d2-91a5-daa480843758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:48.614076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23792e0d-bc9b-47d2-91a5-daa480843758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:48.616138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23792e0d-bc9b-47d2-91a5-daa480843758 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:48.627984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2149/4299] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:49.360078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:49.360923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:49.361442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:49.400627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] MFA Rules not processed for user `42fe36190f974a289414202fe1e4ee44`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:49.403192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:49.514710 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2151/4300] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:49.519790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:49.521437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:49.521987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:49.579492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] MFA Rules not processed for user `42fe36190f974a289414202fe1e4ee44`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:49.583492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:49.603980 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2150/4301] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:49.616657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:49.618529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:49.810280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95501db-d3df-47af-8c3c-27d4d21edbe1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:49.818451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572b56c-d53c-4d87-8db7-02c41cae6cdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:49.819288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b56c-d53c-4d87-8db7-02c41cae6cdc service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:49.820322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b56c-d53c-4d87-8db7-02c41cae6cdc service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:49.820761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b56c-d53c-4d87-8db7-02c41cae6cdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:49.857294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572b56c-d53c-4d87-8db7-02c41cae6cdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:49.867657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572b56c-d53c-4d87-8db7-02c41cae6cdc service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:49.904127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2152/4302] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:50.181260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:50.187775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:50.222796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644da0d1-82de-46e4-a280-bfdab0554c2c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:50.237681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:50.241357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.241703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.242877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] PATH_INFO: `/v3/projects/def868f8a75440f4b0af981ee5bfea7c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.244554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] RBAC: Authorizing `identity:delete_project(project_id=def868f8a75440f4b0af981ee5bfea7c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:50.250971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:50.366344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def868f8a75440f4b0af981ee5bfea7c', 'request_id': 'req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:50.375876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def868f8a75440f4b0af981ee5bfea7c', 'request_id': 'req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:50.380405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:50.391771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def868f8a75440f4b0af981ee5bfea7c', 'request_id': 'req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:50.430658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] The token cache is being invalidate because project def868f8a75440f4b0af981ee5bfea7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:03:50.431039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:50.432097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2ee7432f-b7c2-40c0-88f8-5ddf8296491e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:03:50.441727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2203f0-3eba-4962-9c48-91750d534ee0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.443109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2203f0-3eba-4962-9c48-91750d534ee0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.443623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2203f0-3eba-4962-9c48-91750d534ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.452263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2151/4303] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:03:50 2022] DELETE /identity/v3/projects/def868f8a75440f4b0af981ee5bfea7c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:50.482319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1f2203f0-3eba-4962-9c48-91750d534ee0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:50.484085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1f2203f0-3eba-4962-9c48-91750d534ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:50.526082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:50.655629 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2153/4304] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:50.668534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da5018-0947-4940-a5ab-26eeec8c9236 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.669432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da5018-0947-4940-a5ab-26eeec8c9236 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.669432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da5018-0947-4940-a5ab-26eeec8c9236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.753083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e9da5018-0947-4940-a5ab-26eeec8c9236 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:50.754770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e9da5018-0947-4940-a5ab-26eeec8c9236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:50.778931 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2152/4305] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:50.782483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2203f0-3eba-4962-9c48-91750d534ee0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:50.796580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2203f0-3eba-4962-9c48-91750d534ee0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:50.809933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:50.812533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.813277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.813560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.815439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:50.815752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:50.825162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:50.869228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2154/4306] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:50.874496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da5018-0947-4940-a5ab-26eeec8c9236 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:50.891516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da5018-0947-4940-a5ab-26eeec8c9236 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:50.907771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:50.908414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:50.912244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:50.912627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:50.913398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:50.921671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.008327 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2153/4307] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:50 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:51.013867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.027711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719898f-cf49-4130-b2e0-63a23b228cbe admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.112619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:51.113598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.113978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.114256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.115023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.118477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.150957 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2155/4308] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:51 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:51.156058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.176520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95d468-d395-43d9-b5ec-b73d2ac62905 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.191005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:51.192005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.192447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.192895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.194292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.196283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.227922 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2154/4309] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:51.231907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.270715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76032c9-1f9b-4caa-8056-48f66c95817b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.277064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95086026-58f0-4e09-a263-b58c135aa0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:51.277759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95086026-58f0-4e09-a263-b58c135aa0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.278075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95086026-58f0-4e09-a263-b58c135aa0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.278350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95086026-58f0-4e09-a263-b58c135aa0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.279057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95086026-58f0-4e09-a263-b58c135aa0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.281032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95086026-58f0-4e09-a263-b58c135aa0c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.299240 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2156/4310] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:51.303725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.324805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6819b9-9f65-4f29-94a1-1b00b9598746 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.338682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:51.340027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.340727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.341019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] PATH_INFO: `/v3/projects/958ad0fded7945baa0d98bd7f037a3a9/users/f230e2825abf4997a2f8bdf470a7adfb/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.378257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] RBAC: Authorizing `identity:create_grant(project_id=958ad0fded7945baa0d98bd7f037a3a9, user_id=f230e2825abf4997a2f8bdf470a7adfb, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.385780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.418733 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2155/4311] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:51 2022] PUT /identity/v3/projects/958ad0fded7945baa0d98bd7f037a3a9/users/f230e2825abf4997a2f8bdf470a7adfb/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:51.429185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdabc3-5c36-4e5d-be59-11b0e41b8164 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.429545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdabc3-5c36-4e5d-be59-11b0e41b8164 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.429875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdabc3-5c36-4e5d-be59-11b0e41b8164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.477439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-21bdabc3-5c36-4e5d-be59-11b0e41b8164 None None] MFA Rules not processed for user `f230e2825abf4997a2f8bdf470a7adfb`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:51.478934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-21bdabc3-5c36-4e5d-be59-11b0e41b8164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:51.731635 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2157/4312] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:51.736053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.823219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146a4bb-2735-4e1c-98b6-7d603ce1955c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.833472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c6a49-ef58-4535-aa81-61cc4f68faca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:51.834693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c6a49-ef58-4535-aa81-61cc4f68faca admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:51.835190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c6a49-ef58-4535-aa81-61cc4f68faca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:51.836778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c6a49-ef58-4535-aa81-61cc4f68faca admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:51.837890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c6a49-ef58-4535-aa81-61cc4f68faca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:51.843753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c6a49-ef58-4535-aa81-61cc4f68faca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:51.889910 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2156/4313] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:51 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:03:51.894623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdabc3-5c36-4e5d-be59-11b0e41b8164 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:51.911984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdabc3-5c36-4e5d-be59-11b0e41b8164 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:51.938293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:52.075642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de863c68-7ebe-4203-ba41-b9326df646a1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.082237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de863c68-7ebe-4203-ba41-b9326df646a1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.082910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de863c68-7ebe-4203-ba41-b9326df646a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.135421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-de863c68-7ebe-4203-ba41-b9326df646a1 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:52.140106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de863c68-7ebe-4203-ba41-b9326df646a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:52.166929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:52.177926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.178235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.178513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.179200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:52.181169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:52.236581 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2158/4314] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:03:51 2022] POST /identity/v3/users => generated 292 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:03:52.237518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:52.260836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4adeaa-3456-441b-93b7-42d8f14af3f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:52.275280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:52.275931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.276181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.276614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.277201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:52.278843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:52.301380 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2157/4315] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:03:52.304034 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2159/4316] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:52.308360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de863c68-7ebe-4203-ba41-b9326df646a1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:52.318503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:52.321023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:52.331600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de863c68-7ebe-4203-ba41-b9326df646a1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:52.338175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:52.339908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.340928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.341502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] PATH_INFO: `/v3/projects/4c5b307771994b22848c3290c5496746/users/891b11b732c94e189752db093ef4a5db/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.358596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:52.365971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5b307771994b22848c3290c5496746, user_id=891b11b732c94e189752db093ef4a5db, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:52.376331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:52.399244 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2158/4317] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:52 2022] PUT /identity/v3/projects/4c5b307771994b22848c3290c5496746/users/891b11b732c94e189752db093ef4a5db/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:52.403707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:52.514929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955408ca-0ab5-4212-8fa4-201bdbcc7737 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:52.517980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8410a-21d9-4edb-93fe-7a1d930832ba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:52.526090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:52.528166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.532783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.533461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.534850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:52.536901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:52.556251 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2159/4318] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:03:52.561675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:52.612124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee9216-5c96-49b4-85e0-42e018122f41 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:52.633636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f728fe9-4d9f-45aa-bdc7-5639669a4970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:52.635440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f728fe9-4d9f-45aa-bdc7-5639669a4970 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.638863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f728fe9-4d9f-45aa-bdc7-5639669a4970 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.639231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f728fe9-4d9f-45aa-bdc7-5639669a4970 admin admin] PATH_INFO: `/v3/projects/4c5b307771994b22848c3290c5496746/users/891b11b732c94e189752db093ef4a5db/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.648605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f728fe9-4d9f-45aa-bdc7-5639669a4970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5b307771994b22848c3290c5496746, user_id=891b11b732c94e189752db093ef4a5db, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:52.657807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f728fe9-4d9f-45aa-bdc7-5639669a4970 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:52.675418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:52.676630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.676926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.677423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.715948 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2160/4319] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:03:52 2022] PUT /identity/v3/projects/4c5b307771994b22848c3290c5496746/users/891b11b732c94e189752db093ef4a5db/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:03:52.730700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c764-cd9a-42cd-9749-970f0662292d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:52.731053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c764-cd9a-42cd-9749-970f0662292d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:52.731282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c764-cd9a-42cd-9749-970f0662292d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:52.780468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:52.822767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:52.826409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:52.841254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4619c764-cd9a-42cd-9749-970f0662292d None None] MFA Rules not processed for user `891b11b732c94e189752db093ef4a5db`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:03:52.857202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4619c764-cd9a-42cd-9749-970f0662292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:52.867308 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2160/4320] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:03:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 21:03:53.094535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:53.109556 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2161/4321] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:03:53.277596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacda8f0-8a97-4405-aee6-34c1079a4ebf service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:53.295326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891b11b732c94e189752db093ef4a5db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5b307771994b22848c3290c5496746', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:53.295958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:53.296286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:53.296572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:53.302472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:53.303566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:53.309304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:53.366856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c764-cd9a-42cd-9749-970f0662292d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:53.375647 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2161/4322] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:53.378927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c764-cd9a-42cd-9749-970f0662292d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:53.394021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891b11b732c94e189752db093ef4a5db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5b307771994b22848c3290c5496746', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:53.397998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:53.397998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:53.397998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:53.401080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:53.405887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:53.530983 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2162/4323] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:03:53 2022] POST /identity/v3/projects => generated 348 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:03:53.534567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:53.557298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3159da9-e586-4a9c-b472-b6be03f8eaf5 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:53.860634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891b11b732c94e189752db093ef4a5db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5b307771994b22848c3290c5496746', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:53.861402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:53.862872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:53.865465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:53.867556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:03:53.868815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:53.888670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:53.920379 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2162/4324] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:03:54.104784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:54.108393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:54.302280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:03:54.369087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a120ec-b14b-4d2f-94e3-fc267ace0fc9 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:54.379985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162a4b8-0307-4b1c-9998-b9caf61720a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:54.381138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a4b8-0307-4b1c-9998-b9caf61720a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:54.381440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a4b8-0307-4b1c-9998-b9caf61720a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:54.384866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a4b8-0307-4b1c-9998-b9caf61720a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:54.415568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162a4b8-0307-4b1c-9998-b9caf61720a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:54.419712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162a4b8-0307-4b1c-9998-b9caf61720a2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:54.446637 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2163/4325] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:58.062819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:03:58.069627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:58.156720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee8388-807e-499e-aba7-4aee5e955835 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:03:58.184209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7e973-f610-4592-bd1d-492931f7cf66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:03:58.187212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7e973-f610-4592-bd1d-492931f7cf66 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:58.189928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7e973-f610-4592-bd1d-492931f7cf66 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:58.190445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7e973-f610-4592-bd1d-492931f7cf66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:03:58.198087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-44a7e973-f610-4592-bd1d-492931f7cf66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:03:58.245220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7e973-f610-4592-bd1d-492931f7cf66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:03:58.249954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7e973-f610-4592-bd1d-492931f7cf66 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:03:58.329101 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2163/4326] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:03:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:03:59.898661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:03:59.900344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:03:59.902383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:00.019552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] MFA Rules not processed for user `3561f4fb18f84b899f68c5fcf897ca12`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:00.023208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:00.279828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2164/4327] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:03:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:00.292017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abbcce-ba3b-49d5-b9b6-d20e5852b034 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:00.293328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abbcce-ba3b-49d5-b9b6-d20e5852b034 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:00.293669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abbcce-ba3b-49d5-b9b6-d20e5852b034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:00.428856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-19abbcce-ba3b-49d5-b9b6-d20e5852b034 None None] MFA Rules not processed for user `3561f4fb18f84b899f68c5fcf897ca12`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:00.441721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-19abbcce-ba3b-49d5-b9b6-d20e5852b034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:00.519907 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2164/4328] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:00.522856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:00.525465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:00.729571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084236d3-5080-41b3-89a9-a3a4a0a18296 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:00.749041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20143931-9aee-4a5e-ad46-95032fd15f56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:00.750099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143931-9aee-4a5e-ad46-95032fd15f56 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:00.750859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143931-9aee-4a5e-ad46-95032fd15f56 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:00.751327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143931-9aee-4a5e-ad46-95032fd15f56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:00.778757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143931-9aee-4a5e-ad46-95032fd15f56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:00.783635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143931-9aee-4a5e-ad46-95032fd15f56 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:00.830733 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2165/4329] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:01.431859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a972c-99f1-44c7-a104-03772f37ea49 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:01.432409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a972c-99f1-44c7-a104-03772f37ea49 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:01.432519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a972c-99f1-44c7-a104-03772f37ea49 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:01.434870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2165/4330] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:04:01.446388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7b48d-d551-4f6c-82b3-d3a2042bafbb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:01.447382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7b48d-d551-4f6c-82b3-d3a2042bafbb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:01.447891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7b48d-d551-4f6c-82b3-d3a2042bafbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:01.546400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-21c7b48d-d551-4f6c-82b3-d3a2042bafbb None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:01.548112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21c7b48d-d551-4f6c-82b3-d3a2042bafbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:01.563343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2166/4331] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:04:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:01.821390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a972c-99f1-44c7-a104-03772f37ea49 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:01.858712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a972c-99f1-44c7-a104-03772f37ea49 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:01.888780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:01.889986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:01.890845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:01.891358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:01.941731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:01.941731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:01.965580 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2166/4332] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:03.826357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7b48d-d551-4f6c-82b3-d3a2042bafbb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:03.851793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7b48d-d551-4f6c-82b3-d3a2042bafbb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:03.870738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:03.871767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:03.872048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:03.872316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:03.880122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:03.946889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:03.948914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:03.988644 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2167/4333] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:05.488668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:05.517624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef6104-8536-4f01-9b39-dbedbb3200ae service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:05.543121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '891b11b732c94e189752db093ef4a5db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5b307771994b22848c3290c5496746', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:05.543992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:05.550273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:05.550578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] PATH_INFO: `/v3/projects/211967d47a044909a801c38022235fb9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:05.553175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorizing `identity:delete_project(project_id=211967d47a044909a801c38022235fb9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:05.564088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:05.723462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:05.786897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '211967d47a044909a801c38022235fb9', 'request_id': 'req-14b88f87-c0a2-467c-bb7e-9ebda262c153'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:05.804506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '211967d47a044909a801c38022235fb9', 'request_id': 'req-14b88f87-c0a2-467c-bb7e-9ebda262c153'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:05.830917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '211967d47a044909a801c38022235fb9', 'request_id': 'req-14b88f87-c0a2-467c-bb7e-9ebda262c153'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:05.864164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] The token cache is being invalidate because project 211967d47a044909a801c38022235fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:05.864592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:05.865357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:05.867093 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2167/4334] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:05 2022] DELETE /identity/v3/projects/211967d47a044909a801c38022235fb9 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:05.872658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:05.944352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:06.080114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:06.081880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:06.227602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c535f2-eda8-4f65-9451-f6b7623b9212 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:06.441766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:06.442664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:06.442997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:06.446658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] PATH_INFO: `/v3/users/f230e2825abf4997a2f8bdf470a7adfb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:06.452224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f230e2825abf4997a2f8bdf470a7adfb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:06.459183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:06.477162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b88f87-c0a2-467c-bb7e-9ebda262c153 tempest-QuotasNegativeTest-743241025 tempest-QuotasNegativeTest-743241025-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:06.509815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916a06f-aeb9-4b05-b22e-fc53cb9d445d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:06.510636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916a06f-aeb9-4b05-b22e-fc53cb9d445d service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:06.511438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916a06f-aeb9-4b05-b22e-fc53cb9d445d service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:06.512045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916a06f-aeb9-4b05-b22e-fc53cb9d445d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:06.521966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0916a06f-aeb9-4b05-b22e-fc53cb9d445d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:06.635951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916a06f-aeb9-4b05-b22e-fc53cb9d445d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:06.638661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f230e2825abf4997a2f8bdf470a7adfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:06.639550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916a06f-aeb9-4b05-b22e-fc53cb9d445d service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:06.678222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f230e2825abf4997a2f8bdf470a7adfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:06.688532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f230e2825abf4997a2f8bdf470a7adfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:06.711104 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2168/4335] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:04:06.719066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f230e2825abf4997a2f8bdf470a7adfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:06.723099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f230e2825abf4997a2f8bdf470a7adfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:06.811069 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2168/4336] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:05 2022] DELETE /identity/v3/users/f230e2825abf4997a2f8bdf470a7adfb => generated 0 bytes in 938 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:04:06.815677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:06.815960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:06.816525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:06.923375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:06.931977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:07.236242 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2169/4337] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:07.379608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:07.397308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:07.516461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:07.525802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9e6e6-9442-497c-abb3-b3a94b5e65c5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:07.532413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:07.534127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:07.534562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:07.535109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:07.550675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:07.552917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:07.568694 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2169/4338] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:07.880832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:07.883568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:08.004953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:08.100733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf2bc5-dc08-4b3b-8064-468db9dc0109 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:08.110023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:08.111015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:08.111502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:08.111991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:08.114178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:08.131971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:08.133630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:08.149166 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2170/4339] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:08.998492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:09.000035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:09.020578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed096f1-132a-461d-a851-c297f8c1e157 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:09.031860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:09.032640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:09.033039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:09.033535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] PATH_INFO: `/v3/projects/958ad0fded7945baa0d98bd7f037a3a9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:09.035531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=958ad0fded7945baa0d98bd7f037a3a9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:09.038102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:09.163202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958ad0fded7945baa0d98bd7f037a3a9', 'request_id': 'req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.178605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958ad0fded7945baa0d98bd7f037a3a9', 'request_id': 'req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.200002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958ad0fded7945baa0d98bd7f037a3a9', 'request_id': 'req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.213914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] The token cache is being invalidate because project 958ad0fded7945baa0d98bd7f037a3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:09.214602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.215651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.217596 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2170/4340] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:08 2022] DELETE /identity/v3/projects/958ad0fded7945baa0d98bd7f037a3a9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:09.221736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:09.237629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d910df8-eaff-41d7-a4f6-b5d28edbb651 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:09.308328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:09.308924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:09.309207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:09.309483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] PATH_INFO: `/v3/users/891b11b732c94e189752db093ef4a5db` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:09.310983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=891b11b732c94e189752db093ef4a5db)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:09.315164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:09.376589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891b11b732c94e189752db093ef4a5db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.394809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891b11b732c94e189752db093ef4a5db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.405155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891b11b732c94e189752db093ef4a5db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.418015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891b11b732c94e189752db093ef4a5db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.418992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891b11b732c94e189752db093ef4a5db'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.427046 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2171/4341] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:09 2022] DELETE /identity/v3/users/891b11b732c94e189752db093ef4a5db => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:09.580918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:09.583292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:09.602737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7c3ee-4bc0-4f2a-8ce2-8b9bc5e761aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:09.708117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:09.709098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:09.709520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:09.710029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] PATH_INFO: `/v3/users/b638a9e7f0934e47a3883be65e42d558` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:09.711389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b638a9e7f0934e47a3883be65e42d558)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:09.713516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:09.796041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b638a9e7f0934e47a3883be65e42d558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.824445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b638a9e7f0934e47a3883be65e42d558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.826061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b638a9e7f0934e47a3883be65e42d558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.848928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b638a9e7f0934e47a3883be65e42d558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.849906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b638a9e7f0934e47a3883be65e42d558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:09.862106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:09.880815 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2171/4342] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:09 2022] DELETE /identity/v3/users/b638a9e7f0934e47a3883be65e42d558 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:10.897976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:10.900627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:10.904441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:10.905266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:10.934233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7cc32-32dd-4b33-be2d-d337dded60e4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.036100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852def2-f674-48f6-8351-4d31c6c5943c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.061896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.063079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.063578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.064682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] PATH_INFO: `/v3/projects/4c5b307771994b22848c3290c5496746` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.066154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5b307771994b22848c3290c5496746)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.069062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.076501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:11.093202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.094792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.095386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.096150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] PATH_INFO: `/v3/projects/75ff64ac8631450bb5e2d269118982a8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.098224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ff64ac8631450bb5e2d269118982a8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.101168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.170357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ff64ac8631450bb5e2d269118982a8', 'request_id': 'req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.187914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ff64ac8631450bb5e2d269118982a8', 'request_id': 'req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.202983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ff64ac8631450bb5e2d269118982a8', 'request_id': 'req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.210478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5b307771994b22848c3290c5496746', 'request_id': 'req-b07594a8-5a98-45ac-87ec-0d01f0629bb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.222031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] The token cache is being invalidate because project 75ff64ac8631450bb5e2d269118982a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:11.222593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.223588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5b307771994b22848c3290c5496746', 'request_id': 'req-b07594a8-5a98-45ac-87ec-0d01f0629bb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.224903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b0d4ce8-06ab-4931-a6b6-69bb20ee13bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.231872 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2172/4343] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:10 2022] DELETE /identity/v3/projects/75ff64ac8631450bb5e2d269118982a8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:11.234086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863296f-b1ae-481a-82e1-efc0aa07a0a0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.235484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863296f-b1ae-481a-82e1-efc0aa07a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.235933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863296f-b1ae-481a-82e1-efc0aa07a0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.247480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5b307771994b22848c3290c5496746', 'request_id': 'req-b07594a8-5a98-45ac-87ec-0d01f0629bb8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.271386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a863296f-b1ae-481a-82e1-efc0aa07a0a0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:11.272676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] The token cache is being invalidate because project 4c5b307771994b22848c3290c5496746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:11.272988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.273810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b07594a8-5a98-45ac-87ec-0d01f0629bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:11.275861 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2172/4344] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:10 2022] DELETE /identity/v3/projects/4c5b307771994b22848c3290c5496746 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:11.278666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a863296f-b1ae-481a-82e1-efc0aa07a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:11.281167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392dd3d-5672-4d2a-b37c-8f6a0874fe39 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.281457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392dd3d-5672-4d2a-b37c-8f6a0874fe39 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.282189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392dd3d-5672-4d2a-b37c-8f6a0874fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.317427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c392dd3d-5672-4d2a-b37c-8f6a0874fe39 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:11.319389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c392dd3d-5672-4d2a-b37c-8f6a0874fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:11.377060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2173/4345] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:11.380833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b949-7e3a-466a-a731-6a745c4e2f06 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.381196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b949-7e3a-466a-a731-6a745c4e2f06 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.383670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b949-7e3a-466a-a731-6a745c4e2f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.411242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d672b949-7e3a-466a-a731-6a745c4e2f06 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:11.412596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d672b949-7e3a-466a-a731-6a745c4e2f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:11.422706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2174/4346] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:11.427052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672b949-7e3a-466a-a731-6a745c4e2f06 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.433569 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2173/4347] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:11.451254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672b949-7e3a-466a-a731-6a745c4e2f06 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.477220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cc2e0-fa68-4c5a-8806-b454dbdfffc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.478049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cc2e0-fa68-4c5a-8806-b454dbdfffc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.478581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cc2e0-fa68-4c5a-8806-b454dbdfffc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.478855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cc2e0-fa68-4c5a-8806-b454dbdfffc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.481578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cc2e0-fa68-4c5a-8806-b454dbdfffc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:11.481934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cc2e0-fa68-4c5a-8806-b454dbdfffc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.483812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cc2e0-fa68-4c5a-8806-b454dbdfffc0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.503399 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2175/4348] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:11.508197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392dd3d-5672-4d2a-b37c-8f6a0874fe39 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.519223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392dd3d-5672-4d2a-b37c-8f6a0874fe39 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.524394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.525119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.525465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.525812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.526579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.527798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37052-690e-468d-b62a-0f1a045f5d38 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.528088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37052-690e-468d-b62a-0f1a045f5d38 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.528365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37052-690e-468d-b62a-0f1a045f5d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.529073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.563190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:11.566600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bdc37052-690e-468d-b62a-0f1a045f5d38 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:11.567993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bdc37052-690e-468d-b62a-0f1a045f5d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:11.579565 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2174/4349] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/projects => generated 395 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:11.580232 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2176/4350] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:11.580977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.584897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc37052-690e-468d-b62a-0f1a045f5d38 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.607314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc37052-690e-468d-b62a-0f1a045f5d38 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.680328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:11.688451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be9034-57d6-4062-88e6-d4c756ee484a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.694224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.695292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.695983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.696423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.696729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.697164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.697519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.697902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.698486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:11.698633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.699137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.701291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.704085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.716161 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2177/4351] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:11.720430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.731936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c85d6-f297-4365-8571-ae0e1c0a9950 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.737634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.738287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.738571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.738853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.739493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.741413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.742215 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2175/4352] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:11.746263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.757619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6a67-ecc5-4730-804e-6b3fe6885f36 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.763544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.764370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.764975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.765444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.766573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.768597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.785653 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2178/4353] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/projects => generated 357 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:11.786940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.791660 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2176/4354] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:11.794308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.810063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f13655-a84d-4e15-b855-827d453bb751 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.875859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366cf3b-4dbc-4237-a5f7-904f94d5db40 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.887047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.887961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.888313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.888651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.889350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.891438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.925698 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2179/4355] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:11 2022] POST /identity/v3/users => generated 279 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:11.930340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.930537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.931142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.931435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.931705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.932350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.934848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.944965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e0b0-9a5e-4152-b6c2-dd9f29194de2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:11.952158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:11.953037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:11.953317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:11.953764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:11.954678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:11.956566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:11.960818 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2177/4356] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:11.978651 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2180/4357] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:11.982853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.984178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:11.996200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3985d24-7af9-462d-b6c1-f33a4d6a91d8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.003316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793905db-375d-4a10-9edf-00eec8e57235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.005389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793905db-375d-4a10-9edf-00eec8e57235 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.006216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793905db-375d-4a10-9edf-00eec8e57235 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.006817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793905db-375d-4a10-9edf-00eec8e57235 admin admin] PATH_INFO: `/v3/projects/449ab68ad6e7456dbf9d0ba3a16e2754/users/5cc5d837f74b40118e01a4e53ff194ca/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.016699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a67e7-3a3e-464a-9dba-0b36b0636aac admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.024081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.024716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.025061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.025347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.027341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.029401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:12.038088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793905db-375d-4a10-9edf-00eec8e57235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449ab68ad6e7456dbf9d0ba3a16e2754, user_id=5cc5d837f74b40118e01a4e53ff194ca, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.042169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793905db-375d-4a10-9edf-00eec8e57235 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:12.044397 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2181/4358] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:12.050222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:12.069869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2178/4359] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:11 2022] PUT /identity/v3/projects/449ab68ad6e7456dbf9d0ba3a16e2754/users/5cc5d837f74b40118e01a4e53ff194ca/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:12.077382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2813d-e7dc-45b7-a528-f813c167b1bc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.081543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718891b-a939-4e3a-ad0e-e22375044cef None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.081867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718891b-a939-4e3a-ad0e-e22375044cef None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.082223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718891b-a939-4e3a-ad0e-e22375044cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.148001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a718891b-a939-4e3a-ad0e-e22375044cef None None] MFA Rules not processed for user `5cc5d837f74b40118e01a4e53ff194ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:12.150117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a718891b-a939-4e3a-ad0e-e22375044cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.225335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.226094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.226965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.227254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] PATH_INFO: `/v3/projects/9179d8474dab41ee8e49805864748751/users/01425d028fe145d3a6f213647ceb6518/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.241230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:12.259118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9179d8474dab41ee8e49805864748751, user_id=01425d028fe145d3a6f213647ceb6518, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.266611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:12.281309 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2179/4360] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:12.293438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dc3fa-3670-4f37-873f-e93d7611c3c7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.293731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dc3fa-3670-4f37-873f-e93d7611c3c7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.294073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dc3fa-3670-4f37-873f-e93d7611c3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.299485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2182/4361] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:12 2022] PUT /identity/v3/projects/9179d8474dab41ee8e49805864748751/users/01425d028fe145d3a6f213647ceb6518/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:12.307788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:12.309801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.320211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-879dc3fa-3670-4f37-873f-e93d7611c3c7 None None] MFA Rules not processed for user `5cc5d837f74b40118e01a4e53ff194ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:12.321997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-879dc3fa-3670-4f37-873f-e93d7611c3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.331085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:12.404334 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2180/4362] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:12.406292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f77d1-6334-4aa3-8e2d-06d18691653d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.406761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f77d1-6334-4aa3-8e2d-06d18691653d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.407736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f77d1-6334-4aa3-8e2d-06d18691653d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.410262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2404cec-c427-4e45-a0a2-e5456b16dec2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.418557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.419411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.419699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.419975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] PATH_INFO: `/v3/users/9d6faed7dfdb4655b0574edcb88fcd9e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.421265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d6faed7dfdb4655b0574edcb88fcd9e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.424522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:12.447590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1e4f77d1-6334-4aa3-8e2d-06d18691653d None None] MFA Rules not processed for user `01425d028fe145d3a6f213647ceb6518`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:12.449362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e4f77d1-6334-4aa3-8e2d-06d18691653d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.531441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d6faed7dfdb4655b0574edcb88fcd9e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:12.548982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d6faed7dfdb4655b0574edcb88fcd9e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:12.562450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d6faed7dfdb4655b0574edcb88fcd9e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:12.578530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d6faed7dfdb4655b0574edcb88fcd9e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:12.579462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d6faed7dfdb4655b0574edcb88fcd9e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:12.591555 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2183/4363] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:12 2022] DELETE /identity/v3/users/9d6faed7dfdb4655b0574edcb88fcd9e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:12.591555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:12.592476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.599038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2181/4364] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:12.603258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.603549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.603839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.635954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:12.637341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.727543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af0cb8-ab92-4fbd-a9ad-d2d03f16d8f9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.738253 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2182/4365] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:12.738925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:12.739684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.740411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.740987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.741741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.743254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.745985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.769494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe469be-c658-4bd9-a685-442a73d7bb26 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.781953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.782576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.782857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.783129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.783772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.784095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:12.785723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:12.806390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:12.824134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.825798 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2183/4366] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:12 2022] POST /identity/v3/projects => generated 355 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:12.826122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:12.827254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:12.829622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:12.838181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2184/4367] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:12 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:12.840219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:12.853813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d7fd9-aaa5-48f6-b3ac-e022da695903 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.965379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db846a2e-4299-4061-9924-7a3292145e42 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:12.972505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0b042-ae53-4787-be92-68385131ca45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.973467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b042-ae53-4787-be92-68385131ca45 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.974083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b042-ae53-4787-be92-68385131ca45 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.975154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b042-ae53-4787-be92-68385131ca45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.980772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:12.981433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:12.982324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:12.982324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:12.982760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.984896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:12.992738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0b042-ae53-4787-be92-68385131ca45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:12.995391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0b042-ae53-4787-be92-68385131ca45 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.016645 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2184/4368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:13.017610 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2185/4369] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:12 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:13.019053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b9f85-fe76-4750-a1fe-0e5d3954dafc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.019649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b9f85-fe76-4750-a1fe-0e5d3954dafc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.020118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b9f85-fe76-4750-a1fe-0e5d3954dafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.021197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.039420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fd553-d3e1-4a9a-9710-76255a5521a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.048737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.049719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.050272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.050762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.051613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.052971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3c3b9f85-fe76-4750-a1fe-0e5d3954dafc None None] MFA Rules not processed for user `5cc5d837f74b40118e01a4e53ff194ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:13.054155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.054458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3c3b9f85-fe76-4750-a1fe-0e5d3954dafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.068563 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2186/4370] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:13.075317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.095327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dd8d-daaa-4f77-9509-d4ff591e101a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.103084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb687e93-a163-4e0c-8775-67f68ca84104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.103835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb687e93-a163-4e0c-8775-67f68ca84104 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.104057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb687e93-a163-4e0c-8775-67f68ca84104 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.108482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb687e93-a163-4e0c-8775-67f68ca84104 admin admin] PATH_INFO: `/v3/projects/80e6747d531b4cf48c4c3aec554bc13c/users/765521f6610d4e21826b667976cf6b53/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.151224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb687e93-a163-4e0c-8775-67f68ca84104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e6747d531b4cf48c4c3aec554bc13c, user_id=765521f6610d4e21826b667976cf6b53, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.159561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb687e93-a163-4e0c-8775-67f68ca84104 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.177055 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2185/4371] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:13.193227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2187/4372] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:13 2022] PUT /identity/v3/projects/80e6747d531b4cf48c4c3aec554bc13c/users/765521f6610d4e21826b667976cf6b53/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:13.205733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b9f85-fe76-4750-a1fe-0e5d3954dafc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.210760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6035627-dc1a-4ed2-9ef9-8840d5f0f82c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.211167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6035627-dc1a-4ed2-9ef9-8840d5f0f82c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.211541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6035627-dc1a-4ed2-9ef9-8840d5f0f82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.227581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b9f85-fe76-4750-a1fe-0e5d3954dafc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.237650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b6035627-dc1a-4ed2-9ef9-8840d5f0f82c None None] MFA Rules not processed for user `5cc5d837f74b40118e01a4e53ff194ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:13.238992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b6035627-dc1a-4ed2-9ef9-8840d5f0f82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.308179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2188/4373] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:13.320380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.321019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.321669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.322035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.322662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.329474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.347048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dfbb0-0266-4bf0-9dce-2004740bdfdb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.348023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dfbb0-0266-4bf0-9dce-2004740bdfdb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.348378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dfbb0-0266-4bf0-9dce-2004740bdfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.349996 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2186/4374] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:13.351808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.368462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412008a9-d8b4-49a6-b019-1b35b82c0f0a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.374565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-859dfbb0-0266-4bf0-9dce-2004740bdfdb None None] MFA Rules not processed for user `5cc5d837f74b40118e01a4e53ff194ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:13.374737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb2376-cd6b-4a34-960c-172b15614dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.375343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb2376-cd6b-4a34-960c-172b15614dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.375631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb2376-cd6b-4a34-960c-172b15614dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.375840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-859dfbb0-0266-4bf0-9dce-2004740bdfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.375951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb2376-cd6b-4a34-960c-172b15614dd0 admin admin] PATH_INFO: `/v3/projects/80e6747d531b4cf48c4c3aec554bc13c/users/765521f6610d4e21826b667976cf6b53/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.378666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb2376-cd6b-4a34-960c-172b15614dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e6747d531b4cf48c4c3aec554bc13c, user_id=765521f6610d4e21826b667976cf6b53, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.381913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb2376-cd6b-4a34-960c-172b15614dd0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.385890 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2189/4375] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:13.396381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2187/4376] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:13 2022] PUT /identity/v3/projects/80e6747d531b4cf48c4c3aec554bc13c/users/765521f6610d4e21826b667976cf6b53/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:13.408620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be149f0-c484-432f-8e6a-7f0958caadfb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.408620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be149f0-c484-432f-8e6a-7f0958caadfb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.408620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be149f0-c484-432f-8e6a-7f0958caadfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.423222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.423518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.423849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.442260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8be149f0-c484-432f-8e6a-7f0958caadfb None None] MFA Rules not processed for user `765521f6610d4e21826b667976cf6b53`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:13.443643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8be149f0-c484-432f-8e6a-7f0958caadfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.446090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] MFA Rules not processed for user `5cc5d837f74b40118e01a4e53ff194ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:13.447262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.459191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:13.471930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:13.533340 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2188/4377] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:13.534338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.536057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.577249 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2190/4378] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:13.581047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be149f0-c484-432f-8e6a-7f0958caadfb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.596678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be149f0-c484-432f-8e6a-7f0958caadfb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.606457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '765521f6610d4e21826b667976cf6b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e6747d531b4cf48c4c3aec554bc13c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.607174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.607522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.607865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.608789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:13.609164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.611083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.616549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf532ac7-497c-46a0-a83b-1df97b399bc4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.624169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.624169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.624169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.624169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] PATH_INFO: `/v3/projects/302c207c0fc84ed6ad230678ac644108` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.624761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=302c207c0fc84ed6ad230678ac644108)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.627104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.639239 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2191/4379] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:04:13.643628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.661144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f332-e6c1-4c7e-bbed-d80907956e15 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.666646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '765521f6610d4e21826b667976cf6b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e6747d531b4cf48c4c3aec554bc13c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.667320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.667608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.667905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.668504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.670592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.695823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302c207c0fc84ed6ad230678ac644108', 'request_id': 'req-4cd188da-be81-4e1e-8e53-ddb27e708139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.705268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302c207c0fc84ed6ad230678ac644108', 'request_id': 'req-4cd188da-be81-4e1e-8e53-ddb27e708139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.712155 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2192/4380] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:13 2022] POST /identity/v3/projects => generated 350 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:13.714906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302c207c0fc84ed6ad230678ac644108', 'request_id': 'req-4cd188da-be81-4e1e-8e53-ddb27e708139'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.717301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.747192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] The token cache is being invalidate because project 302c207c0fc84ed6ad230678ac644108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:13.747488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.748169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.752727 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2189/4381] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:13 2022] DELETE /identity/v3/projects/302c207c0fc84ed6ad230678ac644108 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:13.752727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.752727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.798459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9363e15-d347-499b-9c13-91b22a275ffc tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.803286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '765521f6610d4e21826b667976cf6b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e6747d531b4cf48c4c3aec554bc13c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.803896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.804184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.804466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.805142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:13.805455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.807097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.819622 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2193/4382] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:13.819955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd188da-be81-4e1e-8e53-ddb27e708139 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.820617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.822206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.823867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:13.879532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.880173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.880480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.880756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] PATH_INFO: `/v3/users/5cc5d837f74b40118e01a4e53ff194ca` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.881953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc5d837f74b40118e01a4e53ff194ca)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.883878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.894780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225bdcd-6705-4739-8f1d-50ed0a63da19 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:13.899679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:13.900313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:13.900594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:13.900868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] PATH_INFO: `/v3/users/933b9d039f1a4fd3b3ffc01099887f8f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:13.902212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933b9d039f1a4fd3b3ffc01099887f8f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:13.904150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:13.926142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:13.946164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc5d837f74b40118e01a4e53ff194ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.953910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc5d837f74b40118e01a4e53ff194ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.954601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc5d837f74b40118e01a4e53ff194ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.963921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc5d837f74b40118e01a4e53ff194ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.963921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc5d837f74b40118e01a4e53ff194ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.970478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933b9d039f1a4fd3b3ffc01099887f8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.984099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933b9d039f1a4fd3b3ffc01099887f8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:13.989679 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2190/4383] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:13 2022] DELETE /identity/v3/users/5cc5d837f74b40118e01a4e53ff194ca => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:13.990364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:13.991785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:13.994226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933b9d039f1a4fd3b3ffc01099887f8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:14.002713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933b9d039f1a4fd3b3ffc01099887f8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:14.004007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-12dcaf45-8e1b-4ff8-8cfc-94a349e3e2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933b9d039f1a4fd3b3ffc01099887f8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:14.011591 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2194/4384] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:13 2022] DELETE /identity/v3/users/933b9d039f1a4fd3b3ffc01099887f8f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:14.012924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:14.013167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:14.013458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:14.035455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:14.036686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:14.079424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ca922-9db1-4ad0-9784-843b1786c104 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:14.114961 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2195/4385] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:14.134791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:14.138930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:14.144789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:14.166683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fafcad-620a-45d7-b281-c4e4cefaeca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:14.173848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fafcad-620a-45d7-b281-c4e4cefaeca6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:14.174707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fafcad-620a-45d7-b281-c4e4cefaeca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:14.175010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fafcad-620a-45d7-b281-c4e4cefaeca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:14.179768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-52fafcad-620a-45d7-b281-c4e4cefaeca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:14.218945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8090b9-97b8-4efb-a747-f9f6ff558cff None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:14.230074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:14.231225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:14.231750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:14.232251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:14.249784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:14.251377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:14.262080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fafcad-620a-45d7-b281-c4e4cefaeca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:14.263816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fafcad-620a-45d7-b281-c4e4cefaeca6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:14.271304 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2196/4386] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:14.275324 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2191/4387] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:15.131892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:15.134001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:15.157204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e0e53-009c-4278-9b37-e118fa0735dd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:15.163597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:15.164252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:15.164535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:15.164810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] PATH_INFO: `/v3/projects/d1c3701208984be8ad70e79602180ab2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:15.166060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c3701208984be8ad70e79602180ab2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:15.169037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:15.240490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:15.273420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c3701208984be8ad70e79602180ab2', 'request_id': 'req-db4c1542-5dc2-4705-bedb-b4d2eda9a236'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.296596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c3701208984be8ad70e79602180ab2', 'request_id': 'req-db4c1542-5dc2-4705-bedb-b4d2eda9a236'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.308044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c3701208984be8ad70e79602180ab2', 'request_id': 'req-db4c1542-5dc2-4705-bedb-b4d2eda9a236'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.330078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] The token cache is being invalidate because project d1c3701208984be8ad70e79602180ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:15.330541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.331393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db4c1542-5dc2-4705-bedb-b4d2eda9a236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.334040 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2197/4388] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:15 2022] DELETE /identity/v3/projects/d1c3701208984be8ad70e79602180ab2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:15.338329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:15.339391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:15.339391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:15.363634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:15.364866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:15.445242 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2192/4389] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:15.455744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba7ee5-3c8d-4fad-bed7-dcb8f564b239 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:15.456039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba7ee5-3c8d-4fad-bed7-dcb8f564b239 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:15.456368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba7ee5-3c8d-4fad-bed7-dcb8f564b239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:15.475441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:15.477024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:15.499478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-33ba7ee5-3c8d-4fad-bed7-dcb8f564b239 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:15.503842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-33ba7ee5-3c8d-4fad-bed7-dcb8f564b239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:15.519647 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2198/4390] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:15.526951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba7ee5-3c8d-4fad-bed7-dcb8f564b239 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:15.538806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba7ee5-3c8d-4fad-bed7-dcb8f564b239 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:15.546377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:15.547617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:15.547909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:15.548247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:15.549432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:15.549753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:15.565048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:15.582568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c52aa-388a-42ba-a39f-8e624cd8b08c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:15.589575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '765521f6610d4e21826b667976cf6b53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80e6747d531b4cf48c4c3aec554bc13c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:15.590453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:15.590746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:15.591027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] PATH_INFO: `/v3/projects/747e2f8ef93c4be7823b26f5603dd722` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:15.593500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorizing `identity:delete_project(project_id=747e2f8ef93c4be7823b26f5603dd722)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:15.596799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:15.631075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2199/4391] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:15.634125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:15.660256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:15.676622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747e2f8ef93c4be7823b26f5603dd722', 'request_id': 'req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.691424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4c30c-2b67-4a53-afab-a4c4610294bf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:15.692467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747e2f8ef93c4be7823b26f5603dd722', 'request_id': 'req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.710393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747e2f8ef93c4be7823b26f5603dd722', 'request_id': 'req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.760054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] The token cache is being invalidate because project 747e2f8ef93c4be7823b26f5603dd722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:15.761374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.763175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:15.769188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2193/4392] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:15 2022] DELETE /identity/v3/projects/747e2f8ef93c4be7823b26f5603dd722 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:04:15.806230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:15.808980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:15.809898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:15.810386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:15.814417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:15.817088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:15.890338 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2200/4393] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:15 2022] POST /identity/v3/projects => generated 365 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:15.891411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:15.893079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:15.914756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6e34b-8843-4ed7-8c69-a7cb9ff57c14 tempest-QuotasTest-330554422 tempest-QuotasTest-330554422-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.001674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.002660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.003015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.003400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.004128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.006293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.038890 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2194/4394] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:15 2022] POST /identity/v3/users => generated 283 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:16.042580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.042737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.044571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.056943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223e63-8146-43b6-b058-05721eed8bfd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.059307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797547e0-0553-496f-ba5c-305159d6f533 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.066058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.067303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.068109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.068795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.069241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.070131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.070557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.070976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.071912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] PATH_INFO: `/v3/projects/449ab68ad6e7456dbf9d0ba3a16e2754` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.072856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.074112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449ab68ad6e7456dbf9d0ba3a16e2754)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.076624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.086507 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2201/4395] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:16.092670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.123262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb083f-5354-40ff-a1e7-e8335b594b5a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.137500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.138231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.138590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.141054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.142345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.144209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.153554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449ab68ad6e7456dbf9d0ba3a16e2754', 'request_id': 'req-8713d5af-38e0-484b-9a9d-511382179823'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.164739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449ab68ad6e7456dbf9d0ba3a16e2754', 'request_id': 'req-8713d5af-38e0-484b-9a9d-511382179823'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.179804 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2202/4396] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:16.187025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.188201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449ab68ad6e7456dbf9d0ba3a16e2754', 'request_id': 'req-8713d5af-38e0-484b-9a9d-511382179823'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.207900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701377d-8fa4-44d8-abbe-bd641e9f0aed admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.208622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] The token cache is being invalidate because project 449ab68ad6e7456dbf9d0ba3a16e2754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:16.208985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.209675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8713d5af-38e0-484b-9a9d-511382179823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.211750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2195/4397] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:16 2022] DELETE /identity/v3/projects/449ab68ad6e7456dbf9d0ba3a16e2754 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:04:16.218381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9d1f8-10a0-4e28-b567-c47cac73f872 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.218896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9d1f8-10a0-4e28-b567-c47cac73f872 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.219434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9d1f8-10a0-4e28-b567-c47cac73f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.249966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-10b9d1f8-10a0-4e28-b567-c47cac73f872 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:16.251397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10b9d1f8-10a0-4e28-b567-c47cac73f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.315077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:16.322711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:16.327689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3ed8b-8df3-44f0-a757-db3c3ca0edea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.328728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3ed8b-8df3-44f0-a757-db3c3ca0edea admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.329081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3ed8b-8df3-44f0-a757-db3c3ca0edea admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.331453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3ed8b-8df3-44f0-a757-db3c3ca0edea admin admin] PATH_INFO: `/v3/projects/cfca1dc59972463392479e36d625a8c5/users/e8ef99af9514498aa42557c245b83e89/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.351751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3ed8b-8df3-44f0-a757-db3c3ca0edea admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfca1dc59972463392479e36d625a8c5, user_id=e8ef99af9514498aa42557c245b83e89, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.354501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3ed8b-8df3-44f0-a757-db3c3ca0edea admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.387770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2203/4398] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:16 2022] PUT /identity/v3/projects/cfca1dc59972463392479e36d625a8c5/users/e8ef99af9514498aa42557c245b83e89/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:16.389781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.390162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.390577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.407357 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2196/4399] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:16.408493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.408493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.410951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.442481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] MFA Rules not processed for user `e8ef99af9514498aa42557c245b83e89`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:16.445357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.456498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:16.459571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.631602 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2204/4400] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:16.632874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.636189 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2197/4401] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:16.637262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.637565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.643496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.658441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dfc5e4-4335-445b-9af9-48b4e618efdf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.658721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28734d-1435-47c2-877e-465815ebd187 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.662755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.663507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.663668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.663999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.664249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.664457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] PATH_INFO: `/v3/users/01425d028fe145d3a6f213647ceb6518` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.665267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.665650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.666178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01425d028fe145d3a6f213647ceb6518)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.666425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.668864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.669040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.731200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2198/4402] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:16 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:16.733211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.737301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01425d028fe145d3a6f213647ceb6518'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.766206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01425d028fe145d3a6f213647ceb6518'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.774575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01425d028fe145d3a6f213647ceb6518'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.788248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01425d028fe145d3a6f213647ceb6518'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.789139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01425d028fe145d3a6f213647ceb6518'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:16.799377 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2205/4403] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:16 2022] DELETE /identity/v3/users/01425d028fe145d3a6f213647ceb6518 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:16.800378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.802429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.826895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e65a-2d09-4ccd-8466-20cb197c61f1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.863629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:16.892262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160bbe9-afca-434a-9615-403a6c1f86d2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.897644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.898454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.898689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.898970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.899604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.901800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.904281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886b2f2-13dc-4871-97da-559e6030eb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.904913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b2f2-13dc-4871-97da-559e6030eb5e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.905192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b2f2-13dc-4871-97da-559e6030eb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.905463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b2f2-13dc-4871-97da-559e6030eb5e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.906145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886b2f2-13dc-4871-97da-559e6030eb5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:16.906469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886b2f2-13dc-4871-97da-559e6030eb5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.908304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886b2f2-13dc-4871-97da-559e6030eb5e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.911671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:16.923232 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2199/4404] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:16.925421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.926022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.926418 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2206/4405] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:16 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:16.926562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.938940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.951104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57107228-1c29-44d0-a64a-c378a8e5b2d4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.951570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:16.952999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.955890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.956497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.956799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.957074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.957717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:16.959297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:16.965402 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2200/4406] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:16.965402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.971530 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2207/4407] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:16.972321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:16.974313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.982507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:16.988333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23d12c-ef01-44c2-a2a7-a5630055ee4b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:16.996322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:16.996943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:16.997222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:16.997494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:16.998165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.000622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.036874 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2201/4408] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:16 2022] POST /identity/v3/projects => generated 393 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:17.037865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.047426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3eed1c-76e2-4eb4-9af5-3078e0ac9d15 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.110392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.110773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39332d01-f5d6-43b7-bff5-bbad0d6de344 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.111115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.111470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.111812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.116899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.118020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.118020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.118318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] PATH_INFO: `/v3/projects/70750c09ca444dd6ad91614d2e21c7ab/users/58c92a40baab4fdfa54449ef78ce18b0/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.123806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.125525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.140710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=70750c09ca444dd6ad91614d2e21c7ab, user_id=58c92a40baab4fdfa54449ef78ce18b0, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.141525 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2208/4409] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:04:17.142573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.144596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.159393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5647c84-a394-4719-9473-44543dabc6bd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.165666 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2202/4410] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:17 2022] PUT /identity/v3/projects/70750c09ca444dd6ad91614d2e21c7ab/users/58c92a40baab4fdfa54449ef78ce18b0/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:17.170798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.282260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.283023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.283536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.283900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.284846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.287709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.305637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da80554-b01f-41e4-b0cc-c9b546295dfb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.314560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.318393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.318844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.319438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.320473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.322704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.345089 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2203/4411] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:17.348966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.360228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2209/4412] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:17 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:17.365202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.366089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988332f4-825d-474e-bc43-169a995d4f30 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.375221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f4b1c-d372-4dc7-86bf-7c50ac338427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.376490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f4b1c-d372-4dc7-86bf-7c50ac338427 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.377048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f4b1c-d372-4dc7-86bf-7c50ac338427 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.377555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f4b1c-d372-4dc7-86bf-7c50ac338427 admin admin] PATH_INFO: `/v3/projects/70750c09ca444dd6ad91614d2e21c7ab/users/58c92a40baab4fdfa54449ef78ce18b0/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.381506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f4b1c-d372-4dc7-86bf-7c50ac338427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70750c09ca444dd6ad91614d2e21c7ab, user_id=58c92a40baab4fdfa54449ef78ce18b0, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.382359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b22b5-4fde-4a89-8ab0-4003bba9709a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.386654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f4b1c-d372-4dc7-86bf-7c50ac338427 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.389487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.390388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.391615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.392087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.392923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.394990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.397644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:17.411819 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2210/4413] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:17.412744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.421893 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2204/4414] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:17 2022] PUT /identity/v3/projects/70750c09ca444dd6ad91614d2e21c7ab/users/58c92a40baab4fdfa54449ef78ce18b0/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:17.433887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.434181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.434458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.440440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783131a8-b9cc-41d5-ac93-954123e830a1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.472146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] MFA Rules not processed for user `58c92a40baab4fdfa54449ef78ce18b0`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:17.473508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:17.500936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:17.533135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.533776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.534097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.534376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.535086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.536620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.549880 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2211/4415] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:17.552262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.564448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cb03c-b812-44ab-b5fa-51c932e6ddae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.570475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95226661-e3cc-4de9-b8cc-9129dd6095ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.571173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95226661-e3cc-4de9-b8cc-9129dd6095ec admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.571463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95226661-e3cc-4de9-b8cc-9129dd6095ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.571765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95226661-e3cc-4de9-b8cc-9129dd6095ec admin admin] PATH_INFO: `/v3/projects/e9df0b572a1844bda38471874d2afe92/users/bbbfb72cf13347ce901cad42460e7dce/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.572004 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2205/4416] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:17.583195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.585125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:17.590400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95226661-e3cc-4de9-b8cc-9129dd6095ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9df0b572a1844bda38471874d2afe92, user_id=bbbfb72cf13347ce901cad42460e7dce, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.593085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95226661-e3cc-4de9-b8cc-9129dd6095ec admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.611324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2212/4417] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:17 2022] PUT /identity/v3/projects/e9df0b572a1844bda38471874d2afe92/users/bbbfb72cf13347ce901cad42460e7dce/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:17.616588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a12cd-0bcd-43a1-a33d-8347f0e494ad None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.616949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a12cd-0bcd-43a1-a33d-8347f0e494ad None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.617474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a12cd-0bcd-43a1-a33d-8347f0e494ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.669342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-817a12cd-0bcd-43a1-a33d-8347f0e494ad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:17.672252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-817a12cd-0bcd-43a1-a33d-8347f0e494ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:17.692214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc3a57-4114-4c38-95e4-97f9fe04bc45 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.782125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.782769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.783056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.783332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.823424 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2213/4418] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:17.834607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a12cd-0bcd-43a1-a33d-8347f0e494ad None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.851201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a12cd-0bcd-43a1-a33d-8347f0e494ad None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.860192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.860822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.861107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.861383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.872877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.874641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.887880 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2214/4419] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:17.889091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:17.890520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:17.925732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.927441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:17.929486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267bad-07e8-4e44-9a58-c5b4cfd38711 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:17.943507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:17.944424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:17.944582 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2206/4420] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:04:17.945088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:17.945556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] PATH_INFO: `/v3/projects/9179d8474dab41ee8e49805864748751` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:17.947890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9179d8474dab41ee8e49805864748751)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:17.950114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:18.031310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9179d8474dab41ee8e49805864748751', 'request_id': 'req-25c85b3d-7824-4408-9527-83aabc9ae4cb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.034466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:18.039793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9179d8474dab41ee8e49805864748751', 'request_id': 'req-25c85b3d-7824-4408-9527-83aabc9ae4cb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.053895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9179d8474dab41ee8e49805864748751', 'request_id': 'req-25c85b3d-7824-4408-9527-83aabc9ae4cb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.073914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] The token cache is being invalidate because project 9179d8474dab41ee8e49805864748751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:18.074716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.075814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c85b3d-7824-4408-9527-83aabc9ae4cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.077898 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2215/4421] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:17 2022] DELETE /identity/v3/projects/9179d8474dab41ee8e49805864748751 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:18.082331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:18.095411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bfe14-30f5-4fa3-9d15-828619b88921 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:18.130745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:18.226967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:18.228616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:18.229717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:18.231074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] PATH_INFO: `/v3/users/765521f6610d4e21826b667976cf6b53` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:18.232570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=765521f6610d4e21826b667976cf6b53)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:18.234859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:18.338026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765521f6610d4e21826b667976cf6b53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.350748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765521f6610d4e21826b667976cf6b53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.352509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765521f6610d4e21826b667976cf6b53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.369492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765521f6610d4e21826b667976cf6b53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.371361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765521f6610d4e21826b667976cf6b53'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:18.415321 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2207/4422] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:18 2022] DELETE /identity/v3/users/765521f6610d4e21826b667976cf6b53 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:18.458079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d6099-67e9-4e59-af14-348b6c347576 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:18.458079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d6099-67e9-4e59-af14-348b6c347576 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:18.458079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d6099-67e9-4e59-af14-348b6c347576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:18.480824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-059d6099-67e9-4e59-af14-348b6c347576 None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:18.486523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-059d6099-67e9-4e59-af14-348b6c347576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:18.557900 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2216/4423] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:19.398382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:19.400988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:19.550717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a748f92-3558-444a-9add-7f046fb15431 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:19.558104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:19.559347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:19.560865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:19.561388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] PATH_INFO: `/v3/projects/80e6747d531b4cf48c4c3aec554bc13c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:19.565018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e6747d531b4cf48c4c3aec554bc13c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:19.568267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:19.664779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:19.689857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e6747d531b4cf48c4c3aec554bc13c', 'request_id': 'req-64597338-2161-46b1-97c5-478a2a3822bf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:19.726490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e6747d531b4cf48c4c3aec554bc13c', 'request_id': 'req-64597338-2161-46b1-97c5-478a2a3822bf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:19.748285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e6747d531b4cf48c4c3aec554bc13c', 'request_id': 'req-64597338-2161-46b1-97c5-478a2a3822bf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:19.767991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] The token cache is being invalidate because project 80e6747d531b4cf48c4c3aec554bc13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:19.768471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:19.771422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-64597338-2161-46b1-97c5-478a2a3822bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:19.774444 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2208/4424] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:19 2022] DELETE /identity/v3/projects/80e6747d531b4cf48c4c3aec554bc13c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:19.780930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf867df-a119-4864-8741-d26ac766bf6f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:19.781934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf867df-a119-4864-8741-d26ac766bf6f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:19.782436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf867df-a119-4864-8741-d26ac766bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:19.823460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-faf867df-a119-4864-8741-d26ac766bf6f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:19.824970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-faf867df-a119-4864-8741-d26ac766bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:19.976923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2217/4425] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:19.987114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b446-517d-4b5e-97a2-38e1ff9f7978 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:19.987482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b446-517d-4b5e-97a2-38e1ff9f7978 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:19.987829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b446-517d-4b5e-97a2-38e1ff9f7978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.038477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4275b446-517d-4b5e-97a2-38e1ff9f7978 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:20.040905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4275b446-517d-4b5e-97a2-38e1ff9f7978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:20.065587 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2209/4426] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:20.070045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf867df-a119-4864-8741-d26ac766bf6f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:20.085744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf867df-a119-4864-8741-d26ac766bf6f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:20.098393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:20.100861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:20.102034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:20.103822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.105379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:20.108425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:20.111999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:20.127566 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2218/4427] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:20.131800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275b446-517d-4b5e-97a2-38e1ff9f7978 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:20.145229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275b446-517d-4b5e-97a2-38e1ff9f7978 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:20.161983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:20.164121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:20.164427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:20.165730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.183543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:20.191430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:20.269506 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2210/4428] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:20 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:20.273694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:20.290513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde2884-aea6-4685-8018-23adff1b6855 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:20.412368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:20.413074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:20.413365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:20.413934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.414577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:20.419375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:20.457630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:20.458251 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2219/4429] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:20 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:20.475539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e8f59-b693-446c-bf5d-b7d884752fb1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:20.495669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:20.496301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:20.496587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:20.496941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.497594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:20.499435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:20.515164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:20.515583 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2211/4430] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:20.521698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:20.528093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6018-69b4-4c22-958e-594b9ccdb8ec admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:20.535569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184dbfe-ce4c-40bc-a9da-778a583c305d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:20.536906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184dbfe-ce4c-40bc-a9da-778a583c305d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:20.537180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184dbfe-ce4c-40bc-a9da-778a583c305d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:20.537727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184dbfe-ce4c-40bc-a9da-778a583c305d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.539000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184dbfe-ce4c-40bc-a9da-778a583c305d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:20.541225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184dbfe-ce4c-40bc-a9da-778a583c305d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:20.561890 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2220/4431] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:20.565925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:20.594561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e51faa-5a21-4b13-aa3b-7feb0a17b805 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:20.608106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:20.609719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:20.613054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:20.613628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] PATH_INFO: `/v3/projects/d81aa8c53d9c41238de5521448d9613f/users/4759bc2f66374825b80bd2a79da9a260/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.653023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81aa8c53d9c41238de5521448d9613f, user_id=4759bc2f66374825b80bd2a79da9a260, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:20.656270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:20.691865 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2212/4432] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:20 2022] PUT /identity/v3/projects/d81aa8c53d9c41238de5521448d9613f/users/4759bc2f66374825b80bd2a79da9a260/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:04:20.705257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:20.705730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:20.706391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:20.763233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] MFA Rules not processed for user `4759bc2f66374825b80bd2a79da9a260`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:20.764917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:20.863321 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2221/4433] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:20.876437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:20.878581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:20.993998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fced29-1f9c-43d4-86dc-015c183ce643 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:21.000664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:21.001271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:21.001551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:21.001849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:21.012391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:21.013885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:21.024699 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2213/4434] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:21.155520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:21.157034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:21.171553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8dcc-f873-4833-9128-60ff6ff04104 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:21.175867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481e952-d33f-424a-8f96-825750ce8ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:21.176552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e952-d33f-424a-8f96-825750ce8ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:21.176887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e952-d33f-424a-8f96-825750ce8ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:21.177229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481e952-d33f-424a-8f96-825750ce8ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:21.179399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5481e952-d33f-424a-8f96-825750ce8ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:21.245878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481e952-d33f-424a-8f96-825750ce8ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:21.247268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481e952-d33f-424a-8f96-825750ce8ee3 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:21.257200 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2222/4435] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:22.884223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:22.886577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:22.992504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe7617-7c89-4436-ac5e-0b94e0302f65 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:22.997321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:22.998083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:22.998354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:22.998557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] PATH_INFO: `/v3/users/e8ef99af9514498aa42557c245b83e89` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:23.000202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ef99af9514498aa42557c245b83e89)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:23.001570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:23.017818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:23.067632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ef99af9514498aa42557c245b83e89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:23.076672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ef99af9514498aa42557c245b83e89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:23.077391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ef99af9514498aa42557c245b83e89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:23.085182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ef99af9514498aa42557c245b83e89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:23.086450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ef99af9514498aa42557c245b83e89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:23.117780 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2214/4436] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:22 2022] DELETE /identity/v3/users/e8ef99af9514498aa42557c245b83e89 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:23.122321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:23.123238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:23.123538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:23.152665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:23.154511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:23.264621 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2223/4437] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:23.334475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:23.336045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:23.409137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c3afc-c676-42c5-89cd-136292005608 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:23.415574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:23.417141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:23.417440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:23.417728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:23.429594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:23.432505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:23.452704 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2215/4438] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:24.585386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:24.587465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:24.609741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af858eb-f633-4ceb-a9c6-0b50f76e8481 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:24.621004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:24.621939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:24.622392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:24.622833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] PATH_INFO: `/v3/projects/cfca1dc59972463392479e36d625a8c5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:24.626835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfca1dc59972463392479e36d625a8c5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:24.629036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:24.676802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:24.755923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfca1dc59972463392479e36d625a8c5', 'request_id': 'req-2b8c023e-ba43-4c2d-aa19-4eb73d540366'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:24.774053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfca1dc59972463392479e36d625a8c5', 'request_id': 'req-2b8c023e-ba43-4c2d-aa19-4eb73d540366'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:24.792582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfca1dc59972463392479e36d625a8c5', 'request_id': 'req-2b8c023e-ba43-4c2d-aa19-4eb73d540366'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:24.825889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] The token cache is being invalidate because project cfca1dc59972463392479e36d625a8c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:24.826198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:24.827911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:24.833180 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2224/4439] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:24 2022] DELETE /identity/v3/projects/cfca1dc59972463392479e36d625a8c5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:24.833847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:24.851498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858620c5-2288-4d96-8ed7-c8c89f16d63c service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:24.999780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:25.000444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:25.000730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:25.001010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] PATH_INFO: `/v3/users/58c92a40baab4fdfa54449ef78ce18b0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:25.002631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c92a40baab4fdfa54449ef78ce18b0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:25.005099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:25.107825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:25.133990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c92a40baab4fdfa54449ef78ce18b0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:25.152136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c92a40baab4fdfa54449ef78ce18b0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:25.154311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c92a40baab4fdfa54449ef78ce18b0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:25.191524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c92a40baab4fdfa54449ef78ce18b0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:25.192090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-66c2ba0a-c7f3-465a-bf1e-bd9706c1c98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c92a40baab4fdfa54449ef78ce18b0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:25.232986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2216/4440] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:24 2022] DELETE /identity/v3/users/58c92a40baab4fdfa54449ef78ce18b0 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:04:25.877843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:25.880086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:25.902348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c023e-ba43-4c2d-aa19-4eb73d540366 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:26.043993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:26.045057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.045635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.047589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] PATH_INFO: `/v3/projects/70750c09ca444dd6ad91614d2e21c7ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.054436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70750c09ca444dd6ad91614d2e21c7ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:26.056770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:26.158543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70750c09ca444dd6ad91614d2e21c7ab', 'request_id': 'req-6c22b5bc-5515-4b84-a108-d6731f583024'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:26.168599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70750c09ca444dd6ad91614d2e21c7ab', 'request_id': 'req-6c22b5bc-5515-4b84-a108-d6731f583024'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:26.187883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70750c09ca444dd6ad91614d2e21c7ab', 'request_id': 'req-6c22b5bc-5515-4b84-a108-d6731f583024'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:26.205874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] The token cache is being invalidate because project 70750c09ca444dd6ad91614d2e21c7ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:26.206328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:26.207142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6c22b5bc-5515-4b84-a108-d6731f583024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:26.209085 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2225/4441] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:25 2022] DELETE /identity/v3/projects/70750c09ca444dd6ad91614d2e21c7ab => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:26.214910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e6082-2520-4c26-ab2d-436a27e92aa3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.215454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e6082-2520-4c26-ab2d-436a27e92aa3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.215578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e6082-2520-4c26-ab2d-436a27e92aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.243861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-de9e6082-2520-4c26-ab2d-436a27e92aa3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:26.245913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de9e6082-2520-4c26-ab2d-436a27e92aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:26.329749 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2217/4442] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:26.337412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5d84b-1b3e-43c2-a7a2-dc6c447fee87 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.337996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5d84b-1b3e-43c2-a7a2-dc6c447fee87 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.339251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5d84b-1b3e-43c2-a7a2-dc6c447fee87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.367967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c8e5d84b-1b3e-43c2-a7a2-dc6c447fee87 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:26.369198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8e5d84b-1b3e-43c2-a7a2-dc6c447fee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:26.382049 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2226/4443] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:26.387174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e6082-2520-4c26-ab2d-436a27e92aa3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:26.406346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e6082-2520-4c26-ab2d-436a27e92aa3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:26.421056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:26.421686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.422221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.422374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.423017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:26.423553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:26.425120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:26.445886 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2218/4444] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:26.448407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5d84b-1b3e-43c2-a7a2-dc6c447fee87 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:26.468182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5d84b-1b3e-43c2-a7a2-dc6c447fee87 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:26.482178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:26.482807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.483088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.483363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.484010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:26.485976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:26.542534 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2227/4445] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:26 2022] POST /identity/v3/projects => generated 397 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:26.546869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:26.559002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01143f-3240-495e-aa9b-f6f795cc8795 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:26.675187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:26.677474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.678832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.679392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.680572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:26.686279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:26.731406 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2219/4446] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:26 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:26.732273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:26.739306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:26.744268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeaeb6-39ff-4e28-a98b-330a01b11497 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:26.752840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:26.753512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.753849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.754101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.754747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:26.758368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:26.774038 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2228/4447] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:26.775223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:26.789460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eae48d-0653-4b90-b36e-a2029b284da9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:26.797530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eabe1f-c5f7-4548-8e88-946d4adc78da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:26.798652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabe1f-c5f7-4548-8e88-946d4adc78da admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.799162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabe1f-c5f7-4548-8e88-946d4adc78da admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.799859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabe1f-c5f7-4548-8e88-946d4adc78da admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.800790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eabe1f-c5f7-4548-8e88-946d4adc78da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:26.803064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eabe1f-c5f7-4548-8e88-946d4adc78da admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:26.825885 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2220/4448] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:26.826708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:26.838891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b31e-3e03-4c6b-b67c-24670eb446ab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:26.846007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:26.847060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.847402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.848378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] PATH_INFO: `/v3/projects/a5e171c5fbf5475090398de10d401d1f/users/7ed630c9690f4dfca2f13e7afc9bb9de/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.869095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e171c5fbf5475090398de10d401d1f, user_id=7ed630c9690f4dfca2f13e7afc9bb9de, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:26.873485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:26.913713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2229/4449] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:26 2022] PUT /identity/v3/projects/a5e171c5fbf5475090398de10d401d1f/users/7ed630c9690f4dfca2f13e7afc9bb9de/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:26.915661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747158-73f3-4dc2-921d-d31a3711e5a0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:26.916193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747158-73f3-4dc2-921d-d31a3711e5a0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:26.916841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747158-73f3-4dc2-921d-d31a3711e5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:26.998147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-57747158-73f3-4dc2-921d-d31a3711e5a0 None None] MFA Rules not processed for user `7ed630c9690f4dfca2f13e7afc9bb9de`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:27.016409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-57747158-73f3-4dc2-921d-d31a3711e5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:27.032896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:27.184031 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2221/4450] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:27.217535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:27.219369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:27.345352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff3a5c-fd67-48da-86ea-4ec1a793fd58 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:27.353234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dc31e-1861-4dd4-a86c-5a945a2eda06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:27.361240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dc31e-1861-4dd4-a86c-5a945a2eda06 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:27.361468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dc31e-1861-4dd4-a86c-5a945a2eda06 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:27.361960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dc31e-1861-4dd4-a86c-5a945a2eda06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:27.378022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dc31e-1861-4dd4-a86c-5a945a2eda06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:27.380577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dc31e-1861-4dd4-a86c-5a945a2eda06 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:27.394022 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2230/4451] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:27 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:31.985459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:31.985994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:31.986118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:32.062251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] MFA Rules not processed for user `bbbfb72cf13347ce901cad42460e7dce`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:32.063990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:32.219451 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2222/4452] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:32.229298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:32.230766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:32.231236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:32.267178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] MFA Rules not processed for user `bbbfb72cf13347ce901cad42460e7dce`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:32.268722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:32.292001 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2231/4453] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:32.306028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:32.308084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:32.497048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58757f1f-dd61-471f-9b75-5f792c1da1e9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:32.528014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:32.528713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:32.529002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:32.530092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:32.559528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:32.564133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:32.608204 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2223/4454] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:36.787441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:36.789593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:37.028408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e07f0-bfb8-4173-bd0e-2c91729655cf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:37.046609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:37.047269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:37.047553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:37.047871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] PATH_INFO: `/v3/users/42fe36190f974a289414202fe1e4ee44` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:37.048946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fe36190f974a289414202fe1e4ee44)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:37.050933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:37.157250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fe36190f974a289414202fe1e4ee44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:37.189415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fe36190f974a289414202fe1e4ee44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:37.192413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:37.202557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fe36190f974a289414202fe1e4ee44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:37.225684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fe36190f974a289414202fe1e4ee44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:37.229134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9c5f2318-fdfd-4179-974a-4c7afb236bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fe36190f974a289414202fe1e4ee44'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:37.258781 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2232/4455] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:36 2022] DELETE /identity/v3/users/42fe36190f974a289414202fe1e4ee44 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:38.209817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:38.221556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a552a-a962-4bb5-bde3-4d777296cadf service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:38.245365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:38.348988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:38.350134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:38.351199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:38.351715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] PATH_INFO: `/v3/projects/54cc07e66b1d4c55827d9b723a8df459` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:38.353239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=54cc07e66b1d4c55827d9b723a8df459)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:38.357427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:38.441537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54cc07e66b1d4c55827d9b723a8df459', 'request_id': 'req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:38.450490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54cc07e66b1d4c55827d9b723a8df459', 'request_id': 'req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:38.459243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54cc07e66b1d4c55827d9b723a8df459', 'request_id': 'req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:38.473898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] The token cache is being invalidate because project 54cc07e66b1d4c55827d9b723a8df459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:38.474181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:38.475118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f32c99e5-418c-4420-9c10-4fa9f8b9a51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:38.476929 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2224/4456] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:38 2022] DELETE /identity/v3/projects/54cc07e66b1d4c55827d9b723a8df459 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:38.482604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763d15-41db-4126-8ada-cfa6ec92261f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:38.483574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763d15-41db-4126-8ada-cfa6ec92261f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:38.484022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763d15-41db-4126-8ada-cfa6ec92261f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:38.513701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d3763d15-41db-4126-8ada-cfa6ec92261f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:38.515085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3763d15-41db-4126-8ada-cfa6ec92261f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:38.598835 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2233/4457] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:38.600201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c693a-d421-4cdb-a387-8aab5cf4e666 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:38.600489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c693a-d421-4cdb-a387-8aab5cf4e666 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:38.600760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c693a-d421-4cdb-a387-8aab5cf4e666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:38.623604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-709c693a-d421-4cdb-a387-8aab5cf4e666 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:38.624848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-709c693a-d421-4cdb-a387-8aab5cf4e666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:38.638797 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2225/4458] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:38.640020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3763d15-41db-4126-8ada-cfa6ec92261f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:38.650941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3763d15-41db-4126-8ada-cfa6ec92261f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:38.656445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:38.657368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:38.657916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:38.658464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:38.659363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:38.659941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:38.662201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:38.681686 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2234/4459] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:38.682830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c693a-d421-4cdb-a387-8aab5cf4e666 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:38.692748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c693a-d421-4cdb-a387-8aab5cf4e666 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:38.699138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:38.699750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:38.700151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:38.700393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:38.701177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:38.703249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:38.753800 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2226/4460] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:38 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:38.757812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:38.774583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e34553-d8c7-4b2b-a9f5-1d4b39c47c6a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:38.926620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:38.929689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:38.930279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:38.930755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:38.932848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:38.936056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:38.967814 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2235/4461] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:38 2022] POST /identity/v3/users => generated 303 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:38.972967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:38.990628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7875-d76e-45b5-b6d3-114f6d5f30ff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:39.000346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:39.002512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:39.002998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:39.003320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:39.004255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:39.007188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:39.024569 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2227/4462] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:39.028980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:39.048741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2ee24-08ae-4764-beaf-63959cd4cf59 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:39.056239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c4aef-35f4-4591-bc76-346dc5ce76a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:39.058240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c4aef-35f4-4591-bc76-346dc5ce76a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:39.058761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c4aef-35f4-4591-bc76-346dc5ce76a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:39.059248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c4aef-35f4-4591-bc76-346dc5ce76a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:39.061042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c4aef-35f4-4591-bc76-346dc5ce76a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:39.063359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c4aef-35f4-4591-bc76-346dc5ce76a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:39.082464 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2236/4463] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:39.085867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:39.108153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bbe8a-3eb2-4567-b825-bcce18eb2d96 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:39.122950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:39.123923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:39.125525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:39.126575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] PATH_INFO: `/v3/projects/a31cdf32235d44b1bbd691ebea38663b/users/b766ab7d168745a4bb8b07fb8e5df3ed/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:39.158236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31cdf32235d44b1bbd691ebea38663b, user_id=b766ab7d168745a4bb8b07fb8e5df3ed, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:39.162106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:39.211447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2228/4464] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:39 2022] PUT /identity/v3/projects/a31cdf32235d44b1bbd691ebea38663b/users/b766ab7d168745a4bb8b07fb8e5df3ed/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:39.218414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:39.218839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:39.219216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:39.247792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:39.249419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:39.321706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:39.362330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2237/4465] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:39.392178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:39.394479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:39.530150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c08ec-8485-40b9-9211-90010750e0b2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:39.534639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:39.535367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:39.536708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:39.536994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:39.576711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:39.584653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:39.586897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:39.605983 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2229/4466] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:42.809357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:42.817611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:42.838416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77a36c-74db-45fd-8623-fbfc8112805a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:42.844197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:42.845488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:42.846435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:42.846999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] PATH_INFO: `/v3/users/3561f4fb18f84b899f68c5fcf897ca12` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:42.848450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3561f4fb18f84b899f68c5fcf897ca12)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:42.852318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:42.960373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3561f4fb18f84b899f68c5fcf897ca12'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:42.995096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3561f4fb18f84b899f68c5fcf897ca12'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:43.007204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3561f4fb18f84b899f68c5fcf897ca12'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:43.024535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3561f4fb18f84b899f68c5fcf897ca12'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:43.026905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-94b56574-c537-473a-aee0-33a9596a681c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3561f4fb18f84b899f68c5fcf897ca12'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:43.041639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2238/4467] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:42 2022] DELETE /identity/v3/users/3561f4fb18f84b899f68c5fcf897ca12 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:44.180304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:44.193456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f8ac-d983-4491-ba4b-0bdc1b5fc5b6 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:44.328396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:44.329302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:44.329689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:44.330330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] PATH_INFO: `/v3/projects/7c3ba8eb9cfb41cb87be6ffad5cb3789` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:44.331680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c3ba8eb9cfb41cb87be6ffad5cb3789)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:44.333767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:44.437472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c3ba8eb9cfb41cb87be6ffad5cb3789', 'request_id': 'req-a647ec85-99a1-43e7-a224-59a19ecd6296'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:44.448087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c3ba8eb9cfb41cb87be6ffad5cb3789', 'request_id': 'req-a647ec85-99a1-43e7-a224-59a19ecd6296'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:44.464413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c3ba8eb9cfb41cb87be6ffad5cb3789', 'request_id': 'req-a647ec85-99a1-43e7-a224-59a19ecd6296'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:44.482614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] The token cache is being invalidate because project 7c3ba8eb9cfb41cb87be6ffad5cb3789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:44.483225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:44.484491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a647ec85-99a1-43e7-a224-59a19ecd6296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:44.486958 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2230/4468] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:44 2022] DELETE /identity/v3/projects/7c3ba8eb9cfb41cb87be6ffad5cb3789 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:44.495457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a85cbf-e918-48ec-bef9-38515ec5f9d3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:44.495982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a85cbf-e918-48ec-bef9-38515ec5f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:44.497740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a85cbf-e918-48ec-bef9-38515ec5f9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:44.525918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-42a85cbf-e918-48ec-bef9-38515ec5f9d3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:44.530615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-42a85cbf-e918-48ec-bef9-38515ec5f9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:44.651229 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2239/4469] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:44.658975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba01987-d413-4aa7-9cdd-48af230dda2d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:44.659731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba01987-d413-4aa7-9cdd-48af230dda2d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:44.660264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba01987-d413-4aa7-9cdd-48af230dda2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:44.699047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cba01987-d413-4aa7-9cdd-48af230dda2d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:44.702928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cba01987-d413-4aa7-9cdd-48af230dda2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:44.715118 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2231/4470] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:44.719072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a85cbf-e918-48ec-bef9-38515ec5f9d3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:44.739696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a85cbf-e918-48ec-bef9-38515ec5f9d3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:44.745937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:44.746866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:44.747309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:44.747914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:44.748722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:44.749197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:44.751209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:44.770267 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2240/4471] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:44.772777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba01987-d413-4aa7-9cdd-48af230dda2d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:44.801934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba01987-d413-4aa7-9cdd-48af230dda2d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:44.809014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:44.810141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:44.811231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:44.811617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:44.813395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:44.816277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:44.868117 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2232/4472] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:44 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:44.874630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:44.896850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d4c09-e34c-4402-aef0-e5f5f166d7bf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:45.073733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:45.091890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:45.093028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:45.093585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:45.094249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:45.095186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:45.097674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:45.141968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2241/4473] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:44 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:45.146208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:45.167555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4422723-e29d-4e2c-bc42-da3f718cbd63 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:45.173120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:45.174714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:45.175208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:45.175701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:45.176647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:45.179642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:45.195877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2233/4474] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:45.201438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:45.219531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c2697-0956-4b78-bae9-be90f879c8a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:45.238601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65ac5a-15ac-44c6-a77f-7c88958aa93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:45.239717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ac5a-15ac-44c6-a77f-7c88958aa93a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:45.240207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ac5a-15ac-44c6-a77f-7c88958aa93a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:45.240711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ac5a-15ac-44c6-a77f-7c88958aa93a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:45.242553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65ac5a-15ac-44c6-a77f-7c88958aa93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:45.244639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65ac5a-15ac-44c6-a77f-7c88958aa93a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:45.276353 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2242/4475] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:45.277503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:45.323608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c807bb3-be7e-4356-bb63-d44529b6764c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:45.351528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:45.354794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:45.356016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:45.357508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] PATH_INFO: `/v3/projects/10428a8aac4a4b4d94551fe3087df565/users/3190a548330f47d290bdb0bc6c2e69eb/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:45.422912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=10428a8aac4a4b4d94551fe3087df565, user_id=3190a548330f47d290bdb0bc6c2e69eb, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:45.438340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:45.468095 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2234/4476] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:45 2022] PUT /identity/v3/projects/10428a8aac4a4b4d94551fe3087df565/users/3190a548330f47d290bdb0bc6c2e69eb/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:45.473925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa6ddd-8a53-4021-8a50-d4b4f03e78f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:45.475741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa6ddd-8a53-4021-8a50-d4b4f03e78f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:45.476632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa6ddd-8a53-4021-8a50-d4b4f03e78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:45.519511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a8aa6ddd-8a53-4021-8a50-d4b4f03e78f0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:45.521092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a8aa6ddd-8a53-4021-8a50-d4b4f03e78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:45.682837 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2243/4477] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:45.726975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:45.733578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:45.795687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:45.872241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91296a5a-c5fc-4612-8c73-4a8cd918c00e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:45.877237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bbd12-c0f2-4d7d-b374-e17a993a8388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:45.878506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bbd12-c0f2-4d7d-b374-e17a993a8388 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:45.879614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bbd12-c0f2-4d7d-b374-e17a993a8388 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:45.880121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bbd12-c0f2-4d7d-b374-e17a993a8388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:45.893423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bbd12-c0f2-4d7d-b374-e17a993a8388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:45.895201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bbd12-c0f2-4d7d-b374-e17a993a8388 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:45.915458 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2235/4478] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:46.704107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:46.704677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:46.706100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:46.772335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] MFA Rules not processed for user `3190a548330f47d290bdb0bc6c2e69eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:46.774404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:46.942203 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2244/4479] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:46.952742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:46.954300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:46.954740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:46.998887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] MFA Rules not processed for user `3190a548330f47d290bdb0bc6c2e69eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:47.000449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:47.026556 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2236/4480] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:47.027546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:47.029896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:47.164790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:47.190121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ee8c5-7469-43d7-98e2-5c78cba068ac None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:47.196455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177fa24-aa46-47f3-b011-75a9f977eb8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:47.197531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177fa24-aa46-47f3-b011-75a9f977eb8e service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:47.197846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177fa24-aa46-47f3-b011-75a9f977eb8e service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:47.198185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177fa24-aa46-47f3-b011-75a9f977eb8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:47.208659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177fa24-aa46-47f3-b011-75a9f977eb8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:47.211105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177fa24-aa46-47f3-b011-75a9f977eb8e service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:47.223335 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2245/4481] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:47.450849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:47.452515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:47.478067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752788ba-5b3a-40a0-9633-a05dc094424c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:47.484002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:47.484719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:47.485086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:47.485532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] PATH_INFO: `/v3/users/4759bc2f66374825b80bd2a79da9a260` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:47.487451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4759bc2f66374825b80bd2a79da9a260)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:47.489564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:47.585771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4759bc2f66374825b80bd2a79da9a260'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:47.608164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4759bc2f66374825b80bd2a79da9a260'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:47.608961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4759bc2f66374825b80bd2a79da9a260'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:47.625369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4759bc2f66374825b80bd2a79da9a260'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:47.626274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4759bc2f66374825b80bd2a79da9a260'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:47.659595 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2237/4482] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:47 2022] DELETE /identity/v3/users/4759bc2f66374825b80bd2a79da9a260 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:47.664065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:47.664269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:47.664548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:47.709626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:47.710899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:47.835650 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2246/4483] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:47.846653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:47.848140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:48.014120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a15ce-ceb8-48a9-808b-679e671b1511 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:48.025753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18d90d-6311-47f3-bcbd-357c0c3c6a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:48.027238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18d90d-6311-47f3-bcbd-357c0c3c6a01 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:48.027694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18d90d-6311-47f3-bcbd-357c0c3c6a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:48.028173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18d90d-6311-47f3-bcbd-357c0c3c6a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:48.055142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18d90d-6311-47f3-bcbd-357c0c3c6a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:48.060371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18d90d-6311-47f3-bcbd-357c0c3c6a01 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:48.075432 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2238/4484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:48.675088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:48.676582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:48.725194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6443b-03a3-4f94-bba0-f5146a81a1fb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:48.745991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:48.746862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:48.747144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:48.747418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] PATH_INFO: `/v3/projects/d81aa8c53d9c41238de5521448d9613f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:48.748586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d81aa8c53d9c41238de5521448d9613f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:48.753039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:48.828429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd81aa8c53d9c41238de5521448d9613f', 'request_id': 'req-927b59ed-782d-4a97-90e3-a9168d588449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:48.842929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd81aa8c53d9c41238de5521448d9613f', 'request_id': 'req-927b59ed-782d-4a97-90e3-a9168d588449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:48.852017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd81aa8c53d9c41238de5521448d9613f', 'request_id': 'req-927b59ed-782d-4a97-90e3-a9168d588449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:48.876031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] The token cache is being invalidate because project d81aa8c53d9c41238de5521448d9613f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:48.877470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:48.878489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:48.881261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2247/4485] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:48 2022] DELETE /identity/v3/projects/d81aa8c53d9c41238de5521448d9613f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:48.886787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445f440-a769-4bd2-8c81-68c420089a43 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:48.887074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445f440-a769-4bd2-8c81-68c420089a43 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:48.887522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445f440-a769-4bd2-8c81-68c420089a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:48.922883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0445f440-a769-4bd2-8c81-68c420089a43 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:48.937898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0445f440-a769-4bd2-8c81-68c420089a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:48.977295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:48.999800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:49.002784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:49.046219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2239/4486] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:49.054444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee256b3-aa64-4602-b412-29820d1abf61 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.056197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee256b3-aa64-4602-b412-29820d1abf61 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.057211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee256b3-aa64-4602-b412-29820d1abf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.129276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2ee256b3-aa64-4602-b412-29820d1abf61 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:49.130864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2ee256b3-aa64-4602-b412-29820d1abf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:49.154577 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2240/4487] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:49.158566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee256b3-aa64-4602-b412-29820d1abf61 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:49.187041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee256b3-aa64-4602-b412-29820d1abf61 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:49.197662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:49.198692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.199383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.199859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.207090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:49.207593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:49.209340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:49.238390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b59ed-782d-4a97-90e3-a9168d588449 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:49.239188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2241/4488] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:04:49.240196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:49.248523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:49.250315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.251933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.252105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055a77-e326-461b-bf39-cc356291ad83 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:49.252966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] PATH_INFO: `/v3/users/3190a548330f47d290bdb0bc6c2e69eb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.263832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:49.266516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3190a548330f47d290bdb0bc6c2e69eb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:49.270155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.270530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.270882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.273346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:49.274298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:49.276546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:49.306480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:49.338970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2242/4489] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:49 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:49.346858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:49.440720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3190a548330f47d290bdb0bc6c2e69eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:49.472788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3190a548330f47d290bdb0bc6c2e69eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:49.491233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3190a548330f47d290bdb0bc6c2e69eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:49.509235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3190a548330f47d290bdb0bc6c2e69eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:49.510797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3190a548330f47d290bdb0bc6c2e69eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:49.523876 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2248/4490] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:48 2022] DELETE /identity/v3/users/3190a548330f47d290bdb0bc6c2e69eb => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:49.538305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9634-0fce-4dd7-92c2-7542746c39e4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:49.697762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:49.698447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.698709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.698994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.699641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:49.702403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:49.736346 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2243/4491] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:49 2022] POST /identity/v3/users => generated 284 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:49.739687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:49.741393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:49.767398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee47b0a-9054-4718-8efb-235a0bc09a42 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:49.773424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:49.774342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.775399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.775845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.777007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:49.779148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:49.793688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2249/4492] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:49.795747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:49.824226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d321245-a8ed-4720-8725-8af71d74bf45 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:49.830569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87d6a6-e57c-42c1-825d-29c91e57c07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:49.831392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87d6a6-e57c-42c1-825d-29c91e57c07d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.831742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87d6a6-e57c-42c1-825d-29c91e57c07d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.832093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87d6a6-e57c-42c1-825d-29c91e57c07d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.832812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87d6a6-e57c-42c1-825d-29c91e57c07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:49.834695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87d6a6-e57c-42c1-825d-29c91e57c07d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:49.871073 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2244/4493] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:49.876155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:49.890348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbd866-f027-4295-a980-be41e8986a48 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:49.903701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:49.904771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.905803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.906356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] PATH_INFO: `/v3/projects/4a4f0dbb2e68458ab1e50702fc0e9dfa/users/5215babeaff64f97914a58db6dc33db1/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:49.936363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4f0dbb2e68458ab1e50702fc0e9dfa, user_id=5215babeaff64f97914a58db6dc33db1, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:49.939464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:49.961431 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2250/4494] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:49 2022] PUT /identity/v3/projects/4a4f0dbb2e68458ab1e50702fc0e9dfa/users/5215babeaff64f97914a58db6dc33db1/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:49.992564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:49.996277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:49.996683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.037063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] MFA Rules not processed for user `5215babeaff64f97914a58db6dc33db1`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:50.040291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:50.056373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:50.164209 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2245/4495] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:50.177192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:50.180099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:50.197593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:50.204391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:50.350093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e06cf-c72a-4217-ab5d-93aa05a6eae9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:50.363292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420f58f-31d3-4d0c-911f-91422d5a0607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:50.364527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420f58f-31d3-4d0c-911f-91422d5a0607 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:50.365806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420f58f-31d3-4d0c-911f-91422d5a0607 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:50.366130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420f58f-31d3-4d0c-911f-91422d5a0607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.378161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420f58f-31d3-4d0c-911f-91422d5a0607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:50.380138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420f58f-31d3-4d0c-911f-91422d5a0607 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:50.402253 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2251/4496] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:50.408399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa41b44-b78d-4749-ae0e-31db2a9614aa None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:50.414632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:50.415398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:50.415802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:50.416139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] PATH_INFO: `/v3/projects/10428a8aac4a4b4d94551fe3087df565` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.421355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=10428a8aac4a4b4d94551fe3087df565)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:50.423352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:50.488540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10428a8aac4a4b4d94551fe3087df565', 'request_id': 'req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:50.496914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10428a8aac4a4b4d94551fe3087df565', 'request_id': 'req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:50.506522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10428a8aac4a4b4d94551fe3087df565', 'request_id': 'req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:50.520838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] The token cache is being invalidate because project 10428a8aac4a4b4d94551fe3087df565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:50.521230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:50.522944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff1cbdae-dd94-4099-82d9-7cc3dfb0a99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:50.529066 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2246/4497] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:50 2022] DELETE /identity/v3/projects/10428a8aac4a4b4d94551fe3087df565 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:50.531073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363561-e954-4f82-929d-ccc43428341e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:50.531366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363561-e954-4f82-929d-ccc43428341e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:50.531656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363561-e954-4f82-929d-ccc43428341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.565798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ef363561-e954-4f82-929d-ccc43428341e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:50.567595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef363561-e954-4f82-929d-ccc43428341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:50.639307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:50.654938 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2252/4498] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:50.662964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e3cc-1c09-41aa-9f61-158cb365f009 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:50.662964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e3cc-1c09-41aa-9f61-158cb365f009 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:50.662964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9e3cc-1c09-41aa-9f61-158cb365f009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.708600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-85b9e3cc-1c09-41aa-9f61-158cb365f009 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:50.710052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-85b9e3cc-1c09-41aa-9f61-158cb365f009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:50.730383 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2247/4499] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:50.734533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef363561-e954-4f82-929d-ccc43428341e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:50.753299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef363561-e954-4f82-929d-ccc43428341e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:50.759229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:50.760386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:50.760854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:50.761137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.761791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:50.762289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:50.764130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:50.794876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:50.794999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:50.795254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.820771 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2253/4500] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:50.824698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:50.841684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76b071-5c73-4fed-b811-3c9ab37ce2f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:50.868014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:50.868689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:50.869506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:50.869901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:50.870500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:50.872507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:50.878691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] MFA Rules not processed for user `b766ab7d168745a4bb8b07fb8e5df3ed`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:50.881076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:50.914835 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2254/4501] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:50 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:50.916677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:50.922697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:51.035557 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2248/4502] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:51.048447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.051316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:51.088270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87dea-1161-49f5-939b-9f3620a3b9ee None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.092641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105a191-5f0b-41d8-b1d1-54933fa9f5aa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.100979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.102480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.102640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.103541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.104202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.106711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.107234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.107384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.107663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.107943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.108638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.110797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.149444 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2255/4503] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:50 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:51.156030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.172066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003ce2-80b3-4833-b449-5e7fb244f9f7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.174586 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2249/4504] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:51 2022] POST /identity/v3/projects => generated 405 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:51.175451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.308264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203c37-259c-49b5-a3bc-45ffe3b90555 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.311009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.311807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.312210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.312599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.313392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.314205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.314818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.315204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.317228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.323291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.337961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.337961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.338210 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2256/4505] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:51.355267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.361315 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2250/4506] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:51 2022] POST /identity/v3/users => generated 303 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:51.365176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.367786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877230c-b5d5-476f-8cff-0ec78276dc60 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.377034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854682c-d24b-410d-8b63-c2bf5c22139b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.383165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.383398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.383528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.383528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.383695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.383948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.384075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.384176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.384540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.384861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.386498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.386700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.408732 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2251/4507] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:51.413417 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2257/4508] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:51.415346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.420337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.443285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacea2bc-3c50-4ff4-8e7b-60359b5350d4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.451022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0129bc-d61d-459a-93fd-3958f31ba64b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.459838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.461538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.461850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.463205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.463889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.466270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccace810-8314-45b9-a56f-231efaca4895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.467616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace810-8314-45b9-a56f-231efaca4895 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.467916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace810-8314-45b9-a56f-231efaca4895 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.468199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccace810-8314-45b9-a56f-231efaca4895 admin admin] PATH_INFO: `/v3/projects/1a744bcf5d894f408c32e4f691be4bc3/users/fee2190c31e347c5926d9762d95d8dc8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.469556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.496107 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2252/4509] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:51.498402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.499589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccace810-8314-45b9-a56f-231efaca4895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a744bcf5d894f408c32e4f691be4bc3, user_id=fee2190c31e347c5926d9762d95d8dc8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.505207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccace810-8314-45b9-a56f-231efaca4895 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.518553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b34ba-5bea-4626-a2e8-62e4963a0537 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.534154 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2258/4510] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:51 2022] PUT /identity/v3/projects/1a744bcf5d894f408c32e4f691be4bc3/users/fee2190c31e347c5926d9762d95d8dc8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:51.534502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.535301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.535591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.535870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] PATH_INFO: `/v3/projects/385194bd6b1143edb0d72548ec736c88/users/173f60c866c8453a97d2701fee66bfb5/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.546872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0811-e652-4baf-8a1d-af67601a575d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.547217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0811-e652-4baf-8a1d-af67601a575d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.547570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0811-e652-4baf-8a1d-af67601a575d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.576812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=385194bd6b1143edb0d72548ec736c88, user_id=173f60c866c8453a97d2701fee66bfb5, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.583029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:51.589631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:51.612999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b2ee0811-e652-4baf-8a1d-af67601a575d None None] MFA Rules not processed for user `fee2190c31e347c5926d9762d95d8dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:51.613765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2253/4511] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:51 2022] PUT /identity/v3/projects/385194bd6b1143edb0d72548ec736c88/users/173f60c866c8453a97d2701fee66bfb5/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:51.614836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b2ee0811-e652-4baf-8a1d-af67601a575d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:51.698422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2259/4512] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:51.707215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:51.713906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:51.800403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:51.949654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2111d-0884-4753-aa2a-62fb195b6f6a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:51.967950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158c9ce-b924-4cef-9c87-1518fcd32392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:51.968786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c9ce-b924-4cef-9c87-1518fcd32392 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:51.969192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c9ce-b924-4cef-9c87-1518fcd32392 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:51.969566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158c9ce-b924-4cef-9c87-1518fcd32392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:51.987204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158c9ce-b924-4cef-9c87-1518fcd32392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:51.993622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158c9ce-b924-4cef-9c87-1518fcd32392 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:52.024870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2254/4513] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:52.749323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0811-e652-4baf-8a1d-af67601a575d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:52.911047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0811-e652-4baf-8a1d-af67601a575d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:52.920371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:52.921597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:52.922165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:52.922813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] PATH_INFO: `/v3/users/fee2190c31e347c5926d9762d95d8dc8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:52.924722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee2190c31e347c5926d9762d95d8dc8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:52.927502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:53.050047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee2190c31e347c5926d9762d95d8dc8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:53.083691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee2190c31e347c5926d9762d95d8dc8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:53.112227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee2190c31e347c5926d9762d95d8dc8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:53.131703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee2190c31e347c5926d9762d95d8dc8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:53.132273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee2190c31e347c5926d9762d95d8dc8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:53.155193 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2260/4514] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:52 2022] DELETE /identity/v3/users/fee2190c31e347c5926d9762d95d8dc8 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:53.159581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99392275-785a-47e9-9946-e3a0686d913c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:53.161099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99392275-785a-47e9-9946-e3a0686d913c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:53.162155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99392275-785a-47e9-9946-e3a0686d913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:53.223072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-99392275-785a-47e9-9946-e3a0686d913c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:53.233524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-99392275-785a-47e9-9946-e3a0686d913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:53.486364 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2255/4515] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:53.562755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:53.565249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:53.601815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:53.747989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c40fd1-c1c9-453d-83df-37c3eddbaa8d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:53.766208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631b3b7-bc7a-4ce5-9cd4-cdcb158b9b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:53.767571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631b3b7-bc7a-4ce5-9cd4-cdcb158b9b27 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:53.768023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631b3b7-bc7a-4ce5-9cd4-cdcb158b9b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:53.768530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631b3b7-bc7a-4ce5-9cd4-cdcb158b9b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:53.810687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631b3b7-bc7a-4ce5-9cd4-cdcb158b9b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:53.813459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631b3b7-bc7a-4ce5-9cd4-cdcb158b9b27 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:53.845866 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2261/4516] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:04:55.603057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99392275-785a-47e9-9946-e3a0686d913c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:55.605400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-99392275-785a-47e9-9946-e3a0686d913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:55.628230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99392275-785a-47e9-9946-e3a0686d913c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:55.640928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:55.641716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:55.642131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:55.642507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] PATH_INFO: `/v3/projects/1a744bcf5d894f408c32e4f691be4bc3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:55.644151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a744bcf5d894f408c32e4f691be4bc3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:55.646550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:55.752164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a744bcf5d894f408c32e4f691be4bc3', 'request_id': 'req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:55.770920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a744bcf5d894f408c32e4f691be4bc3', 'request_id': 'req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:55.781736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a744bcf5d894f408c32e4f691be4bc3', 'request_id': 'req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:55.796923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] The token cache is being invalidate because project 1a744bcf5d894f408c32e4f691be4bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:55.797042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:55.800755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1a6622ed-4a40-484e-8cdc-1f4badfab0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:55.803107 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2256/4517] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:55 2022] DELETE /identity/v3/projects/1a744bcf5d894f408c32e4f691be4bc3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:55.810154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e2bd-452e-4759-b9b3-ffa6f7394c28 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:55.810669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e2bd-452e-4759-b9b3-ffa6f7394c28 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:55.810669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e2bd-452e-4759-b9b3-ffa6f7394c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:55.834607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-09b0e2bd-452e-4759-b9b3-ffa6f7394c28 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:55.836014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-09b0e2bd-452e-4759-b9b3-ffa6f7394c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:55.923298 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2262/4518] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:55.924828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539d2cf-c4fc-4dae-bed2-1e0105b44259 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:55.925435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539d2cf-c4fc-4dae-bed2-1e0105b44259 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:55.925961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539d2cf-c4fc-4dae-bed2-1e0105b44259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:55.966240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b539d2cf-c4fc-4dae-bed2-1e0105b44259 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:55.967809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b539d2cf-c4fc-4dae-bed2-1e0105b44259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:55.985382 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2257/4519] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:04:55.987086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e2bd-452e-4759-b9b3-ffa6f7394c28 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.015142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e2bd-452e-4759-b9b3-ffa6f7394c28 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.021291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.022089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.022708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.023065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.023880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:56.024233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.026615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.081881 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2263/4520] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:56.086129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539d2cf-c4fc-4dae-bed2-1e0105b44259 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.097269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539d2cf-c4fc-4dae-bed2-1e0105b44259 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.102466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.104324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.104771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.107312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.108186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.112526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.166528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2258/4521] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:56 2022] POST /identity/v3/projects => generated 367 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:56.170732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.182692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102736-1bef-4e43-833b-49fafde00501 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.267945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.268561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.268847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.269149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.269797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.272044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.303199 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2264/4522] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:56 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:56.304302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.324568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90342c5-ce9b-447b-b8e3-8913da64f35f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.329410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.330052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.330338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.330613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.331257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.332897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.345879 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2259/4523] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:56.348690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.360709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6db4-6af0-41d2-aaa2-0bade24a932f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.365742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efcf7f-4ae1-4eff-87d1-c5fcd4d210d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.366381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efcf7f-4ae1-4eff-87d1-c5fcd4d210d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.366707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efcf7f-4ae1-4eff-87d1-c5fcd4d210d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.366976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efcf7f-4ae1-4eff-87d1-c5fcd4d210d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.367614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efcf7f-4ae1-4eff-87d1-c5fcd4d210d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.369252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efcf7f-4ae1-4eff-87d1-c5fcd4d210d3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.383844 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2265/4524] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:56.385188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.396253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65afca-b3a1-473d-86f3-e62a4034632e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.406283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.406965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.407255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.407567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] PATH_INFO: `/v3/projects/64e1934d881847cf972681f24ecfa2dc/users/2adbe03bd6e843a48a56ec31098b78b3/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.413178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:56.429723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e1934d881847cf972681f24ecfa2dc, user_id=2adbe03bd6e843a48a56ec31098b78b3, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.435197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.456105 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2260/4525] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:56 2022] PUT /identity/v3/projects/64e1934d881847cf972681f24ecfa2dc/users/2adbe03bd6e843a48a56ec31098b78b3/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:56.467171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34cf44-69ab-41d6-89fe-967d5de7aa68 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.467463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34cf44-69ab-41d6-89fe-967d5de7aa68 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.467761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34cf44-69ab-41d6-89fe-967d5de7aa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.498271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5b34cf44-69ab-41d6-89fe-967d5de7aa68 None None] MFA Rules not processed for user `2adbe03bd6e843a48a56ec31098b78b3`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:56.499515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5b34cf44-69ab-41d6-89fe-967d5de7aa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:56.518615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:56.584746 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2266/4526] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:56.586052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.667901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556e26b-3294-4898-babb-0e5882a4ee96 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.673247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.673906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.674196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.674614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.675419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.678321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.749051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34cf44-69ab-41d6-89fe-967d5de7aa68 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.753926 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2261/4527] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:56 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:56.762593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34cf44-69ab-41d6-89fe-967d5de7aa68 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.807308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.812031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:56.860615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.862153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.862837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.863318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.864334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.867154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.909381 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2267/4528] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:56 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:56.910767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.927133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0eb0bd-74a2-47df-abf5-3a9c45117741 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.933942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.935223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.935684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.936240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.942169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.946104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:56.960438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384cafc-a5e4-4837-a62a-99794a37a66e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:56.971224 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2268/4529] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:04:56.977586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:56.979298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:56.980201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:56.980688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:56.981172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] PATH_INFO: `/v3/users/7ed630c9690f4dfca2f13e7afc9bb9de` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:56.983111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed630c9690f4dfca2f13e7afc9bb9de)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:56.994390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:57.002444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160fe9c-22a7-49c5-a78c-17247a35db4b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:57.008468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:57.009354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:57.009791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:57.010271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] PATH_INFO: `/v3/projects/0a3250da49b7427b960a8bbf2f7678be/users/9cb0d3009f244a8a979c6f27e7779257/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:57.040610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3250da49b7427b960a8bbf2f7678be, user_id=9cb0d3009f244a8a979c6f27e7779257, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:57.044802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:57.072241 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2269/4530] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:56 2022] PUT /identity/v3/projects/0a3250da49b7427b960a8bbf2f7678be/users/9cb0d3009f244a8a979c6f27e7779257/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:57.076336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:57.085196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed630c9690f4dfca2f13e7afc9bb9de'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:57.104537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed630c9690f4dfca2f13e7afc9bb9de'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:57.105511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed630c9690f4dfca2f13e7afc9bb9de'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:57.116496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed630c9690f4dfca2f13e7afc9bb9de'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:57.118298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dacbcfe0-b0f2-4cfd-bbd8-4b6e1802fe94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed630c9690f4dfca2f13e7afc9bb9de'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:57.132829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:57.150575 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2262/4531] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:04:56 2022] DELETE /identity/v3/users/7ed630c9690f4dfca2f13e7afc9bb9de => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:04:57.155763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:57.156863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:57.157357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:57.202487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:57.204168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:57.233135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626109c-e8d8-4451-9350-45f8e5cf87e0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:57.290235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:57.324507 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2263/4532] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:57.349355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:57.351810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:57.365211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:57.366053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:57.366404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:57.366840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:57.367567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:57.370719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:57.388849 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2270/4533] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:04:57.390712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:57.394415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:57.415152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe707ee-b9b1-4156-9165-6ef693995f41 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:57.429209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4238d0-503e-45bd-ad76-3153bd7945b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:57.432155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4238d0-503e-45bd-ad76-3153bd7945b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:57.432439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4238d0-503e-45bd-ad76-3153bd7945b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:57.432723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4238d0-503e-45bd-ad76-3153bd7945b6 admin admin] PATH_INFO: `/v3/projects/0a3250da49b7427b960a8bbf2f7678be/users/9cb0d3009f244a8a979c6f27e7779257/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:57.437542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4238d0-503e-45bd-ad76-3153bd7945b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3250da49b7427b960a8bbf2f7678be, user_id=9cb0d3009f244a8a979c6f27e7779257, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:57.440949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4238d0-503e-45bd-ad76-3153bd7945b6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:57.463063 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2271/4534] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:04:57 2022] PUT /identity/v3/projects/0a3250da49b7427b960a8bbf2f7678be/users/9cb0d3009f244a8a979c6f27e7779257/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:57.468017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d90b61-9b18-40a5-b251-b4ec4bfd734d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:57.468128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d90b61-9b18-40a5-b251-b4ec4bfd734d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:57.468398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d90b61-9b18-40a5-b251-b4ec4bfd734d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:57.487603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334ef8-3cd1-404a-8aee-b93731b8b248 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:57.538322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b2d90b61-9b18-40a5-b251-b4ec4bfd734d None None] MFA Rules not processed for user `9cb0d3009f244a8a979c6f27e7779257`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:57.539777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b2d90b61-9b18-40a5-b251-b4ec4bfd734d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:57.594966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:57.598243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:57.598697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:57.599114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:57.699602 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2272/4535] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:57.703726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d90b61-9b18-40a5-b251-b4ec4bfd734d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:57.725415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d90b61-9b18-40a5-b251-b4ec4bfd734d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:57.736924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:57.737481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:57.737768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:57.738418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:57.739400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:57.740427 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:04:57.740427 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:04:57.740427 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:04:57.741324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:57.749143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:57.768653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:57.771399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:57.780562 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2273/4536] 10.222.0.33 () {60 vars in 1150 bytes} [Sat Sep 10 21:04:57 2022] POST /identity/v3/domains => generated 277 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:57.781898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:57.796837 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2264/4537] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:04:58.023701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ef76-e82d-4a22-b076-d3a801b51b24 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:58.036773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:58.043513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:58.043806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:58.044093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:58.045212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:58.048955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:58.103584 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2274/4538] 10.222.0.33 () {60 vars in 1150 bytes} [Sat Sep 10 21:04:57 2022] POST /identity/v3/domains => generated 276 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:58.108918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:58.143582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5d64d-d6cd-43e5-b155-413cdd66f7c4 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:58.235617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:58.236546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:58.237044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:58.238082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:58.239132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:58.240421 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:04:58.240421 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:04:58.240421 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:04:58.242407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:58.276529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:58.278996 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2265/4539] 10.222.0.33 () {60 vars in 1150 bytes} [Sat Sep 10 21:04:58 2022] POST /identity/v3/domains => generated 277 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:58.289408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0d45e-613f-4e58-8404-4caba6d7c70f tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:58.301917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:58.395953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:58.396974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:58.397499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:58.398010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:58.398937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:58.402707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:58.441841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:58.445749 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2275/4540] 10.222.0.33 () {60 vars in 1150 bytes} [Sat Sep 10 21:04:58 2022] POST /identity/v3/domains => generated 280 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:58.461727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498dffe-c55a-4707-a0c0-411903603d56 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:58.511081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:58.513565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:58.616280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:58.616943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:58.617399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:58.617700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/e2da4b2315e040ae989be63150efd4dd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:58.640394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=e2da4b2315e040ae989be63150efd4dd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:58.642768 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:04:58.642768 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:04:58.642768 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:04:58.644669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:58.672202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a6fc3-296a-4e38-8f8a-428b5917c4ba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:58.690407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:58.691066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:58.691349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:58.691632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] PATH_INFO: `/v3/projects/a5e171c5fbf5475090398de10d401d1f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:58.694904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e171c5fbf5475090398de10d401d1f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:58.698361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:58.704567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.717011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.790350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e171c5fbf5475090398de10d401d1f', 'request_id': 'req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.798223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e171c5fbf5475090398de10d401d1f', 'request_id': 'req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.808682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e171c5fbf5475090398de10d401d1f', 'request_id': 'req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.831739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] The token cache is being invalidate because project a5e171c5fbf5475090398de10d401d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:58.832903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.833716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6d5a3c9c-786a-4b8d-a693-9b285a5276e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.836773 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2276/4541] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:04:58 2022] DELETE /identity/v3/projects/a5e171c5fbf5475090398de10d401d1f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:58.842704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24102-c7a6-4bbb-8f30-bf7a4412e5d7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:58.843049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24102-c7a6-4bbb-8f30-bf7a4412e5d7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:58.843336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24102-c7a6-4bbb-8f30-bf7a4412e5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:58.844211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd', 'request_id': 'req-33f8d527-64df-4923-9b84-b59aac4ec38b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.861119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd', 'request_id': 'req-33f8d527-64df-4923-9b84-b59aac4ec38b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.872671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:58.886617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd', 'request_id': 'req-33f8d527-64df-4923-9b84-b59aac4ec38b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.905098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7da24102-c7a6-4bbb-8f30-bf7a4412e5d7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:58.910100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7da24102-c7a6-4bbb-8f30-bf7a4412e5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:58.911078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] The token cache is being invalidate because project e2da4b2315e040ae989be63150efd4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:04:58.913371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.914703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.956680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd', 'request_id': 'req-33f8d527-64df-4923-9b84-b59aac4ec38b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.957611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd', 'request_id': 'req-33f8d527-64df-4923-9b84-b59aac4ec38b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.958697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'e2da4b2315e040ae989be63150efd4dd', 'request_id': 'req-33f8d527-64df-4923-9b84-b59aac4ec38b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:58.997747 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2266/4542] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:04:58 2022] DELETE /identity/v3/domains/e2da4b2315e040ae989be63150efd4dd => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:04:59.001706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:59.003397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:59.073551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:59.148513 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2277/4543] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:59.154819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd007dc-d48e-446b-a439-27eebe7f5254 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:59.155113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd007dc-d48e-446b-a439-27eebe7f5254 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:59.155392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd007dc-d48e-446b-a439-27eebe7f5254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:59.207281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-abd007dc-d48e-446b-a439-27eebe7f5254 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:04:59.208528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-abd007dc-d48e-446b-a439-27eebe7f5254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:59.211261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d527-64df-4923-9b84-b59aac4ec38b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:59.227743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:59.228202 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2278/4544] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:04:59.229985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:59.232338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:59.232785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:59.233634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:59.234637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd007dc-d48e-446b-a439-27eebe7f5254 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:59.236519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:59.254553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd007dc-d48e-446b-a439-27eebe7f5254 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:59.272625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:59.273465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:59.273753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:59.274614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:59.275642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:04:59.277987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:59.281362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:59.286500 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2267/4545] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:04:59 2022] POST /identity/v3/domains => generated 249 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:59.293481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:59.316471 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2279/4546] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:04:59.320481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:59.344331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598de97e-2ffe-4f3b-9ad3-4d7a046d3a0c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:59.460043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c6bfc-5141-433c-bb2e-72c194bfc0e2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:59.470558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:59.472201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:59.473279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:59.473719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/d9c849aa0c7745e7b42097185f9f9bfb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:59.495194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:59.496565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:59.497792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:59.498690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d9c849aa0c7745e7b42097185f9f9bfb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:59.499236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:59.500501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:59.501283 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:04:59.501283 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:04:59.501283 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:04:59.503577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:59.517528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:59.565730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:59.567184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:59.581384 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2280/4547] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:04:59 2022] POST /identity/v3/projects => generated 381 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:04:59.585589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:59.592212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:59.650010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:59.651691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:04:59.667565 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2268/4548] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:04:59 2022] PATCH /identity/v3/domains/d9c849aa0c7745e7b42097185f9f9bfb => generated 250 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:04:59.670811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:59.673067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:59.756467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:04:59.810126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a798e-6a9b-440d-81e0-a6048ff88549 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:59.823300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:59.826466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:59.826789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:59.827682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:59.829549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:59.833211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:04:59.878031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2281/4549] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:04:59 2022] POST /identity/v3/users => generated 291 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:04:59.879264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:04:59.880727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0259bcb-d154-4d10-a191-fbaa505348db tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:59.884470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:04:59.940893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbceae-1d7f-438c-b806-975cc8bbcee7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:04:59.957167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:04:59.960702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:04:59.962925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:04:59.964400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:04:59.967206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:04:59.972006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:00.000100 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2282/4550] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:00.005865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:00.051646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b498a0-d540-4fa3-962a-a9d847e6af01 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:00.075055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:00.076858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:00.077151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:00.077435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:00.078141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:00.079800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:00.132278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:00.135321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:00.135624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:00.135898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/d9c849aa0c7745e7b42097185f9f9bfb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:00.140531 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2283/4551] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:00.151673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:00.184654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f0952-7d28-42f3-a802-b71693ec5711 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:00.186026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d9c849aa0c7745e7b42097185f9f9bfb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:00.187868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:00.201519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea542d5-e738-4aff-ae89-557fada62f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:00.202237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea542d5-e738-4aff-ae89-557fada62f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:00.202517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea542d5-e738-4aff-ae89-557fada62f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:00.202789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea542d5-e738-4aff-ae89-557fada62f0f admin admin] PATH_INFO: `/v3/projects/10a007de9bc9437398bf90ae404cbef6/users/8a7b2b5ab3604b8fa25def34711efa1a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:00.210416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:00.253946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.265140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.280282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea542d5-e738-4aff-ae89-557fada62f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a007de9bc9437398bf90ae404cbef6, user_id=8a7b2b5ab3604b8fa25def34711efa1a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:00.286837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea542d5-e738-4aff-ae89-557fada62f0f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:00.342905 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2284/4552] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:00 2022] PUT /identity/v3/projects/10a007de9bc9437398bf90ae404cbef6/users/8a7b2b5ab3604b8fa25def34711efa1a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:00.355571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:00.356092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:00.356840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:00.386240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb', 'request_id': 'req-70942e48-ec29-4350-a433-9b2c7d566183'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.408046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb', 'request_id': 'req-70942e48-ec29-4350-a433-9b2c7d566183'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.426829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb', 'request_id': 'req-70942e48-ec29-4350-a433-9b2c7d566183'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.454268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] MFA Rules not processed for user `8a7b2b5ab3604b8fa25def34711efa1a`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:00.462173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:00.464868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] The token cache is being invalidate because project d9c849aa0c7745e7b42097185f9f9bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:00.465196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.467394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.490256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb', 'request_id': 'req-70942e48-ec29-4350-a433-9b2c7d566183'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.492315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb', 'request_id': 'req-70942e48-ec29-4350-a433-9b2c7d566183'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.493441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd9c849aa0c7745e7b42097185f9f9bfb', 'request_id': 'req-70942e48-ec29-4350-a433-9b2c7d566183'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:00.534912 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2269/4553] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:04:59 2022] DELETE /identity/v3/domains/d9c849aa0c7745e7b42097185f9f9bfb => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:00.536225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:00.539453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:00.550052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:00.627716 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2285/4554] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:05:00.671402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:00.672816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:00.775595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70942e48-ec29-4350-a433-9b2c7d566183 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:00.805734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:00.806957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:00.807247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:00.807521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:00.814111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:00.825251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:00.826715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf193f-2313-4a02-bfff-e76cdb60e9f5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:00.843554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:00.844413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:00.845926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:00.846261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:00.865392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:00.867123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:00.872668 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2270/4555] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:05:00 2022] POST /identity/v3/domains => generated 279 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:00.876778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:00.902782 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2286/4556] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:05:00.925869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:00.927723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:00.989230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3aee3-e90b-4fc2-882e-cfa5b32b4385 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:00.995835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:00.997402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:00.997789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:00.998557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/a3b688079959490ba1dffa69d196e366` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.013688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a3b688079959490ba1dffa69d196e366)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:01.015859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:01.038787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30af6-aabd-4fe1-acae-042e7e3759b9 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.045788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a3b688079959490ba1dffa69d196e366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.046653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a3b688079959490ba1dffa69d196e366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.068787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:01.085672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a3b688079959490ba1dffa69d196e366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.089317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a3b688079959490ba1dffa69d196e366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.095284 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2271/4557] 10.222.0.33 () {60 vars in 1250 bytes} [Sat Sep 10 21:05:00 2022] PATCH /identity/v3/domains/a3b688079959490ba1dffa69d196e366 => generated 279 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:01.096478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:01.098343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:01.218620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.219275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.219556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.219830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] PATH_INFO: `/v3/users/8a7b2b5ab3604b8fa25def34711efa1a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.221196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7b2b5ab3604b8fa25def34711efa1a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:01.228569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:01.241487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:01.285381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca139-ca23-438b-9aab-ba979e34ac0b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.292561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.293211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.293501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.293784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/a3b688079959490ba1dffa69d196e366` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.325529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=a3b688079959490ba1dffa69d196e366)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:01.330995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:01.337225 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2272/4558] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:01 2022] GET /identity/v3/domains/a3b688079959490ba1dffa69d196e366 => generated 279 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:01.341098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:01.363288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7b2b5ab3604b8fa25def34711efa1a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.372793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30de05-3a48-463d-94cc-90629c009248 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.382670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7b2b5ab3604b8fa25def34711efa1a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.383709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.384652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.385042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.385622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/a3b688079959490ba1dffa69d196e366` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.387257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a3b688079959490ba1dffa69d196e366)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:01.391307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:01.393439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7b2b5ab3604b8fa25def34711efa1a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.411523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7b2b5ab3604b8fa25def34711efa1a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.420678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-df5a7115-e000-40b9-a73d-e49aaf93d56e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7b2b5ab3604b8fa25def34711efa1a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.430226 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2287/4559] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:00 2022] DELETE /identity/v3/users/8a7b2b5ab3604b8fa25def34711efa1a => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:01.435593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.436369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.436481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.480387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a3b688079959490ba1dffa69d196e366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.487497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a3b688079959490ba1dffa69d196e366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.508188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:01.509568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:01.575732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b688079959490ba1dffa69d196e366', 'request_id': 'req-893a1f43-bbec-4cf3-a786-4a2106af7156'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.585665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b688079959490ba1dffa69d196e366', 'request_id': 'req-893a1f43-bbec-4cf3-a786-4a2106af7156'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.606005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b688079959490ba1dffa69d196e366', 'request_id': 'req-893a1f43-bbec-4cf3-a786-4a2106af7156'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.623100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] The token cache is being invalidate because project a3b688079959490ba1dffa69d196e366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:01.623651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.625281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.638870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a3b688079959490ba1dffa69d196e366', 'request_id': 'req-893a1f43-bbec-4cf3-a786-4a2106af7156'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.640798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a3b688079959490ba1dffa69d196e366', 'request_id': 'req-893a1f43-bbec-4cf3-a786-4a2106af7156'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.641586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a3b688079959490ba1dffa69d196e366', 'request_id': 'req-893a1f43-bbec-4cf3-a786-4a2106af7156'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:01.655362 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2273/4560] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:05:01 2022] DELETE /identity/v3/domains/a3b688079959490ba1dffa69d196e366 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:01.659002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:01.660876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:01.724052 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2288/4561] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:01.738202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:01.739792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:01.816383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a1f43-bbec-4cf3-a786-4a2106af7156 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.823404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.824112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.824401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.824681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.825318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:01.827811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:01.848383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:01.856093 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2274/4562] 10.222.0.33 () {58 vars in 1128 bytes} [Sat Sep 10 21:05:01 2022] GET /identity/v3/domains => generated 1348 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:01.860254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:01.868757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78372e09-9a78-433d-81e6-8a469a7d3546 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.885491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9361dd-2588-4489-9876-d28ff89a3cba tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.895223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.896000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.896363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.901109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/a3b688079959490ba1dffa69d196e366` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.919155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find domain: a3b688079959490ba1dffa69d196e366.: keystone.exception.DomainNotFound: Could not find domain: a3b688079959490ba1dffa69d196e366. Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a3b688079959490ba1dffa69d196e366. Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:01.927838 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:01.929033 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: a3b688079959490ba1dffa69d196e366. Sep 10 21:05:01.930076 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:05:01.931459 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2275/4563] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:01 2022] PATCH /identity/v3/domains/a3b688079959490ba1dffa69d196e366 => generated 112 bytes in 71 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:05:01.935717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:01.953468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.954171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9061f-a825-477a-969a-191025bc5fa9 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:01.957032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.957600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.958029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.976258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:01.976803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:01.977088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:01.977366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:01.979037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:01.982349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:02.002512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:02.017326 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2276/4564] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:05:01 2022] POST /identity/v3/domains => generated 277 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:02.019702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:02.094055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:02.095778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:02.132377 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2289/4565] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:02.153397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9954430-f0ce-4d5a-814e-d0f2187278b1 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:02.161861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:02.162688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:02.162978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:02.163260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:02.163912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:02.167629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:02.225648 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2277/4566] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:02 2022] POST /identity/v3/users => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:02.230780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:02.251329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2522ff-d3c5-4b59-ad57-d123001025e9 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:02.256450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:02.257081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:02.257368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:02.257646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:02.258318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:02.259402 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:02.259402 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:02.259402 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:02.263747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:02.300340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2290/4567] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:02 2022] POST /identity/v3/groups => generated 323 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:02.304725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:02.320817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d092e4-d7d1-47ae-a517-abbfd1feeb43 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:02.330184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:02.331076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:02.331444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:02.331827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/d6492bc2a33544fea837f8b70e19b13e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:02.333383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d6492bc2a33544fea837f8b70e19b13e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:02.336303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:02.369941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:02.371255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:02.414718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:02.417082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-57bf68b4-c021-49bb-8800-21f18154e688 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:02.422837 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2278/4568] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:02 2022] PATCH /identity/v3/domains/d6492bc2a33544fea837f8b70e19b13e => generated 278 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:02.424167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:02.426275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:02.442770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b897a1d-0006-4c6e-bb7a-5da28ed5868b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:02.552406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9dafe-7781-441d-9e9a-0a4e43b52e94 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:02.552748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9dafe-7781-441d-9e9a-0a4e43b52e94 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:02.552990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9dafe-7781-441d-9e9a-0a4e43b52e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:02.608744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:02.609417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:02.609710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:02.610137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/d6492bc2a33544fea837f8b70e19b13e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:02.624799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d6492bc2a33544fea837f8b70e19b13e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:02.625802 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:02.625802 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:02.625802 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:02.631306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:02.632362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-63d9dafe-7781-441d-9e9a-0a4e43b52e94 None None] MFA Rules not processed for user `173f60c866c8453a97d2701fee66bfb5`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:02.634869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-63d9dafe-7781-441d-9e9a-0a4e43b52e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:02.680224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:02.695930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:02.700845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:02.850852 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2279/4569] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:05:02.867464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:02.869518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:02.869940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:02.923106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] MFA Rules not processed for user `b766ab7d168745a4bb8b07fb8e5df3ed`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:02.927948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:02.981992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:03.053980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd97f49ffaa4370a250b10331385d4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.069986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2280/4570] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:03.069986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.079219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:03.088842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd97f49ffaa4370a250b10331385d4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.098919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd97f49ffaa4370a250b10331385d4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.112337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd97f49ffaa4370a250b10331385d4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.113154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd97f49ffaa4370a250b10331385d4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.193906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b515916-5322-47cd-998e-f0ec4632231f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:03.211510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e', 'request_id': 'req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.220518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e', 'request_id': 'req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.234735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e', 'request_id': 'req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.262422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] The token cache is being invalidate because project d6492bc2a33544fea837f8b70e19b13e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:03.262825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.263582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.290174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:03.291515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=74,keystone:DELETE=24,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:03.292719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.294190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.295373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] PATH_INFO: `/v3/projects/10a007de9bc9437398bf90ae404cbef6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.296434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e', 'request_id': 'req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.298426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e', 'request_id': 'req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.301405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a007de9bc9437398bf90ae404cbef6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:03.304425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:03.306920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd6492bc2a33544fea837f8b70e19b13e', 'request_id': 'req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.316911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2291/4571] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:05:02 2022] DELETE /identity/v3/domains/d6492bc2a33544fea837f8b70e19b13e => generated 0 bytes in 893 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:03.320962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.323445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:03.407186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a007de9bc9437398bf90ae404cbef6', 'request_id': 'req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.419150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a007de9bc9437398bf90ae404cbef6', 'request_id': 'req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.430362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a007de9bc9437398bf90ae404cbef6', 'request_id': 'req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.441429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d970bb-d12a-4d5a-b574-e435b7dd7bc7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:03.451189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] The token cache is being invalidate because project 10a007de9bc9437398bf90ae404cbef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:03.452056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.453039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:03.458737 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2281/4572] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:03 2022] DELETE /identity/v3/projects/10a007de9bc9437398bf90ae404cbef6 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:03.458737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.462001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:03.488348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:03.569202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:03.570213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.570710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.571219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.574303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:03.582332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bc9d2-0b57-4b70-9186-1b53be06ac5e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:03.590001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bcbd8-1147-4bbb-be48-af0c15ca8e6a tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:03.590654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcbd8-1147-4bbb-be48-af0c15ca8e6a tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.590932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcbd8-1147-4bbb-be48-af0c15ca8e6a tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.591212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bcbd8-1147-4bbb-be48-af0c15ca8e6a tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/d6492bc2a33544fea837f8b70e19b13e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.606010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bcbd8-1147-4bbb-be48-af0c15ca8e6a tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=d6492bc2a33544fea837f8b70e19b13e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:03.608494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bcbd8-1147-4bbb-be48-af0c15ca8e6a tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-db4bcbd8-1147-4bbb-be48-af0c15ca8e6a tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find domain: d6492bc2a33544fea837f8b70e19b13e.: keystone.exception.DomainNotFound: Could not find domain: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:03.632509 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:03.633461 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:03.634460 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:05:03.635550 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2282/4573] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:03 2022] GET /identity/v3/domains/d6492bc2a33544fea837f8b70e19b13e => generated 112 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:05:03.636958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc5b8e-df63-418e-ae5c-c93f51fdc343 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.637561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc5b8e-df63-418e-ae5c-c93f51fdc343 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.637665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc5b8e-df63-418e-ae5c-c93f51fdc343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.662452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ccfc5b8e-df63-418e-ae5c-c93f51fdc343 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:03.663723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ccfc5b8e-df63-418e-ae5c-c93f51fdc343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:03.687538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:03.689417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:03.716567 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2292/4574] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:03 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:03.717841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.752084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b8a91-4f74-4653-93a7-66f17f43b779 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:03.753655 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2283/4575] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:03.755893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe50de1-34fd-43af-a8f9-69a92ce6867b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.756413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe50de1-34fd-43af-a8f9-69a92ce6867b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.756902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe50de1-34fd-43af-a8f9-69a92ce6867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.757647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:03.766462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.766811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.767104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/users/6cd97f49ffaa4370a250b10331385d4a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.786158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:get_user(user_id=6cd97f49ffaa4370a250b10331385d4a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:03.787462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0fe50de1-34fd-43af-a8f9-69a92ce6867b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:03.788093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:03.789286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0fe50de1-34fd-43af-a8f9-69a92ce6867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:03.803927 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2284/4576] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:03.805502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe50de1-34fd-43af-a8f9-69a92ce6867b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.823091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe50de1-34fd-43af-a8f9-69a92ce6867b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:03.830240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:03.831129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.831519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.831912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.832740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:03.833155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:03.835277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:03.869151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2285/4577] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:03.874830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.886463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbaedd-f921-4632-a5a2-efd70e7ebc59 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:03.891567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:03.892164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.892447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.892729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.893545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find user: 6cd97f49ffaa4370a250b10331385d4a.: keystone.exception.UserNotFound: Could not find user: 6cd97f49ffaa4370a250b10331385d4a. Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 21:05:03.894849 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:03.895844 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 6cd97f49ffaa4370a250b10331385d4a. Sep 10 21:05:03.896862 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:03.897589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:03.901347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2293/4578] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:05:03 2022] GET /identity/v3/users/6cd97f49ffaa4370a250b10331385d4a => generated 110 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:05:03.907322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.939074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce6331-6d29-47b9-bdc7-c50a71968bf2 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:03.947797 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2286/4579] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:03.952217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:03.956372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:03.958398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:03.958777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:03.960924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/groups/aad6d458aac14e1bb3896709d68e8f6b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:03.985239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:get_group(group_id=aad6d458aac14e1bb3896709d68e8f6b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:03.986965 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:03.986965 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:03.986965 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:03.988140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.002974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find group: aad6d458aac14e1bb3896709d68e8f6b.: keystone.exception.GroupNotFound: Could not find group: aad6d458aac14e1bb3896709d68e8f6b. Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_group(group_id) Sep 10 21:05:04.019952 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:04.021129 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: aad6d458aac14e1bb3896709d68e8f6b. Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.022062 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2294/4580] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:03 2022] GET /identity/v3/groups/aad6d458aac14e1bb3896709d68e8f6b => generated 111 bytes in 117 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:05:04.025785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.051470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25d74e-01c6-41e0-9741-71056b33a5f0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.155217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e39d7-3e71-49b4-be83-e81fed9d7089 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.176038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.176680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.176969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.177249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.178893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.187756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.203495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.204523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.205039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.205516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/groups/aad6d458aac14e1bb3896709d68e8f6b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.210137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:04.211237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find group: aad6d458aac14e1bb3896709d68e8f6b.: keystone.exception.GroupNotFound: Could not find group: aad6d458aac14e1bb3896709d68e8f6b. Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Sep 10 21:05:04.216966 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:04.217731 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: aad6d458aac14e1bb3896709d68e8f6b. Sep 10 21:05:04.218554 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.220923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2295/4581] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:04 2022] DELETE /identity/v3/groups/aad6d458aac14e1bb3896709d68e8f6b => generated 111 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:05:04.225646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.235046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2287/4582] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:03 2022] POST /identity/v3/users => generated 285 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:04.239617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc3f0-1ff3-419b-8d7d-a7f235db185b tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.240156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.245499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.246399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.246830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.247250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/users/6cd97f49ffaa4370a250b10331385d4a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.257813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_user(user_id=6cd97f49ffaa4370a250b10331385d4a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.261035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.268215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b2efe-13c8-44de-aff5-ff3aed3e274e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find user: 6cd97f49ffaa4370a250b10331385d4a.: keystone.exception.UserNotFound: Could not find user: 6cd97f49ffaa4370a250b10331385d4a. Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Sep 10 21:05:04.280485 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:04.281409 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 6cd97f49ffaa4370a250b10331385d4a. Sep 10 21:05:04.282399 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.286059 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2296/4583] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:04 2022] DELETE /identity/v3/users/6cd97f49ffaa4370a250b10331385d4a => generated 110 bytes in 61 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:05:04.287541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.288735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.288735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.288890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.289474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.290823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.296030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.305136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efdebda-2c7f-4b4e-b72e-5c94e58ca0a0 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.314204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.315119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.315715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.316130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/d6492bc2a33544fea837f8b70e19b13e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.333663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:05:04.339976 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2288/4584] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find domain: d6492bc2a33544fea837f8b70e19b13e.: keystone.exception.DomainNotFound: Could not find domain: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:04.341052 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:04.342026 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:04.342900 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.343967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.345704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2297/4585] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:04 2022] PATCH /identity/v3/domains/d6492bc2a33544fea837f8b70e19b13e => generated 112 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:05:04.351321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.361814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50446b2-d113-47be-9368-d6e20a0fad52 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.365975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85ffe-9716-45a1-bae0-a6ceec6722d5 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.369155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.371214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.371807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.372095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.373777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.374626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.374742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.375183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.375605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.377388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.377507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.380641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.393717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2289/4586] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:04.395160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.398719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:04.408450 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2298/4587] 10.222.0.33 () {58 vars in 1128 bytes} [Sat Sep 10 21:05:04 2022] GET /identity/v3/domains => generated 1348 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:04.410345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.449709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04902e-e9c8-4436-8c18-558a6d78394d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.458475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.460144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.460613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.461071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] PATH_INFO: `/v3/projects/9665812469c1426691889fca6f3c7be2/users/30a110f84c5346c6a5b6e3416bfb1f67/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.486894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9665812469c1426691889fca6f3c7be2, user_id=30a110f84c5346c6a5b6e3416bfb1f67, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.491370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.507739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d2ec6-1f4c-4a06-bc7f-009265306b1c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.517903 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2299/4588] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:04 2022] PUT /identity/v3/projects/9665812469c1426691889fca6f3c7be2/users/30a110f84c5346c6a5b6e3416bfb1f67/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:04.519282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.623433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67b06f-9b4e-4f5d-9aa6-aa259c9dd18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.624061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67b06f-9b4e-4f5d-9aa6-aa259c9dd18b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.624743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67b06f-9b4e-4f5d-9aa6-aa259c9dd18b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.625034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67b06f-9b4e-4f5d-9aa6-aa259c9dd18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.628035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1d67b06f-9b4e-4f5d-9aa6-aa259c9dd18b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:04.636908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49818eb-9b4d-4ef5-b80a-9a65061cfcb7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.643165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.643327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67b06f-9b4e-4f5d-9aa6-aa259c9dd18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.644208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.644975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67b06f-9b4e-4f5d-9aa6-aa259c9dd18b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.645377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.645979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.646842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:04.647348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.651034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.660140 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2290/4589] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:04.661641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.662151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.662367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.691802 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2300/4590] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:04 2022] GET /identity/v3/domains?enabled=True => generated 1095 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:04.698769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.720360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be6cda-0a8e-41de-bbb1-512bcfeddc1d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.729466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] MFA Rules not processed for user `30a110f84c5346c6a5b6e3416bfb1f67`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:04.731086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:04.732906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.736490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.737061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.738043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.739378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-2108721059) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:04.740001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.741935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.764977 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2301/4591] 10.222.0.33 () {58 vars in 1199 bytes} [Sat Sep 10 21:05:04 2022] GET /identity/v3/domains?name=tempest-test_domain-2108721059 => generated 409 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:04.771060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.801382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9f3cc-a5b2-491b-933d-46295bea195c tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.817329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.819188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.819695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.820180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/d6492bc2a33544fea837f8b70e19b13e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.840860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Could not find domain: d6492bc2a33544fea837f8b70e19b13e.: keystone.exception.DomainNotFound: Could not find domain: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:05:04.854932 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:05:04.856676 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d6492bc2a33544fea837f8b70e19b13e. Sep 10 21:05:04.857942 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:05:04.864025 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2302/4592] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:04 2022] PATCH /identity/v3/domains/d6492bc2a33544fea837f8b70e19b13e => generated 112 bytes in 94 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:05:04.868147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.881299 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2291/4593] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:04.887873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b46758-637d-486e-81ee-858ba140e9ae tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.891905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:04.896729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:04.919807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.923158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.924626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.925868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/c56424ccd62b405699b6ebb544a09728` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.928772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091ea89-97f5-4215-be00-0de620cb3863 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:04.942667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:04.944768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:04.952164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:04.952642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:04.981289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:04.983916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c56424ccd62b405699b6ebb544a09728)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.986447 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:04.986447 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:04.986447 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:04.987531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:04.989237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:04.989858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.003086 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2292/4594] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:05.086629 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2303/4595] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:04 2022] PATCH /identity/v3/domains/c56424ccd62b405699b6ebb544a09728 => generated 277 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:05:05.086812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.100491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:05.113914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceb025-09e0-421c-ae28-1a594f964027 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.145894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.147045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.147243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.147744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/c56424ccd62b405699b6ebb544a09728` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.166275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c56424ccd62b405699b6ebb544a09728)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.168652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.224121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c56424ccd62b405699b6ebb544a09728'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.233964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c56424ccd62b405699b6ebb544a09728'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.355882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56424ccd62b405699b6ebb544a09728', 'request_id': 'req-fc0052b2-1a1a-4e36-9405-250ff4c00d69'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.378069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56424ccd62b405699b6ebb544a09728', 'request_id': 'req-fc0052b2-1a1a-4e36-9405-250ff4c00d69'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.402520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56424ccd62b405699b6ebb544a09728', 'request_id': 'req-fc0052b2-1a1a-4e36-9405-250ff4c00d69'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.481283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] The token cache is being invalidate because project c56424ccd62b405699b6ebb544a09728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:05.484963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.488119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.523718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c56424ccd62b405699b6ebb544a09728', 'request_id': 'req-fc0052b2-1a1a-4e36-9405-250ff4c00d69'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.525734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c56424ccd62b405699b6ebb544a09728', 'request_id': 'req-fc0052b2-1a1a-4e36-9405-250ff4c00d69'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.531650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c56424ccd62b405699b6ebb544a09728', 'request_id': 'req-fc0052b2-1a1a-4e36-9405-250ff4c00d69'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.577941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2293/4596] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:05:05 2022] DELETE /identity/v3/domains/c56424ccd62b405699b6ebb544a09728 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:05.582192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.583913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:05.598395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec5284-6a6d-4ffb-a6fc-1fd79e875fd7 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:05.710821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:05.711497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:05.711785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:05.712741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/88fdd196d3ef4b4181c37f64188097f3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:05.765598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=88fdd196d3ef4b4181c37f64188097f3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:05.779341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:05.848506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.849575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.929636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.930581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:05.941198 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2304/4597] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:05 2022] PATCH /identity/v3/domains/88fdd196d3ef4b4181c37f64188097f3 => generated 277 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:05.948352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:05.964626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0052b2-1a1a-4e36-9405-250ff4c00d69 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:06.190644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:06.191529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:06.193167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:06.195223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/88fdd196d3ef4b4181c37f64188097f3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:06.240772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=88fdd196d3ef4b4181c37f64188097f3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:06.257085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:06.327070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.331585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.334616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:06.516263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3', 'request_id': 'req-e56619dc-1b6b-4e2d-b454-161e229b6444'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.530687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3', 'request_id': 'req-e56619dc-1b6b-4e2d-b454-161e229b6444'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.543980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3', 'request_id': 'req-e56619dc-1b6b-4e2d-b454-161e229b6444'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.568788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] The token cache is being invalidate because project 88fdd196d3ef4b4181c37f64188097f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:06.569087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.573469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.586651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3', 'request_id': 'req-e56619dc-1b6b-4e2d-b454-161e229b6444'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.588119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3', 'request_id': 'req-e56619dc-1b6b-4e2d-b454-161e229b6444'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.589398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '88fdd196d3ef4b4181c37f64188097f3', 'request_id': 'req-e56619dc-1b6b-4e2d-b454-161e229b6444'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.614053 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2294/4598] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:05:05 2022] DELETE /identity/v3/domains/88fdd196d3ef4b4181c37f64188097f3 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:06.616095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:06.620786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:06.651911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161063fd-4ffa-4dd0-a802-6198b005035d tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:06.798360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:06.799396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:06.800329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:06.800822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/be53c5deec174395af41e3a7f211994a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:06.824631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=be53c5deec174395af41e3a7f211994a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:06.828218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:06.870460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:06.890913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'be53c5deec174395af41e3a7f211994a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.893158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'be53c5deec174395af41e3a7f211994a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.986291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'be53c5deec174395af41e3a7f211994a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.988887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'be53c5deec174395af41e3a7f211994a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:06.997915 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2305/4599] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:06 2022] PATCH /identity/v3/domains/be53c5deec174395af41e3a7f211994a => generated 278 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:07.002686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:07.004676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:07.020727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56619dc-1b6b-4e2d-b454-161e229b6444 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:07.228626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cb0d3009f244a8a979c6f27e7779257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a3250da49b7427b960a8bbf2f7678be', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:07.230031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:07.230318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:07.230597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] PATH_INFO: `/v3/domains/be53c5deec174395af41e3a7f211994a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:07.255355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=be53c5deec174395af41e3a7f211994a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:07.257447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:07.309292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'be53c5deec174395af41e3a7f211994a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.319030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'be53c5deec174395af41e3a7f211994a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.403104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:07.418332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be53c5deec174395af41e3a7f211994a', 'request_id': 'req-f946be78-3fd8-4ec0-bca0-d2aebafb5868'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.450053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be53c5deec174395af41e3a7f211994a', 'request_id': 'req-f946be78-3fd8-4ec0-bca0-d2aebafb5868'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.470331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be53c5deec174395af41e3a7f211994a', 'request_id': 'req-f946be78-3fd8-4ec0-bca0-d2aebafb5868'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.500866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] The token cache is being invalidate because project be53c5deec174395af41e3a7f211994a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:07.501193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.503122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.528672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'be53c5deec174395af41e3a7f211994a', 'request_id': 'req-f946be78-3fd8-4ec0-bca0-d2aebafb5868'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.530148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'be53c5deec174395af41e3a7f211994a', 'request_id': 'req-f946be78-3fd8-4ec0-bca0-d2aebafb5868'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.532315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f946be78-3fd8-4ec0-bca0-d2aebafb5868 tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'be53c5deec174395af41e3a7f211994a', 'request_id': 'req-f946be78-3fd8-4ec0-bca0-d2aebafb5868'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:07.573592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2295/4600] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:05:07 2022] DELETE /identity/v3/domains/be53c5deec174395af41e3a7f211994a => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:07.577489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:07.579371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:07.747997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19102808-53b0-43b7-a093-82a327feaecb tempest-DomainsTestJSON-9182862 tempest-DomainsTestJSON-9182862-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:07.972430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:07.973731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:07.973731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:07.973731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] PATH_INFO: `/v3/users/2adbe03bd6e843a48a56ec31098b78b3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:07.980251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2adbe03bd6e843a48a56ec31098b78b3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:07.981887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:08.126620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2adbe03bd6e843a48a56ec31098b78b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:08.190297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2adbe03bd6e843a48a56ec31098b78b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:08.211577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2adbe03bd6e843a48a56ec31098b78b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:08.247802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2adbe03bd6e843a48a56ec31098b78b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:08.250428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2adbe03bd6e843a48a56ec31098b78b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:08.266128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2306/4601] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:07 2022] DELETE /identity/v3/users/2adbe03bd6e843a48a56ec31098b78b3 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:08.272287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998f19-5ba1-4a17-90ef-bae49920296f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:08.273132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998f19-5ba1-4a17-90ef-bae49920296f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:08.273557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998f19-5ba1-4a17-90ef-bae49920296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:08.338054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-54998f19-5ba1-4a17-90ef-bae49920296f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:08.339918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-54998f19-5ba1-4a17-90ef-bae49920296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:08.527516 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2296/4602] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:08.557461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:08.560534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:08.583156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:08.782411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d1a6f-6f62-4325-9c5d-360275f81e0b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:08.791086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:08.792221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:08.792723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:08.793169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:08.807447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:08.816054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:08.834510 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2307/4603] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:10.210793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:10.212526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:10.213930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:10.291194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:10.293298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:10.425867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2297/4604] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:10.512119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:10.513850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:10.664106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f08bd2-171b-4b7b-8c50-8531a658ffa1 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:10.671663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:10.672597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:10.673041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:10.673425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:10.684884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:10.687237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:10.702853 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2308/4605] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:05:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:05:11.064591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:11.066206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:11.100133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4335916-94be-4a6a-b079-fc907674c5fe None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:11.107004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:11.107657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:11.107944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:11.108221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] PATH_INFO: `/v3/projects/64e1934d881847cf972681f24ecfa2dc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:11.111236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e1934d881847cf972681f24ecfa2dc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:11.113330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:11.192384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e1934d881847cf972681f24ecfa2dc', 'request_id': 'req-616fe356-02ad-4b73-93bd-9b82d140ec80'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.193670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:11.204811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e1934d881847cf972681f24ecfa2dc', 'request_id': 'req-616fe356-02ad-4b73-93bd-9b82d140ec80'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.221441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e1934d881847cf972681f24ecfa2dc', 'request_id': 'req-616fe356-02ad-4b73-93bd-9b82d140ec80'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.244975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] The token cache is being invalidate because project 64e1934d881847cf972681f24ecfa2dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:11.245686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.249122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.261638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:11.261934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2298/4606] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:11 2022] DELETE /identity/v3/projects/64e1934d881847cf972681f24ecfa2dc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:11.278403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71861c4-c5d3-4f68-be23-d688d6eea9c1 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:11.381045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:11.381925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:11.382390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:11.382772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] PATH_INFO: `/v3/users/9cb0d3009f244a8a979c6f27e7779257` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:11.388479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb0d3009f244a8a979c6f27e7779257)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:11.392769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:11.467056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:11.496179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb0d3009f244a8a979c6f27e7779257'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.522972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb0d3009f244a8a979c6f27e7779257'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.541441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb0d3009f244a8a979c6f27e7779257'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.557717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb0d3009f244a8a979c6f27e7779257'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.559268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-db6e5bec-842a-424c-8185-7c84cc1a2629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb0d3009f244a8a979c6f27e7779257'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:11.573458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2309/4607] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:11 2022] DELETE /identity/v3/users/9cb0d3009f244a8a979c6f27e7779257 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:14.058285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:14.060695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:14.093017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe356-02ad-4b73-93bd-9b82d140ec80 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:14.250687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:14.251844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:14.252449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:14.253503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] PATH_INFO: `/v3/projects/0a3250da49b7427b960a8bbf2f7678be` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:14.257482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3250da49b7427b960a8bbf2f7678be)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:14.262213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:14.380233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3250da49b7427b960a8bbf2f7678be', 'request_id': 'req-7f570018-404f-4007-a559-6f2c808f5597'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:14.395242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3250da49b7427b960a8bbf2f7678be', 'request_id': 'req-7f570018-404f-4007-a559-6f2c808f5597'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:14.422898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3250da49b7427b960a8bbf2f7678be', 'request_id': 'req-7f570018-404f-4007-a559-6f2c808f5597'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:14.441660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] The token cache is being invalidate because project 0a3250da49b7427b960a8bbf2f7678be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:14.442128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:14.444327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7f570018-404f-4007-a559-6f2c808f5597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:14.448962 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2299/4608] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:14 2022] DELETE /identity/v3/projects/0a3250da49b7427b960a8bbf2f7678be => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:14.453169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb255f-2001-4e32-b2fe-050942542a69 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:14.453792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb255f-2001-4e32-b2fe-050942542a69 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:14.455472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb255f-2001-4e32-b2fe-050942542a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:14.515230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-05bb255f-2001-4e32-b2fe-050942542a69 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:14.518250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-05bb255f-2001-4e32-b2fe-050942542a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:14.730497 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2310/4609] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:14.738044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8712c-66c0-4552-941f-02cd5caec1e1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:14.738286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8712c-66c0-4552-941f-02cd5caec1e1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:14.738571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8712c-66c0-4552-941f-02cd5caec1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:14.774918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-23b8712c-66c0-4552-941f-02cd5caec1e1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:14.776473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23b8712c-66c0-4552-941f-02cd5caec1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:14.803722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2300/4610] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:14.807999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb255f-2001-4e32-b2fe-050942542a69 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:14.822803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb255f-2001-4e32-b2fe-050942542a69 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:14.839079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347949-5888-4563-913c-1972e446a618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:14.839643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347949-5888-4563-913c-1972e446a618 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:14.839938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347949-5888-4563-913c-1972e446a618 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:14.840581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347949-5888-4563-913c-1972e446a618 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:14.841256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347949-5888-4563-913c-1972e446a618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:14.841569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347949-5888-4563-913c-1972e446a618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:14.846300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347949-5888-4563-913c-1972e446a618 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:14.872993 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2311/4611] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:14.879767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8712c-66c0-4552-941f-02cd5caec1e1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:14.892784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8712c-66c0-4552-941f-02cd5caec1e1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:14.905918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:14.906539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:14.906837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:14.907102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:14.907735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:14.909660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:14.992386 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2301/4612] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:14 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:14.997283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347949-5888-4563-913c-1972e446a618 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:15.010929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347949-5888-4563-913c-1972e446a618 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:15.176017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:15.176705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:15.177052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:15.177331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:15.178222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:15.180347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:15.237807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:15.253544 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2312/4613] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:14 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:15.257567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9582fd7-6204-47db-8d5c-d5a8eaced69f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:15.264261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:15.265399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:15.265876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:15.266515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:15.267460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:15.270121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:15.300311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:15.308917 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2302/4614] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:15.313930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d3d49-3ab5-46c9-8f3c-c2d13649e9b8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:15.325797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76484-006c-43c3-a19a-57136fc1caf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:15.327230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76484-006c-43c3-a19a-57136fc1caf5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:15.332787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76484-006c-43c3-a19a-57136fc1caf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:15.333436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76484-006c-43c3-a19a-57136fc1caf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:15.334913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee76484-006c-43c3-a19a-57136fc1caf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:15.338165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee76484-006c-43c3-a19a-57136fc1caf5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:15.369985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2313/4615] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:15.375031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:15.394888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020acc18-366a-43a8-87e4-cbf20840f709 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:15.417957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:15.418645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:15.418932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:15.419227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] PATH_INFO: `/v3/projects/77ee365fc4464b26aa3c79c302e02266/users/5a4626bcd5814e8b84c6f948692d41d5/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:15.455363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ee365fc4464b26aa3c79c302e02266, user_id=5a4626bcd5814e8b84c6f948692d41d5, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:15.458706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:15.487771 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2303/4616] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:15 2022] PUT /identity/v3/projects/77ee365fc4464b26aa3c79c302e02266/users/5a4626bcd5814e8b84c6f948692d41d5/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:15.501857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091a96-958c-4e86-8865-83634f605ad4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:15.502502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091a96-958c-4e86-8865-83634f605ad4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:15.502502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091a96-958c-4e86-8865-83634f605ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:15.565550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-55091a96-958c-4e86-8865-83634f605ad4 None None] MFA Rules not processed for user `5a4626bcd5814e8b84c6f948692d41d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:15.566959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-55091a96-958c-4e86-8865-83634f605ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:15.588885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:15.747798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2314/4617] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:15.747994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:15.754156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:15.878619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf6429-7f54-4ec0-afb1-7283202b6239 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:15.886389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:15.887328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:15.887888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:15.888334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:15.889391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:15.891800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:15.960418 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2304/4618] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:15 2022] POST /identity/v3/projects => generated 367 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:15.964662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091a96-958c-4e86-8865-83634f605ad4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:15.987587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091a96-958c-4e86-8865-83634f605ad4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:16.204457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:16.205273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:16.205638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:16.206078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:16.207138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:16.209215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:16.261219 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2315/4619] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:15 2022] POST /identity/v3/users => generated 290 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:16.262138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:16.277707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2436c6-1dcc-42c2-849f-1f8a54bc8d60 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:16.293275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:16.294236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:16.294729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:16.295224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:16.296648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:16.298607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:16.335584 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2305/4620] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:16.339672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:16.355209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e485ea-abfc-4d53-92fd-9176b4135045 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:16.374896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:16.375594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:16.375875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:16.376154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] PATH_INFO: `/v3/projects/3d852fd39dac4e5aac8a467f0b911eb6/users/e78c1c000f51483486a94183100c5d3d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:16.427744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d852fd39dac4e5aac8a467f0b911eb6, user_id=e78c1c000f51483486a94183100c5d3d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:16.431411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:16.461226 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2316/4621] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:16 2022] PUT /identity/v3/projects/3d852fd39dac4e5aac8a467f0b911eb6/users/e78c1c000f51483486a94183100c5d3d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:16.466728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:16.485768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0f486-2be9-4f0d-9951-bc7b9af85764 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:16.684831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3728fec-d7dd-481e-b0dd-2e9be5747625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:16.685795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3728fec-d7dd-481e-b0dd-2e9be5747625 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:16.686385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3728fec-d7dd-481e-b0dd-2e9be5747625 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:16.686890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3728fec-d7dd-481e-b0dd-2e9be5747625 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:16.687616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3728fec-d7dd-481e-b0dd-2e9be5747625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:16.689972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3728fec-d7dd-481e-b0dd-2e9be5747625 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:16.712826 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2306/4622] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:16.715274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:16.731521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa40303-5421-4cc9-94fa-c77b5354e249 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:16.740823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:16.741789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:16.744220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:16.744603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] PATH_INFO: `/v3/projects/3d852fd39dac4e5aac8a467f0b911eb6/users/e78c1c000f51483486a94183100c5d3d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:16.746963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d852fd39dac4e5aac8a467f0b911eb6, user_id=e78c1c000f51483486a94183100c5d3d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:16.750672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:16.767704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2317/4623] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:16 2022] PUT /identity/v3/projects/3d852fd39dac4e5aac8a467f0b911eb6/users/e78c1c000f51483486a94183100c5d3d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:16.865870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d05c7f-8da6-4415-98cf-c6ed7932bf7e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:16.866403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d05c7f-8da6-4415-98cf-c6ed7932bf7e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:16.866685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d05c7f-8da6-4415-98cf-c6ed7932bf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:16.935599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-09d05c7f-8da6-4415-98cf-c6ed7932bf7e None None] MFA Rules not processed for user `e78c1c000f51483486a94183100c5d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:16.937779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-09d05c7f-8da6-4415-98cf-c6ed7932bf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:17.114737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:17.128799 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2307/4624] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:17.129502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.150001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bd1fe-0428-40b1-9f3f-c160efad85b7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.158754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.159897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.161133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.161478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.162402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:17.164860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:17.211083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d05c7f-8da6-4415-98cf-c6ed7932bf7e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.215031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2318/4625] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:05:17 2022] POST /identity/v3/domains => generated 277 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:17.226705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d05c7f-8da6-4415-98cf-c6ed7932bf7e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.335153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.335780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.336062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.336343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.338495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:17.339543 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:17.339543 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:17.339543 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:17.342098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:17.391410 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2308/4626] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:17 2022] POST /identity/v3/groups => generated 279 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:17.391507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.406765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53f38f-9b4a-4947-92af-9cde4e3b7e36 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.412122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.412897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.413182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.413646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/6d4070771d514fcfbe1d02894c08264d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.422790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:update_group(group_id=6d4070771d514fcfbe1d02894c08264d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:17.424047 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:17.424047 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:17.424047 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:17.425315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:17.469269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.482468 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2319/4627] 10.222.0.33 () {60 vars in 1247 bytes} [Sat Sep 10 21:05:17 2022] PATCH /identity/v3/groups/6d4070771d514fcfbe1d02894c08264d => generated 291 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:05:17.484176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb84f24-ad14-4068-9a6f-8e9b9d1a1fec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.507789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.508545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.508930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.509397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/6d4070771d514fcfbe1d02894c08264d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.518365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:get_group(group_id=6d4070771d514fcfbe1d02894c08264d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:17.519407 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:17.519407 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:17.519407 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:17.520908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:17.527792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.541211 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2309/4628] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:17 2022] GET /identity/v3/groups/6d4070771d514fcfbe1d02894c08264d => generated 291 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:17.542542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b996f-0220-4482-8078-f498b9c8dca0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.551040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.552765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.552765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.552968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/6d4070771d514fcfbe1d02894c08264d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.554277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:update_group(group_id=6d4070771d514fcfbe1d02894c08264d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:17.558302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:17.582269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.584452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:17.610061 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2320/4629] 10.222.0.33 () {60 vars in 1246 bytes} [Sat Sep 10 21:05:17 2022] PATCH /identity/v3/groups/6d4070771d514fcfbe1d02894c08264d => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:17.611220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.638132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726d26-9c9f-4032-8b13-0a543bce2fe7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.652583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.653134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.653421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.656938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/6d4070771d514fcfbe1d02894c08264d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.672582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_group(group_id=6d4070771d514fcfbe1d02894c08264d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:17.675850 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:17.675850 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:17.675850 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:17.678987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:17.689470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:UPDATE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:17.823539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3eacd-8416-410b-9916-bdd31a905ee2 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.842382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2321/4630] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:17 2022] DELETE /identity/v3/groups/6d4070771d514fcfbe1d02894c08264d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:17.844736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:17.845272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.845956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.846742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.848008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.851830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:17.886819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:17.890994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:17.914364 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2310/4631] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:17.989273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5377f0-a19d-4294-b7d9-ea82b6ebd2f3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:17.995615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:17.996243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:17.996714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:17.996995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:17.997626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:18.000014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:18.032317 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2322/4632] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:17 2022] POST /identity/v3/groups => generated 325 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:18.034136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:18.052492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60b077-76a1-47a8-849f-8a42edf8f2be service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:18.071012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:18.072155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:18.072780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:18.073071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:18.074149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:18.078768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:18.143074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:18.154146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2311/4633] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:18 2022] POST /identity/v3/users => generated 321 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:18.156933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ace1a-a6c3-4751-b9d2-861ff8ff6b88 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:18.168686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:18.193181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:18.195707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:18.196030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/de020f1328984ab4a4aa014123fb7267` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:18.261522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=de020f1328984ab4a4aa014123fb7267)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:18.262959 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:18.262959 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:18.262959 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:18.265183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:18.366330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2323/4634] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:05:18 2022] PUT /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/de020f1328984ab4a4aa014123fb7267 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:18.368081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:18.391419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a1bf3-3ffb-4c90-aa01-d5d2a1abcf14 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:18.503221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:18.538598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:18.543964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:18.545004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:18.545124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:18.546965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:18.553191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:18.625639 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2312/4635] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:18 2022] POST /identity/v3/users => generated 321 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:18.630414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:18.684069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc0b2c-504b-44ac-85a4-e9c041ac2165 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:18.706255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:18.710832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:18.713236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:18.715315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/b312c34c155f4e66b923344f620df816` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:18.775203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=b312c34c155f4e66b923344f620df816)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:18.781069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:18.864354 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2324/4636] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:05:18 2022] PUT /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/b312c34c155f4e66b923344f620df816 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:18.866971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:18.891007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736601-9062-4d0d-87f4-90cd9d503e1a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:19.051993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:19.052627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:19.052949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:19.053324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:19.053996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:19.056588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:19.095458 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2313/4637] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:18 2022] POST /identity/v3/users => generated 321 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:19.099623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:19.113371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df30f0-d1bf-400c-817a-31efcf9bed62 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:19.121358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:19.122417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:19.122869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:19.123308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/41bd6a9b2073445d905ef5526ab759cb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:19.146673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=41bd6a9b2073445d905ef5526ab759cb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:19.148953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:19.187409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2325/4638] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:05:19 2022] PUT /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/41bd6a9b2073445d905ef5526ab759cb => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:19.189711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:19.220238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db18735-7238-4a53-ada4-2adc69018b82 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:19.382626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:19.386264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:19.388374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:19.388374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:19.394201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:19.405566 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:19.405566 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:19.405566 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:19.409645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:19.502697 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2314/4639] 10.222.0.33 () {58 vars in 1242 bytes} [Sat Sep 10 21:05:19 2022] GET /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users => generated 1079 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:19.507189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:19.520427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7217f7d-0147-463a-a7bd-eaeb9e25887d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:19.545333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:19.546065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:19.546359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:19.546651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/de020f1328984ab4a4aa014123fb7267` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:19.551481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=de020f1328984ab4a4aa014123fb7267)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:19.552521 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:19.552521 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:19.552521 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:19.553733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:19.572386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:19.620445 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2326/4640] 10.222.0.33 () {58 vars in 1342 bytes} [Sat Sep 10 21:05:19 2022] HEAD /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/de020f1328984ab4a4aa014123fb7267 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:19.622043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:19.634022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad553d4-59f7-46d5-a753-82521f3c4ce3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:19.638862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:19.639520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:19.639804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:19.640080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/de020f1328984ab4a4aa014123fb7267` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:19.641776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=de020f1328984ab4a4aa014123fb7267)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:19.642819 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:19.642819 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:19.642819 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:19.644014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:19.681795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'de020f1328984ab4a4aa014123fb7267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:19.722629 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2315/4641] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:05:19 2022] DELETE /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/de020f1328984ab4a4aa014123fb7267 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:19.725320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:19.744973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2f30-c194-44bb-83aa-716e0b7a3c9a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:19.893917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:19.895320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:19.895812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:19.896304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/b312c34c155f4e66b923344f620df816` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:19.898823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=b312c34c155f4e66b923344f620df816)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:19.902007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:19.937115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2327/4642] 10.222.0.33 () {58 vars in 1342 bytes} [Sat Sep 10 21:05:19 2022] HEAD /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/b312c34c155f4e66b923344f620df816 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:19.941358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:19.964224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad5968-ab27-4714-aca2-f3e564afc819 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:19.982523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:19.983610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:19.984645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:19.984936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/b312c34c155f4e66b923344f620df816` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:19.989744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=b312c34c155f4e66b923344f620df816)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:19.995679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:20.026239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b312c34c155f4e66b923344f620df816'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:20.068009 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2316/4643] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:05:19 2022] DELETE /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/b312c34c155f4e66b923344f620df816 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:20.078649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:20.093794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd67e5-a238-42bf-a5c2-f77d3521db40 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:20.281130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:20.285087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:20.285530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:20.286107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/41bd6a9b2073445d905ef5526ab759cb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:20.291298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=41bd6a9b2073445d905ef5526ab759cb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:20.298200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:20.329215 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2328/4644] 10.222.0.33 () {58 vars in 1342 bytes} [Sat Sep 10 21:05:20 2022] HEAD /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/41bd6a9b2073445d905ef5526ab759cb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:20.330079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:20.345683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a8701-d36d-4b89-8d85-a29fa8a2207e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:20.353354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:20.354218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:20.354704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:20.355142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/41bd6a9b2073445d905ef5526ab759cb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:20.360303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b, user_id=41bd6a9b2073445d905ef5526ab759cb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:20.362898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:20.381769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '41bd6a9b2073445d905ef5526ab759cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:20.396021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:INSERT=9,keystone:DELETE=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:20.431349 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2317/4645] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:05:20 2022] DELETE /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users/41bd6a9b2073445d905ef5526ab759cb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:20.436030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:20.456561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01862-37d6-4ab0-ab19-f9069d8dfe85 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:20.619777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:20.620448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:20.620738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:20.621015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:20.623353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:20.623566 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:20.623566 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:20.623566 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:20.625093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:20.671774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:20.672061 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2329/4646] 10.222.0.33 () {58 vars in 1242 bytes} [Sat Sep 10 21:05:20 2022] GET /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b/users => generated 145 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:20.692880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc818f8-e82e-4c2e-8093-c17866c883df tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:20.715810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:20.716517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:20.716762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:20.725404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users/41bd6a9b2073445d905ef5526ab759cb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:20.730174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_user(user_id=41bd6a9b2073445d905ef5526ab759cb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:20.746095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:20.881804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bd6a9b2073445d905ef5526ab759cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:20.918862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bd6a9b2073445d905ef5526ab759cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:20.922487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bd6a9b2073445d905ef5526ab759cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:20.967482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bd6a9b2073445d905ef5526ab759cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:20.968774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bd6a9b2073445d905ef5526ab759cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.025622 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2318/4647] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:20 2022] DELETE /identity/v3/users/41bd6a9b2073445d905ef5526ab759cb => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:21.029784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:21.033459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:21.059097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9174caf-6e6c-415e-a9d9-2fcbd01089a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:21.109160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:21.189685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:21.190526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:21.190906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:21.191932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users/b312c34c155f4e66b923344f620df816` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:21.194730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b312c34c155f4e66b923344f620df816)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:21.198375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:21.281488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b312c34c155f4e66b923344f620df816'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.304335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b312c34c155f4e66b923344f620df816'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.322665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b312c34c155f4e66b923344f620df816'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.333718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b312c34c155f4e66b923344f620df816'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.334469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b312c34c155f4e66b923344f620df816'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.347046 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2330/4648] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:21 2022] DELETE /identity/v3/users/b312c34c155f4e66b923344f620df816 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:21.351534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:21.369888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99efdc-c709-41f7-9d92-9c52d065b18c tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:21.446168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:21.447883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:21.448334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:21.449054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users/de020f1328984ab4a4aa014123fb7267` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:21.454369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_user(user_id=de020f1328984ab4a4aa014123fb7267)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:21.457321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:21.533961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de020f1328984ab4a4aa014123fb7267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.544412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de020f1328984ab4a4aa014123fb7267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.545311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de020f1328984ab4a4aa014123fb7267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.558431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de020f1328984ab4a4aa014123fb7267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.558989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de020f1328984ab4a4aa014123fb7267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:21.567998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:21.590853 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2319/4649] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:21 2022] DELETE /identity/v3/users/de020f1328984ab4a4aa014123fb7267 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:21.595002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:21.596560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:21.615966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74964b-7de0-4e4c-bf59-4c852689734e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:21.702291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:21.703090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:21.703378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:21.703712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:21.705370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_group(group_id=ec24aae65f5f44ea8aeabd5c7cc7fd9b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:21.707730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:21.770248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:21.790846 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2331/4650] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:21 2022] DELETE /identity/v3/groups/ec24aae65f5f44ea8aeabd5c7cc7fd9b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:21.794256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:21.798951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:21.831244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28927d55-0fc3-4ab7-81a1-fe410d6e6248 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:21.944361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:21.945002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:21.945297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:21.945592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:21.947176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:21.949183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:21.981779 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2320/4651] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:21 2022] POST /identity/v3/groups => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:21.988425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:22.008379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5657f-842a-4eb5-b087-0181f1acb6d3 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:22.022733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:22.023372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:22.023658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:22.023934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:22.025143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:22.027285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:22.051968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2332/4652] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:21 2022] POST /identity/v3/groups => generated 326 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:22.058265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:22.077286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbdd0a-f09a-4f7a-99ae-7561ae821b2d tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:22.083715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:22.084553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:22.084940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:22.085478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:22.087282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:22.090096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:22.122457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2321/4653] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:22 2022] POST /identity/v3/groups => generated 327 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:22.126116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:22.148320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ed8de-15b1-4e62-918f-f3a708a8fa3e tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:22.162386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:22.163022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:22.163304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:22.163579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:22.164227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:22.168114 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:22.168114 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:22.168114 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:22.168997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:22.198330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2333/4654] 10.222.0.33 () {58 vars in 1125 bytes} [Sat Sep 10 21:05:22 2022] GET /identity/v3/groups => generated 1490 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:22.198703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:22.218640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23031b-6aeb-449e-bc2e-e3e4157f56a7 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:22.227321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:22.227997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:22.228287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:22.228553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/d847b780cfe846f682b274431db2fb33` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:22.237757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_group(group_id=d847b780cfe846f682b274431db2fb33)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:22.237989 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:22.237989 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:22.237989 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:22.239194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:22.373453 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2322/4655] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:22 2022] DELETE /identity/v3/groups/d847b780cfe846f682b274431db2fb33 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:22.378063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:22.406270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfeaf8-4cea-4fb0-889e-a8c01f31631f tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:22.553459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:22.554157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:22.554449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:22.554734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/fde0ccd35f934061a93ce462f770b50c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:22.574820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_group(group_id=fde0ccd35f934061a93ce462f770b50c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:22.578640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:22.737780 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2334/4656] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:22 2022] DELETE /identity/v3/groups/fde0ccd35f934061a93ce462f770b50c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:22.741952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:22.775357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bedd2-1196-452d-a803-0183ebe65dec tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:22.942621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:22.943642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:22.944650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:22.944650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/59f9774ce9e648b4a4feaf8b7d46c552` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:22.959690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_group(group_id=59f9774ce9e648b4a4feaf8b7d46c552)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:22.962492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:22.985578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:23.092560 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2323/4657] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:22 2022] DELETE /identity/v3/groups/59f9774ce9e648b4a4feaf8b7d46c552 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:23.097648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:23.110832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51ac90-2540-485e-88d5-83ccca364802 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:23.256300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:23.257086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:23.257477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:23.257928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:23.258754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:23.260723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:23.320168 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2335/4658] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:23 2022] POST /identity/v3/users => generated 321 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:23.321885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:23.337788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21127-47ab-4d69-98b8-e873b38558a0 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:23.362591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:23.363448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:23.363998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:23.364515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:23.365656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:23.368977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:23.428109 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2324/4659] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:23 2022] POST /identity/v3/groups => generated 326 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:23.429802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:23.447597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dcca5-91e2-4b5c-bade-5195485ced38 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:23.464880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:23.467365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:23.468213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:23.468213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/062b1f20144f41e895ba5ac0a9beecf9/users/7b2f6ed3f5704ce99e30f06b6219eb83` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:23.504770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:23.526153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=062b1f20144f41e895ba5ac0a9beecf9, user_id=7b2f6ed3f5704ce99e30f06b6219eb83)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:23.528306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:23.580681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:23.589661 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2336/4660] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:05:23 2022] PUT /identity/v3/groups/062b1f20144f41e895ba5ac0a9beecf9/users/7b2f6ed3f5704ce99e30f06b6219eb83 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:23.597539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ff30-b1fd-4b07-91f9-e7e08c5ae121 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:23.780792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:23.782214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:23.782703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:23.783232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:23.784063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:23.792106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:23.860940 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2325/4661] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:23 2022] POST /identity/v3/groups => generated 325 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:23.862346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:23.876245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d515-efda-4833-9a58-713be1669a4a tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:23.887798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:23.888544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:23.888861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:23.889175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/7ed34e90ccb44772976cfb566fa3ec4c/users/7b2f6ed3f5704ce99e30f06b6219eb83` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:23.898756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=7ed34e90ccb44772976cfb566fa3ec4c, user_id=7b2f6ed3f5704ce99e30f06b6219eb83)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:23.919283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:23.978197 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2337/4662] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:05:23 2022] PUT /identity/v3/groups/7ed34e90ccb44772976cfb566fa3ec4c/users/7b2f6ed3f5704ce99e30f06b6219eb83 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:23.982438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:24.016462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef1cd7-a003-42be-bba7-5100b6e860f5 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:24.186603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:24.189345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:24.255859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:24.256873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:24.257319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:24.257744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users/7b2f6ed3f5704ce99e30f06b6219eb83/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:24.268012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=7b2f6ed3f5704ce99e30f06b6219eb83)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:24.271382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:24.318525 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2326/4663] 10.222.0.33 () {58 vars in 1242 bytes} [Sat Sep 10 21:05:23 2022] GET /identity/v3/users/7b2f6ed3f5704ce99e30f06b6219eb83/groups => generated 837 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:24.323530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:24.348475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb4763-2695-4be8-854a-8089ee42fe71 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:24.366799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:24.367536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:24.367773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:24.368885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/7ed34e90ccb44772976cfb566fa3ec4c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:24.371676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_group(group_id=7ed34e90ccb44772976cfb566fa3ec4c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:24.373051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:24.381391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a69b4-346f-43d1-b81f-f65001442be4 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:24.395942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:24.396636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:24.396942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:24.397204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:24.404769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:24.446211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:24.477091 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2327/4664] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:24 2022] DELETE /identity/v3/groups/7ed34e90ccb44772976cfb566fa3ec4c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:24.481743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:24.533394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:24.539279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:24.574561 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2338/4665] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:24 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:05:24.643144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270cb89-1117-4a89-ad90-2a20876b6d86 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:24.655949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:24.656620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:24.656907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:24.657185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/groups/062b1f20144f41e895ba5ac0a9beecf9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:24.658515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_group(group_id=062b1f20144f41e895ba5ac0a9beecf9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:24.660832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:24.776007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:24.776540 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2328/4666] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:24 2022] DELETE /identity/v3/groups/062b1f20144f41e895ba5ac0a9beecf9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:24.794753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e558-ce02-4fe8-a66d-8a8818b3ca58 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:24.950780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:24.974641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:24.975577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:24.975909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:24.976128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/users/7b2f6ed3f5704ce99e30f06b6219eb83` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:24.982943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_user(user_id=7b2f6ed3f5704ce99e30f06b6219eb83)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:24.984994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:25.124520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2f6ed3f5704ce99e30f06b6219eb83'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.146188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2f6ed3f5704ce99e30f06b6219eb83'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.176903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2f6ed3f5704ce99e30f06b6219eb83'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.202854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2f6ed3f5704ce99e30f06b6219eb83'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.203886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2f6ed3f5704ce99e30f06b6219eb83'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.219564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:25.220921 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2339/4667] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:24 2022] DELETE /identity/v3/users/7b2f6ed3f5704ce99e30f06b6219eb83 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:25.222294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:25.248468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d28aed-5a8b-4169-a1d8-b7e9dbbe9080 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:25.382948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:25.384733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:25.385168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:25.385602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/domains/c6554872c8704d63b34f49359c696a4e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:25.387245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c6554872c8704d63b34f49359c696a4e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:25.389151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:25.415370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c6554872c8704d63b34f49359c696a4e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.417558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c6554872c8704d63b34f49359c696a4e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.481185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c6554872c8704d63b34f49359c696a4e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.483312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c6554872c8704d63b34f49359c696a4e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.490203 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2329/4668] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:25 2022] PATCH /identity/v3/domains/c6554872c8704d63b34f49359c696a4e => generated 278 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:25.493215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:25.511160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86f24c-49c0-4726-8811-6ba869fe5086 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:25.653722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e78c1c000f51483486a94183100c5d3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d852fd39dac4e5aac8a467f0b911eb6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:25.654498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:25.654857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:25.655204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] PATH_INFO: `/v3/domains/c6554872c8704d63b34f49359c696a4e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:25.715223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c6554872c8704d63b34f49359c696a4e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:25.721036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:25.796337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c6554872c8704d63b34f49359c696a4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.809408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c6554872c8704d63b34f49359c696a4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.909297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:25.937806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6554872c8704d63b34f49359c696a4e', 'request_id': 'req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.953226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6554872c8704d63b34f49359c696a4e', 'request_id': 'req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:25.972053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6554872c8704d63b34f49359c696a4e', 'request_id': 'req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.107751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] The token cache is being invalidate because project c6554872c8704d63b34f49359c696a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:26.109366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.110385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.137602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c6554872c8704d63b34f49359c696a4e', 'request_id': 'req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.142035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c6554872c8704d63b34f49359c696a4e', 'request_id': 'req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.148913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c6554872c8704d63b34f49359c696a4e', 'request_id': 'req-e4ea8f15-ca15-4a20-85d1-21b5e3dc2044'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.181866 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2340/4669] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:05:25 2022] DELETE /identity/v3/domains/c6554872c8704d63b34f49359c696a4e => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:26.186389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:26.188712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:26.236143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:26.318180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e5de0-e726-44d9-96bd-c7273bcf9a90 tempest-GroupsV3TestJSON-200272452 tempest-GroupsV3TestJSON-200272452-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:26.435543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:26.435901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:26.436238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:26.436514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] PATH_INFO: `/v3/users/5a4626bcd5814e8b84c6f948692d41d5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:26.438518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a4626bcd5814e8b84c6f948692d41d5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:26.441335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:26.555711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a4626bcd5814e8b84c6f948692d41d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.575894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a4626bcd5814e8b84c6f948692d41d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.576670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a4626bcd5814e8b84c6f948692d41d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.601580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a4626bcd5814e8b84c6f948692d41d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.602994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a4626bcd5814e8b84c6f948692d41d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:26.663256 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2330/4670] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:26 2022] DELETE /identity/v3/users/5a4626bcd5814e8b84c6f948692d41d5 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:26.668437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:26.669465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:26.669978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:26.704790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:26.710571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:26.806228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2341/4671] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:26.827364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:26.829066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:26.943223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:26.986484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51642ee6-f342-4020-bc51-5427582da6da admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:26.999335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:27.000263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:27.000814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:27.001409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:27.046777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:27.049089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:27.063940 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2331/4672] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:30.018632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:30.022271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:30.067391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d24fc-5709-4bd2-b3b0-8875f42a8ce9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:30.095964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:30.103375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:30.103816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:30.104568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] PATH_INFO: `/v3/projects/77ee365fc4464b26aa3c79c302e02266` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:30.106384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ee365fc4464b26aa3c79c302e02266)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:30.109618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:30.249861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ee365fc4464b26aa3c79c302e02266', 'request_id': 'req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.258111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ee365fc4464b26aa3c79c302e02266', 'request_id': 'req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.272007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ee365fc4464b26aa3c79c302e02266', 'request_id': 'req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.301528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] The token cache is being invalidate because project 77ee365fc4464b26aa3c79c302e02266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:30.302004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.302813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.304787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2342/4673] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:30 2022] DELETE /identity/v3/projects/77ee365fc4464b26aa3c79c302e02266 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:30.309221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:30.319492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4cacc-7aaa-493f-88fc-061c6b68448e service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:30.450195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:30.450994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:30.451408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:30.451845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] PATH_INFO: `/v3/users/e78c1c000f51483486a94183100c5d3d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:30.453431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78c1c000f51483486a94183100c5d3d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:30.456061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:30.562743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78c1c000f51483486a94183100c5d3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.579619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78c1c000f51483486a94183100c5d3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.580552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78c1c000f51483486a94183100c5d3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.605171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78c1c000f51483486a94183100c5d3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.606404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f040d651-b1ef-41ad-bb2b-c25613c49466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78c1c000f51483486a94183100c5d3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:30.658438 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2332/4674] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:30 2022] DELETE /identity/v3/users/e78c1c000f51483486a94183100c5d3d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:33.748674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:33.750732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:33.790875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba60c0-0eb5-4167-a46d-b71d7a82a22b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:33.833587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:33.896121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:33.896971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:33.897319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:33.897668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] PATH_INFO: `/v3/projects/3d852fd39dac4e5aac8a467f0b911eb6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:33.901053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d852fd39dac4e5aac8a467f0b911eb6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:33.903171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:34.050292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d852fd39dac4e5aac8a467f0b911eb6', 'request_id': 'req-b0343d61-dcc2-4cba-8239-8d97a704d32d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:34.061455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d852fd39dac4e5aac8a467f0b911eb6', 'request_id': 'req-b0343d61-dcc2-4cba-8239-8d97a704d32d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:34.069243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d852fd39dac4e5aac8a467f0b911eb6', 'request_id': 'req-b0343d61-dcc2-4cba-8239-8d97a704d32d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:34.109250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] The token cache is being invalidate because project 3d852fd39dac4e5aac8a467f0b911eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:34.110327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:34.110327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0343d61-dcc2-4cba-8239-8d97a704d32d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:34.118794 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2343/4675] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:33 2022] DELETE /identity/v3/projects/3d852fd39dac4e5aac8a467f0b911eb6 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:34.119894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcac06-db9c-4271-af33-7414fe4501a4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:34.120352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcac06-db9c-4271-af33-7414fe4501a4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:34.120533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcac06-db9c-4271-af33-7414fe4501a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:34.160835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0ffcac06-db9c-4271-af33-7414fe4501a4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:34.162020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ffcac06-db9c-4271-af33-7414fe4501a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:34.295278 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2333/4676] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:34.305919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ac376-a883-4b0e-9b7b-f5e5dca28604 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:34.306309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ac376-a883-4b0e-9b7b-f5e5dca28604 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:34.306571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ac376-a883-4b0e-9b7b-f5e5dca28604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:34.380694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4e9ac376-a883-4b0e-9b7b-f5e5dca28604 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:34.382026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4e9ac376-a883-4b0e-9b7b-f5e5dca28604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:34.442012 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2344/4677] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:34.442686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcac06-db9c-4271-af33-7414fe4501a4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:34.448614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:34.456706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcac06-db9c-4271-af33-7414fe4501a4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:34.467321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:34.470038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:34.472817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:34.472817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:34.473338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:34.473536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:34.475645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:34.505971 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2334/4678] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:34.508973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ac376-a883-4b0e-9b7b-f5e5dca28604 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:34.521949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ac376-a883-4b0e-9b7b-f5e5dca28604 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:34.539507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:34.542651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:34.542945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:34.543226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:34.543879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:34.550109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:34.605774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2345/4679] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:34 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:34.609095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:34.623157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b88f16-63dc-4e93-a2d8-817769201123 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:34.878356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:34.892277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:34.892791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:34.893165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:34.895731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:34.899939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.022409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.025190 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2335/4680] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:34 2022] POST /identity/v3/users => generated 287 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:35.044512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab497a-85a1-4a0b-b4e1-ee30984bc08c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.051892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.052775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.053322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.054205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.056938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:35.061675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.079907 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2346/4681] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:35.084293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.098289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68432014-d094-4e70-ae0b-2ff30665f2ed admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.110794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89468948-b0d5-4081-9301-8e4da13ec56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.111243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89468948-b0d5-4081-9301-8e4da13ec56c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.111538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89468948-b0d5-4081-9301-8e4da13ec56c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.111931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89468948-b0d5-4081-9301-8e4da13ec56c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.115409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89468948-b0d5-4081-9301-8e4da13ec56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:35.121763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89468948-b0d5-4081-9301-8e4da13ec56c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.150062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.159688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2336/4682] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:35.161746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb46ad3-af17-4e50-a00d-95d931d55101 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.172152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.174384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.174672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.174957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] PATH_INFO: `/v3/projects/c1130c1b899f497a99b217e6fadfa4e9/users/de901ac6cd1a4a728cf9252ead28859e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.217044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1130c1b899f497a99b217e6fadfa4e9, user_id=de901ac6cd1a4a728cf9252ead28859e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:35.223139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.265068 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2347/4683] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:35 2022] PUT /identity/v3/projects/c1130c1b899f497a99b217e6fadfa4e9/users/de901ac6cd1a4a728cf9252ead28859e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:35.267082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.267632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.267734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.359562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] MFA Rules not processed for user `de901ac6cd1a4a728cf9252ead28859e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:35.363964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:35.523181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.532504 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2337/4684] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:35.609255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.611803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:35.628434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:35.638083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2cb3b5-e10d-48b2-89cc-ed3b752388ae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.668363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.669073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.669456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.669793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.670544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:35.673633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.734793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:35.736513 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2348/4685] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:35 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:35.741256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.746030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937aecd-de6b-4ff9-9882-81a6a7ff7cfe None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.853281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9815ac4-fdeb-44dd-b896-ce6636f02ee3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.858653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.859899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.860190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.860472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.861425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:35.864245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.864901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.865149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.865231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.865446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] PATH_INFO: `/v3/users/30a110f84c5346c6a5b6e3416bfb1f67` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.866659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a110f84c5346c6a5b6e3416bfb1f67)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:35.869526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.897570 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2349/4686] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:35 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:35.902477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.930890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417056a7-5734-411a-9379-77a3468cbfad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.940931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.942403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.943113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.944081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:35.945121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:35.947807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:35.965372 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2350/4687] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:35.969913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:35.987946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b2d34-b182-417f-af79-0470aa7539b3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:35.991362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a110f84c5346c6a5b6e3416bfb1f67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:35.995251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:35.996075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:35.996469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:35.996845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] PATH_INFO: `/v3/projects/545a3b69459f455ba54696c057815ae3/users/4283c4cfbe3844c2a5c3023bbd901049/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.017045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a110f84c5346c6a5b6e3416bfb1f67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:36.017760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=545a3b69459f455ba54696c057815ae3, user_id=4283c4cfbe3844c2a5c3023bbd901049, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:36.017909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a110f84c5346c6a5b6e3416bfb1f67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:36.021451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:36.034410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a110f84c5346c6a5b6e3416bfb1f67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:36.035143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d44e5389-3309-4374-a4ad-64cf5f1c15e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a110f84c5346c6a5b6e3416bfb1f67'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:36.057428 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2351/4688] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:35 2022] PUT /identity/v3/projects/545a3b69459f455ba54696c057815ae3/users/4283c4cfbe3844c2a5c3023bbd901049/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:36.073560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:36.074590 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2338/4689] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:35 2022] DELETE /identity/v3/users/30a110f84c5346c6a5b6e3416bfb1f67 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:36.075684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:36.076173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:36.076474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:36.077669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.119089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:36.120372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:36.224105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8cc7f-1029-43e6-85c4-dff50583ed8a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:36.260287 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2339/4690] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:36.282680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:36.284346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:36.299576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:36.332875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:36.333844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:36.334142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:36.334423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.335318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:36.337917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:36.341218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:36.357106 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2352/4691] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:36.361419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:36.376795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d13d4c-f26e-478e-98e6-0030e1843dc0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:36.382632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4813481-faa3-4514-ada6-3fee8555944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:36.383547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4813481-faa3-4514-ada6-3fee8555944b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:36.383932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4813481-faa3-4514-ada6-3fee8555944b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:36.384312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4813481-faa3-4514-ada6-3fee8555944b admin admin] PATH_INFO: `/v3/projects/545a3b69459f455ba54696c057815ae3/users/4283c4cfbe3844c2a5c3023bbd901049/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.386614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4813481-faa3-4514-ada6-3fee8555944b admin admin] RBAC: Authorizing `identity:create_grant(project_id=545a3b69459f455ba54696c057815ae3, user_id=4283c4cfbe3844c2a5c3023bbd901049, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:36.391869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4813481-faa3-4514-ada6-3fee8555944b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:36.393140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e1003-fc2d-49ed-8987-fc6f4590c24d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:36.400011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:36.400654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:36.401002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:36.401306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.408764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2353/4692] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:36 2022] PUT /identity/v3/projects/545a3b69459f455ba54696c057815ae3/users/4283c4cfbe3844c2a5c3023bbd901049/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:36.420442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32ab23-2369-462d-a701-4879ac12fa1b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:36.422340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32ab23-2369-462d-a701-4879ac12fa1b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:36.422749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32ab23-2369-462d-a701-4879ac12fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.425610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:36.428619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:36.451701 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2340/4693] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:05:36.467933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1c32ab23-2369-462d-a701-4879ac12fa1b None None] MFA Rules not processed for user `4283c4cfbe3844c2a5c3023bbd901049`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:36.470278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1c32ab23-2369-462d-a701-4879ac12fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:36.613187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:36.618757 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2354/4694] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:36.708987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5326240-04e3-41b5-94a5-f75843e9791a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:36.719749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:36.720588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:36.721142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:36.721666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.722346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:36.724528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:36.761617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2341/4695] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:05:36 2022] POST /identity/v3/domains => generated 277 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:36.767063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32ab23-2369-462d-a701-4879ac12fa1b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:36.779723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32ab23-2369-462d-a701-4879ac12fa1b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:36.889105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:36.889731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:36.890048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:36.890328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:36.890966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:36.893564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:36.943651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:36.945571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:36.948538 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2355/4696] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:36 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:36.957292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:36.995901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:37.052124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b86d-2788-4c3a-9ae4-83a1119c81e3 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.058786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.059493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.059696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.059975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.060646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.063228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.086539 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2356/4697] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:05:36 2022] POST /identity/v3/groups => generated 301 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:37.090191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56537b9c-8e30-48d3-a312-20c058b19bb8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.091320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.116008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54a86-d81f-49f6-adac-21f611f6b901 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.140157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.142073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.142384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.142693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.143359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.150060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.208760 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2357/4698] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:37 2022] POST /identity/v3/users => generated 445 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:37.213481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.215322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.216023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.216313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.216637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] PATH_INFO: `/v3/projects/9665812469c1426691889fca6f3c7be2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.219343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9665812469c1426691889fca6f3c7be2)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.225311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.236984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f45f7-3371-4288-9be7-3495642389b2 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.250663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.251489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.252202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.252841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.253663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.255790 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:37.255790 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:37.255790 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:37.259951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.272679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:37.298678 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2358/4699] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:05:37 2022] POST /identity/v3/roles => generated 240 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:37.302509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.324821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4bf69-c5ec-4d70-ad6f-aa50c1aaa27f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.341434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.342194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.342481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.342854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/c83274498be24dce92c19aa07468a158/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.356151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_grant(project_id=db5679bd32a44b7dbe518bde455feab6, group_id=3a0772bb9ef44cb9937f9cd026db864b, role_id=c83274498be24dce92c19aa07468a158)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.359147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.377696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9665812469c1426691889fca6f3c7be2', 'request_id': 'req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.386403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9665812469c1426691889fca6f3c7be2', 'request_id': 'req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.398251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9665812469c1426691889fca6f3c7be2', 'request_id': 'req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.402447 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2359/4700] 10.222.0.33 () {60 vars in 1585 bytes} [Sat Sep 10 21:05:37 2022] PUT /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/c83274498be24dce92c19aa07468a158/inherited_to_projects => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:37.404306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.419971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] The token cache is being invalidate because project 9665812469c1426691889fca6f3c7be2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:37.420270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.421528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6a8b0b1-b73c-49bc-bfb4-6f83580f85c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.429993 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2342/4701] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:36 2022] DELETE /identity/v3/projects/9665812469c1426691889fca6f3c7be2 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:37.434463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c845ec4-29c4-4694-96f6-59ca38f7ca3a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.434763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c845ec4-29c4-4694-96f6-59ca38f7ca3a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.435045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c845ec4-29c4-4694-96f6-59ca38f7ca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.463116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2c845ec4-29c4-4694-96f6-59ca38f7ca3a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:37.466020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c845ec4-29c4-4694-96f6-59ca38f7ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:37.512785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8164a0-cc98-4dfc-8b1f-1c4e61d6b147 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.518756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.520247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.520614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.521039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/c83274498be24dce92c19aa07468a158/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.525010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:check_grant(project_id=db5679bd32a44b7dbe518bde455feab6, group_id=3a0772bb9ef44cb9937f9cd026db864b, role_id=c83274498be24dce92c19aa07468a158)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.526321 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:37.526321 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:37.526321 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:37.528546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.538906 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2360/4702] 10.222.0.33 () {58 vars in 1567 bytes} [Sat Sep 10 21:05:37 2022] HEAD /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/c83274498be24dce92c19aa07468a158/inherited_to_projects => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:37.543006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.544733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:37.552050 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2343/4703] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:37.555461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524b5ce-02ad-4a56-9762-6085d59c8d50 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.557224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524b5ce-02ad-4a56-9762-6085d59c8d50 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.557716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524b5ce-02ad-4a56-9762-6085d59c8d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.567674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d50f6-6294-482d-b1e0-d99aa5a9a82a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.573479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.574342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.574629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.574910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/c83274498be24dce92c19aa07468a158/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.578632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=db5679bd32a44b7dbe518bde455feab6, group_id=3a0772bb9ef44cb9937f9cd026db864b, role_id=c83274498be24dce92c19aa07468a158)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.579932 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:37.579932 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:37.579932 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:37.582884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.583910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1524b5ce-02ad-4a56-9762-6085d59c8d50 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:37.585280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1524b5ce-02ad-4a56-9762-6085d59c8d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:37.594187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role c83274498be24dce92c19aa07468a158 was removed from group 3a0772bb9ef44cb9937f9cd026db864b on project db5679bd32a44b7dbe518bde455feab6. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:37.594802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.595801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.603411 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2344/4704] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:37.609108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524b5ce-02ad-4a56-9762-6085d59c8d50 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.625774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2361/4705] 10.222.0.33 () {58 vars in 1569 bytes} [Sat Sep 10 21:05:37 2022] DELETE /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/c83274498be24dce92c19aa07468a158/inherited_to_projects => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:37.626177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524b5ce-02ad-4a56-9762-6085d59c8d50 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.629637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.631296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:37.705386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:37.717319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.718076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.718364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.718642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.719303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:37.719611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.721298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.725974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c4007-cb0c-4ed9-bbb6-801c38c0f461 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.735712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.736871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.737254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.737770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles/c83274498be24dce92c19aa07468a158` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.738574 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2345/4706] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:37.743178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.743432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c83274498be24dce92c19aa07468a158)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.744717 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:37.744717 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:37.744717 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:37.745379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:37.746087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.764139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04164877-e864-476b-98e5-6e0644811c3e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:37.770124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:37.771112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:37.771582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:37.772039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:37.773047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:37.776226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:37.830772 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2346/4707] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:37 2022] POST /identity/v3/projects => generated 359 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:37.838489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role c83274498be24dce92c19aa07468a158 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:37.838797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.840127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:37.848586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.849919 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2362/4708] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:37 2022] DELETE /identity/v3/roles/c83274498be24dce92c19aa07468a158 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:37.851183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:37.855978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:37.857597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:37.862903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:37.994048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0a439-3d2c-4250-9775-c2953fa2a88c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.029664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563e1de-f10d-455e-bf47-3b25b39401ab tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.044448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.046463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.047255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.049369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.050170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.055386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.084291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.084909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.085290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.086068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.087861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.090346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.103345 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2363/4709] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:05:37 2022] POST /identity/v3/roles => generated 239 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:38.107661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.126864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc268c0f-5091-456f-b433-cf2ac53f6aba tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.134406 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2347/4710] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:37 2022] POST /identity/v3/users => generated 280 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:38.135940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.136085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.136735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.137077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.137420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/3483ab9787b9496d91615dbb6a7f35aa/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.151172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1416d-f522-468d-83a7-48c6cb7ce99f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.157754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.158244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_grant(project_id=db5679bd32a44b7dbe518bde455feab6, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=3483ab9787b9496d91615dbb6a7f35aa)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.159174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.160497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.160932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.161121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.162104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.164121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.201958 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2348/4711] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:38 2022] GET /identity/v3/roles => generated 1650 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:38.205499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.218749 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2364/4712] 10.222.0.33 () {60 vars in 1582 bytes} [Sat Sep 10 21:05:38 2022] PUT /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/3483ab9787b9496d91615dbb6a7f35aa/inherited_to_projects => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:38.223345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.236534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc3a83-42e7-422c-a2ed-245f4db8c3a3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.329340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.330094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.330381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.330766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.331592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.333417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.346618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bb3ff-b93a-431f-bce7-d49571362230 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.347451 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2349/4713] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:38 2022] GET /identity/v3/roles => generated 1650 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:38.351630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.351950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.352913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.353197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.353476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/3483ab9787b9496d91615dbb6a7f35aa/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.355771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:check_grant(project_id=db5679bd32a44b7dbe518bde455feab6, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=3483ab9787b9496d91615dbb6a7f35aa)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.358717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.366582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67643ae-c197-4727-ba8c-d8d5ce10f831 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.369911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2365/4714] 10.222.0.33 () {58 vars in 1564 bytes} [Sat Sep 10 21:05:38 2022] HEAD /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/3483ab9787b9496d91615dbb6a7f35aa/inherited_to_projects => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:38.372265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.376793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.377644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.378178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.378618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] PATH_INFO: `/v3/projects/0980532ffcf34fcba842779e24616001/users/c3a02193633a44dda1885d363fcd5361/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.386857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7df8e-0dac-4084-be0e-71b485543530 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.392409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.393362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.393627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.394034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/3483ab9787b9496d91615dbb6a7f35aa/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.396635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=db5679bd32a44b7dbe518bde455feab6, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=3483ab9787b9496d91615dbb6a7f35aa)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.400916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.401413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0980532ffcf34fcba842779e24616001, user_id=c3a02193633a44dda1885d363fcd5361, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.404894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.408142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 3483ab9787b9496d91615dbb6a7f35aa was removed from user 16a4ad29ef104c28b869bf4cb250b8d8 on project db5679bd32a44b7dbe518bde455feab6. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:38.408623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:38.409396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29134647-10ac-4694-a271-f6ebee4cc06d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:38.426137 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2350/4715] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:38 2022] PUT /identity/v3/projects/0980532ffcf34fcba842779e24616001/users/c3a02193633a44dda1885d363fcd5361/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:38.429062 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2366/4716] 10.222.0.33 () {58 vars in 1566 bytes} [Sat Sep 10 21:05:38 2022] DELETE /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/3483ab9787b9496d91615dbb6a7f35aa/inherited_to_projects => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:38.433355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.436104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc82d4-3e01-4288-955b-fb2776e83b7c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.436557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc82d4-3e01-4288-955b-fb2776e83b7c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.437038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc82d4-3e01-4288-955b-fb2776e83b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.437469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:38.444109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:38.469341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:38.483857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2cdc82d4-3e01-4288-955b-fb2776e83b7c None None] MFA Rules not processed for user `c3a02193633a44dda1885d363fcd5361`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:38.486985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2cdc82d4-3e01-4288-955b-fb2776e83b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:38.536256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4654b-dbe9-4e33-9dc9-558e68f70616 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.583751 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2367/4717] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:38.625957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.626914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.627344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.627943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles/3483ab9787b9496d91615dbb6a7f35aa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.629777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_role(role_id=3483ab9787b9496d91615dbb6a7f35aa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.631423 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:38.631423 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:38.631423 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:38.632672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.685477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 3483ab9787b9496d91615dbb6a7f35aa has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:38.686070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:38.687714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:38.690197 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2351/4718] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:38 2022] DELETE /identity/v3/roles/3483ab9787b9496d91615dbb6a7f35aa => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:38.696107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc82d4-3e01-4288-955b-fb2776e83b7c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.710788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc82d4-3e01-4288-955b-fb2776e83b7c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.787964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.788595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.788958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.789285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.790342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.792217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.830014 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2368/4719] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:05:38 2022] POST /identity/v3/roles => generated 239 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:38.832388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.834173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:38.850348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5bbfd-8c4b-40f6-a60b-44c675878d7d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.855825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.857154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.857444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.857721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/21e0e4bdba784385967a03c87ae2d0b5/inherited_to_projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.860049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, group_id=3a0772bb9ef44cb9937f9cd026db864b, role_id=21e0e4bdba784385967a03c87ae2d0b5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.862774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:38.880402 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2352/4720] 10.222.0.33 () {60 vars in 1582 bytes} [Sat Sep 10 21:05:38 2022] PUT /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/21e0e4bdba784385967a03c87ae2d0b5/inherited_to_projects => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:38.881665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:38.893995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9c5f-6c54-4341-80a1-43d054d4a50a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:38.970097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:38.971189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:38.971664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:38.972361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:38.976205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, group_id=3a0772bb9ef44cb9937f9cd026db864b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:38.978630 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:38.978630 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:38.978630 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:38.984225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.006422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2369/4721] 10.222.0.33 () {58 vars in 1464 bytes} [Sat Sep 10 21:05:38 2022] GET /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/inherited_to_projects => generated 447 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:05:39.010394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.012344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:39.018736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.053707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c7ad1-29b4-4f7f-b100-9aef66977dae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.061617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.062883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.063172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.063456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/21e0e4bdba784385967a03c87ae2d0b5/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.074777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, group_id=3a0772bb9ef44cb9937f9cd026db864b, role_id=21e0e4bdba784385967a03c87ae2d0b5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.078817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.089872 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2370/4722] 10.222.0.33 () {58 vars in 1564 bytes} [Sat Sep 10 21:05:39 2022] HEAD /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/21e0e4bdba784385967a03c87ae2d0b5/inherited_to_projects => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:39.097345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.117994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1332854-dd59-4178-8d6e-c4f7f19260fb tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.130349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.131087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.131501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0c358-8c5d-4a2e-b7f3-a3cd2ef5b4d1 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.132057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.132368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/21e0e4bdba784385967a03c87ae2d0b5/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.135373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, group_id=3a0772bb9ef44cb9937f9cd026db864b, role_id=21e0e4bdba784385967a03c87ae2d0b5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.138144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.138936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.139607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.139747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.140157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.142150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:39.145656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 21e0e4bdba784385967a03c87ae2d0b5 was removed from group 3a0772bb9ef44cb9937f9cd026db864b on domain ad51f82fb78449eb86a5c3c40b9fba41. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:39.146004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.146681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.161555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:39.164177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:39.171040 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2371/4723] 10.222.0.33 () {58 vars in 1566 bytes} [Sat Sep 10 21:05:39 2022] DELETE /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/groups/3a0772bb9ef44cb9937f9cd026db864b/roles/21e0e4bdba784385967a03c87ae2d0b5/inherited_to_projects => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:39.175468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.179266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:39.235076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.239310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.253802 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2353/4724] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:39.268609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11568ed-7e0a-488e-8b05-11c962878ad8 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.273333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.274401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.274863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.275139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles/21e0e4bdba784385967a03c87ae2d0b5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.277879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_role(role_id=21e0e4bdba784385967a03c87ae2d0b5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.278646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.326934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 21e0e4bdba784385967a03c87ae2d0b5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:39.327276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.327961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.329863 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2372/4725] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:39 2022] DELETE /identity/v3/roles/21e0e4bdba784385967a03c87ae2d0b5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:39.335044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.350363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae24de1-ba68-45be-b330-303547a0916f service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.432503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.433215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.433507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.433808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.434512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.435591 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:39.435591 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:39.435591 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:39.436495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.459134 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2354/4726] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:05:39 2022] POST /identity/v3/roles => generated 240 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:39.460330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.462168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:39.477791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714905e-b083-47be-9cfb-2b94c0cdd75f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.485736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.486601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.486953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.487390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/75b9c9f880ef4661a163e9bcf13ce7e8/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.490471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=75b9c9f880ef4661a163e9bcf13ce7e8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.494014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.512273 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2373/4727] 10.222.0.33 () {60 vars in 1579 bytes} [Sat Sep 10 21:05:39 2022] PUT /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/75b9c9f880ef4661a163e9bcf13ce7e8/inherited_to_projects => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:39.513517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.525182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992f89b-5a80-474d-a626-ae1a6a8f07e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.613782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.614644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.615128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.616188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/inherited_to_projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.619201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, user_id=16a4ad29ef104c28b869bf4cb250b8d8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.620346 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:39.620346 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:39.620346 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:39.622587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.641292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2355/4728] 10.222.0.33 () {58 vars in 1461 bytes} [Sat Sep 10 21:05:39 2022] GET /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/inherited_to_projects => generated 447 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:39.645562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.657550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed425b63-8d49-4cf2-9e82-346a178f451b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.662777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.663578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.663962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.665010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/75b9c9f880ef4661a163e9bcf13ce7e8/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.667428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=75b9c9f880ef4661a163e9bcf13ce7e8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.670330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.677233 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2374/4729] 10.222.0.33 () {58 vars in 1561 bytes} [Sat Sep 10 21:05:39 2022] HEAD /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/75b9c9f880ef4661a163e9bcf13ce7e8/inherited_to_projects => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:39.682231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.694181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c36833-5abc-4533-a691-476b75a97ec4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.702782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.704253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.704676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.705865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/75b9c9f880ef4661a163e9bcf13ce7e8/inherited_to_projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.708406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=75b9c9f880ef4661a163e9bcf13ce7e8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.709592 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:39.709592 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:39.709592 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:39.712484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.725748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 75b9c9f880ef4661a163e9bcf13ce7e8 was removed from user 16a4ad29ef104c28b869bf4cb250b8d8 on domain ad51f82fb78449eb86a5c3c40b9fba41. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:39.726229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.728365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.754571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.754906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2356/4730] 10.222.0.33 () {58 vars in 1563 bytes} [Sat Sep 10 21:05:39 2022] DELETE /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/75b9c9f880ef4661a163e9bcf13ce7e8/inherited_to_projects => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:39.756383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:39.771183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8dcf8-5257-4c51-b03f-7ac45d001def tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.848476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:39.848476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:39.848836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:39.849201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles/75b9c9f880ef4661a163e9bcf13ce7e8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:39.851084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_role(role_id=75b9c9f880ef4661a163e9bcf13ce7e8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:39.860108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:39.917261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 75b9c9f880ef4661a163e9bcf13ce7e8 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:39.918183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.919090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:39.928184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:39.934560 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2375/4731] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:39 2022] DELETE /identity/v3/roles/75b9c9f880ef4661a163e9bcf13ce7e8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:39.940340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cce91-dde1-4f6e-8eb4-773f4ed0a24d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:39.969910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:40.027708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.029054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.029170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.029727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.030780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.032875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:40.070072 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2357/4732] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:05:39 2022] POST /identity/v3/roles => generated 240 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:40.073057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:40.074934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:40.102752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b6b28-8d04-4a97-a1a7-9275e93633b5 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:40.110846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.111471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.111755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.112028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.112886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.115008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:40.125669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:40.165199 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2376/4733] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:40 2022] POST /identity/v3/projects => generated 409 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:40.169165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:40.184550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a58d6-d943-4865-b3aa-70dd56e8de0e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:40.300946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.301753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.302249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.302636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/40abd2a351a14ddeb3489debcbf57ef5/inherited_to_projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.307982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=40abd2a351a14ddeb3489debcbf57ef5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.310896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:40.331616 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2358/4734] 10.222.0.33 () {60 vars in 1579 bytes} [Sat Sep 10 21:05:40 2022] PUT /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/40abd2a351a14ddeb3489debcbf57ef5/inherited_to_projects => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:40.337755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:40.354534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e4f10-2979-4f64-ae4d-464592ae4f7f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:40.474747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.476370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.476950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.477814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.479689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Adding query filter params (scope.project.id=db5679bd32a44b7dbe518bde455feab6, user.id=16a4ad29ef104c28b869bf4cb250b8d8) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:40.480208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.481488 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:40.481488 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:05:40.481488 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:05:40.486804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:40.591730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:40.592732 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2377/4735] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:05:40 2022] GET /identity/v3/role_assignments?scope.project.id=db5679bd32a44b7dbe518bde455feab6&user.id=16a4ad29ef104c28b869bf4cb250b8d8&effective => generated 648 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:40.608253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ac73e-1784-4ffc-a658-12cb033e5308 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:40.626372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.627109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.627487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.627861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.628707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Adding query filter params (scope.project.id=006fae96af0247219be726b4d42a85b9, user.id=16a4ad29ef104c28b869bf4cb250b8d8) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:40.629436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.630506 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:05:40.630506 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:05:40.630506 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:05:40.631951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:40.730966 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2359/4736] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:05:40 2022] GET /identity/v3/role_assignments?scope.project.id=006fae96af0247219be726b4d42a85b9&user.id=16a4ad29ef104c28b869bf4cb250b8d8&effective => generated 648 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:05:40.735267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:40.753271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211305a-e3e8-4fea-9a8a-79457331ea4e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:40.769767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.770897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.771183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.771461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/40abd2a351a14ddeb3489debcbf57ef5/inherited_to_projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.794641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=ad51f82fb78449eb86a5c3c40b9fba41, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=40abd2a351a14ddeb3489debcbf57ef5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.809333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:40.823894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 40abd2a351a14ddeb3489debcbf57ef5 was removed from user 16a4ad29ef104c28b869bf4cb250b8d8 on domain ad51f82fb78449eb86a5c3c40b9fba41. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:40.823894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:40.824271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:40.852352 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2378/4737] 10.222.0.33 () {58 vars in 1563 bytes} [Sat Sep 10 21:05:40 2022] DELETE /identity/v3/OS-INHERIT/domains/ad51f82fb78449eb86a5c3c40b9fba41/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/40abd2a351a14ddeb3489debcbf57ef5/inherited_to_projects => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:40.856965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:40.858852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:40.879058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aef3c6-02e6-4cad-8ac2-70f5eb51a128 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:40.942198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:40.951530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:40.952668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:40.953674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:40.953674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:40.954507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Adding query filter params (scope.project.id=db5679bd32a44b7dbe518bde455feab6, user.id=16a4ad29ef104c28b869bf4cb250b8d8) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:40.954883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:40.956892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:41.021804 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2360/4738] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:05:40 2022] GET /identity/v3/role_assignments?scope.project.id=db5679bd32a44b7dbe518bde455feab6&user.id=16a4ad29ef104c28b869bf4cb250b8d8&effective => generated 228 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:41.027142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:41.038159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716263fb-8313-4d3e-ab56-cd9d7bafbb5b tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:41.044334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:41.044488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:41.044710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:41.044919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:41.045694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Adding query filter params (scope.project.id=006fae96af0247219be726b4d42a85b9, user.id=16a4ad29ef104c28b869bf4cb250b8d8) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:41.046666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:41.047885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:41.114669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2379/4739] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:05:41 2022] GET /identity/v3/role_assignments?scope.project.id=006fae96af0247219be726b4d42a85b9&user.id=16a4ad29ef104c28b869bf4cb250b8d8&effective => generated 228 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:41.115787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:41.128370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8974299-3067-45d2-87e7-8aabc375275d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:41.133257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:41.134905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:41.134905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:41.134905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/projects/006fae96af0247219be726b4d42a85b9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:41.135973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_project(project_id=006fae96af0247219be726b4d42a85b9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:41.138015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:41.192597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006fae96af0247219be726b4d42a85b9', 'request_id': 'req-68147071-b8bd-4d6b-aad3-436c355dcd96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:41.199960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006fae96af0247219be726b4d42a85b9', 'request_id': 'req-68147071-b8bd-4d6b-aad3-436c355dcd96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:41.214922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006fae96af0247219be726b4d42a85b9', 'request_id': 'req-68147071-b8bd-4d6b-aad3-436c355dcd96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:41.245798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] The token cache is being invalidate because project 006fae96af0247219be726b4d42a85b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:41.246205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:41.246833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:41.251720 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2361/4740] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:41 2022] DELETE /identity/v3/projects/006fae96af0247219be726b4d42a85b9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:41.253417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:41.259020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:41.275870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5946b-2a3a-463d-8dcb-fd679e6de337 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:41.311007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:41.420410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:41.423952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:41.424472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:41.424759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles/40abd2a351a14ddeb3489debcbf57ef5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:41.429536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_role(role_id=40abd2a351a14ddeb3489debcbf57ef5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:41.433908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:41.543300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role 40abd2a351a14ddeb3489debcbf57ef5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:41.543656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:41.544639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:41.546856 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2380/4741] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:41 2022] DELETE /identity/v3/roles/40abd2a351a14ddeb3489debcbf57ef5 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:41.552293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:41.566140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147071-b8bd-4d6b-aad3-436c355dcd96 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:41.700246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:41.700948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:41.701235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:41.701524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:41.702826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:41.706764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:41.738205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2362/4742] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:05:41 2022] POST /identity/v3/roles => generated 239 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:41.742002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:41.743727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:41.757200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861ba61-b10a-4539-82a4-243b0301f29d tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:41.766524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:41.767342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:41.767953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:41.768387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:41.769142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:41.772003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:41.844833 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2381/4743] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:41 2022] POST /identity/v3/projects => generated 408 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:41.848961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:41.869582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b636ee-b2e3-4a9c-a743-762a146499c4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:41.916271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:41.993363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:41.994381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:41.994705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:41.995219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/bde17775393345dcbc00644a508eab93/inherited_to_projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:42.008218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:create_grant(project_id=db5679bd32a44b7dbe518bde455feab6, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=bde17775393345dcbc00644a508eab93)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:42.011032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:42.034083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2363/4744] 10.222.0.33 () {60 vars in 1582 bytes} [Sat Sep 10 21:05:41 2022] PUT /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/bde17775393345dcbc00644a508eab93/inherited_to_projects => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:42.035704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:42.051555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fa50b-a025-467f-83cb-ceebac8bbe87 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:42.162522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:42.163685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:42.164241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:42.165210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:42.167840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Adding query filter params (scope.project.id=98ea9e1ef2894618bc74c24620371c51, user.id=16a4ad29ef104c28b869bf4cb250b8d8) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:42.167840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:42.169929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:42.274129 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2382/4745] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:05:42 2022] GET /identity/v3/role_assignments?scope.project.id=98ea9e1ef2894618bc74c24620371c51&user.id=16a4ad29ef104c28b869bf4cb250b8d8&effective => generated 649 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:42.276820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:42.288050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2186b76-b594-412b-97db-b82a775b58cd tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:42.297040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:42.298012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:42.298463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:42.298896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/bde17775393345dcbc00644a508eab93/inherited_to_projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:42.301583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=db5679bd32a44b7dbe518bde455feab6, user_id=16a4ad29ef104c28b869bf4cb250b8d8, role_id=bde17775393345dcbc00644a508eab93)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:42.304625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:42.311100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role bde17775393345dcbc00644a508eab93 was removed from user 16a4ad29ef104c28b869bf4cb250b8d8 on project db5679bd32a44b7dbe518bde455feab6. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:42.311715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:42.312586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:42.333184 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2364/4746] 10.222.0.33 () {58 vars in 1566 bytes} [Sat Sep 10 21:05:42 2022] DELETE /identity/v3/OS-INHERIT/projects/db5679bd32a44b7dbe518bde455feab6/users/16a4ad29ef104c28b869bf4cb250b8d8/roles/bde17775393345dcbc00644a508eab93/inherited_to_projects => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:42.337908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:42.340517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:42.361006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b725f-96a1-4e55-95ba-ca93907e4c5a tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:42.384778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:42.454935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de338194-c239-46a6-ab1d-986bdd3def09 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:42.455755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de338194-c239-46a6-ab1d-986bdd3def09 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:42.456270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de338194-c239-46a6-ab1d-986bdd3def09 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:42.456706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de338194-c239-46a6-ab1d-986bdd3def09 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:42.457444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de338194-c239-46a6-ab1d-986bdd3def09 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Adding query filter params (scope.project.id=98ea9e1ef2894618bc74c24620371c51, user.id=16a4ad29ef104c28b869bf4cb250b8d8) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:42.457752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de338194-c239-46a6-ab1d-986bdd3def09 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:42.459774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de338194-c239-46a6-ab1d-986bdd3def09 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:42.577787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2383/4747] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:05:42 2022] GET /identity/v3/role_assignments?scope.project.id=98ea9e1ef2894618bc74c24620371c51&user.id=16a4ad29ef104c28b869bf4cb250b8d8&effective => generated 228 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:42.578997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:42.592939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd30ce-6d7e-4bb8-acaf-91431fc790e4 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:42.606394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:42.606927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:42.607399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:42.607789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/projects/98ea9e1ef2894618bc74c24620371c51` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:42.609690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_project(project_id=98ea9e1ef2894618bc74c24620371c51)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:42.611728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:42.669022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681e099-e994-462f-8022-8f6858451a0d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:42.669022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681e099-e994-462f-8022-8f6858451a0d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:42.669022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681e099-e994-462f-8022-8f6858451a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:42.703001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f681e099-e994-462f-8022-8f6858451a0d None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:42.709337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f681e099-e994-462f-8022-8f6858451a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:42.811938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98ea9e1ef2894618bc74c24620371c51', 'request_id': 'req-9dc41e5d-0110-4f5a-a7ef-8883e0986066'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:42.825322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98ea9e1ef2894618bc74c24620371c51', 'request_id': 'req-9dc41e5d-0110-4f5a-a7ef-8883e0986066'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:42.862962 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2384/4748] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:05:42.863611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98ea9e1ef2894618bc74c24620371c51', 'request_id': 'req-9dc41e5d-0110-4f5a-a7ef-8883e0986066'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:42.866369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681e099-e994-462f-8022-8f6858451a0d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:42.870538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f681e099-e994-462f-8022-8f6858451a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:42.895238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] The token cache is being invalidate because project 98ea9e1ef2894618bc74c24620371c51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:42.895769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:42.897660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:42.905158 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2365/4749] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:42 2022] DELETE /identity/v3/projects/98ea9e1ef2894618bc74c24620371c51 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:42.909744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:42.912729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:43.041196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681e099-e994-462f-8022-8f6858451a0d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:43.071702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:43.108607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41e5d-0110-4f5a-a7ef-8883e0986066 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:43.114161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:43.114961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:43.115351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:43.115750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/roles/bde17775393345dcbc00644a508eab93` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:43.118253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bde17775393345dcbc00644a508eab93)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:43.120002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:43.148077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:43.150416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:43.151994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:43.152749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:43.159847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:43.211969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invalidating the token cache because role bde17775393345dcbc00644a508eab93 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:43.212442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:43.214978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:43.218276 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2366/4750] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:42 2022] DELETE /identity/v3/roles/bde17775393345dcbc00644a508eab93 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:43.223360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:43.227461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:43.251753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:43.364166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:43.367866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:43.377842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191864-cd63-446e-a018-35c5f5c1e35c tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:43.383280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:43.384395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:43.384746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:43.386379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/users/16a4ad29ef104c28b869bf4cb250b8d8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:43.388072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_user(user_id=16a4ad29ef104c28b869bf4cb250b8d8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:43.395695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:43.408275 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2385/4751] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:05:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:05:43.506221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a4ad29ef104c28b869bf4cb250b8d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:43.526446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a4ad29ef104c28b869bf4cb250b8d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:43.527499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a4ad29ef104c28b869bf4cb250b8d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:43.543839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a4ad29ef104c28b869bf4cb250b8d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:43.545208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a4ad29ef104c28b869bf4cb250b8d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:43.611154 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2367/4752] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:43 2022] DELETE /identity/v3/users/16a4ad29ef104c28b869bf4cb250b8d8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:43.612381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:43.639173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17663a-dd92-4d32-9db6-ba1f177fdc69 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:43.793891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:43.799070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:43.799363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:43.799654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/groups/3a0772bb9ef44cb9937f9cd026db864b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:43.802904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_group(group_id=3a0772bb9ef44cb9937f9cd026db864b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:43.811288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:43.909449 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2386/4753] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:05:43 2022] DELETE /identity/v3/groups/3a0772bb9ef44cb9937f9cd026db864b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:43.913113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:43.914990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:43.952263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beb17e-a051-40fb-996e-f1c85ed8b3ae tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:44.122185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:44.122850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:44.123168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:44.123446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/projects/db5679bd32a44b7dbe518bde455feab6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:44.129017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_project(project_id=db5679bd32a44b7dbe518bde455feab6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:44.131027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:44.192872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:44.233991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5679bd32a44b7dbe518bde455feab6', 'request_id': 'req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.243057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5679bd32a44b7dbe518bde455feab6', 'request_id': 'req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.253414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5679bd32a44b7dbe518bde455feab6', 'request_id': 'req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.282477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] The token cache is being invalidate because project db5679bd32a44b7dbe518bde455feab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:44.282774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.283458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.287965 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2368/4754] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:43 2022] DELETE /identity/v3/projects/db5679bd32a44b7dbe518bde455feab6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:44.290872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:44.294251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:44.333922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd106d5-75bc-4edb-88b1-b72bab4ed79f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:44.372024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:44.475382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:44.478559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:44.478977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:44.480374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/domains/ad51f82fb78449eb86a5c3c40b9fba41` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:44.484645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=ad51f82fb78449eb86a5c3c40b9fba41)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:44.489670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:44.571285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.573894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.656467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.657436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.663753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2387/4755] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:05:44 2022] PATCH /identity/v3/domains/ad51f82fb78449eb86a5c3c40b9fba41 => generated 278 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:44.668371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:44.683577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e15d15-12e7-4e2a-b75f-ae45d9ce8a85 tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:44.857264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4283c4cfbe3844c2a5c3023bbd901049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '545a3b69459f455ba54696c057815ae3', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:44.858174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:44.858463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:44.858741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] PATH_INFO: `/v3/domains/ad51f82fb78449eb86a5c3c40b9fba41` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:44.877526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=ad51f82fb78449eb86a5c3c40b9fba41)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:44.879668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:44.915096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:44.920389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.009667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41', 'request_id': 'req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.022471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41', 'request_id': 'req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.036389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41', 'request_id': 'req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.049277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] The token cache is being invalidate because project ad51f82fb78449eb86a5c3c40b9fba41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:45.049925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.050884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.059535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:45.063106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41', 'request_id': 'req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.065040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41', 'request_id': 'req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.066087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'ad51f82fb78449eb86a5c3c40b9fba41', 'request_id': 'req-857a5944-02d8-4bd4-99fe-ebfcb4686a9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.083511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:45.085778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:45.088389 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2369/4756] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:05:44 2022] DELETE /identity/v3/domains/ad51f82fb78449eb86a5c3c40b9fba41 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:45.211623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162212c-ab18-496d-9c9d-45b25e20c80e tempest-InheritsV3TestJSON-1759576930 tempest-InheritsV3TestJSON-1759576930-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:45.309693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:45.310868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:45.311368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:45.311795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] PATH_INFO: `/v3/users/de901ac6cd1a4a728cf9252ead28859e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:45.313635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=de901ac6cd1a4a728cf9252ead28859e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:45.316689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:45.415613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:45.445056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de901ac6cd1a4a728cf9252ead28859e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.467413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de901ac6cd1a4a728cf9252ead28859e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.475870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de901ac6cd1a4a728cf9252ead28859e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.488263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de901ac6cd1a4a728cf9252ead28859e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.489234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de901ac6cd1a4a728cf9252ead28859e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:45.501497 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2388/4757] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:45 2022] DELETE /identity/v3/users/de901ac6cd1a4a728cf9252ead28859e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:45.506142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:45.506338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:45.506476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:45.536254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:45.537673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:45.660061 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2370/4758] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:45.687448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:45.688978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:45.780062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab12549-c7d2-4e63-ba96-885c523c05ce admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:45.788626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:45.789520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:45.789798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:45.790684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:45.817894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:45.821882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:45.852428 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2389/4759] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:46.402320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:46.405761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:46.643841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238775ed-dcba-4682-aa90-f9b4dc0f9c3a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:46.652927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:46.654011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:46.654568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:46.655045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:46.661079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:46.688652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:46.691952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:46.765512 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2371/4760] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:46.892220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:46.894549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:46.919708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ccd02-0d11-4f94-bfb2-0dc29ecf6764 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:46.925196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:46.926086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:46.926379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:46.929533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] PATH_INFO: `/v3/users/b766ab7d168745a4bb8b07fb8e5df3ed` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:46.929533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b766ab7d168745a4bb8b07fb8e5df3ed)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:46.930905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:47.038028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b766ab7d168745a4bb8b07fb8e5df3ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.073245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b766ab7d168745a4bb8b07fb8e5df3ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.081786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b766ab7d168745a4bb8b07fb8e5df3ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.092693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:47.094654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:47.106960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b766ab7d168745a4bb8b07fb8e5df3ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.107769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5a2a53db-0d85-4eea-88c3-25384aa3e46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b766ab7d168745a4bb8b07fb8e5df3ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.125633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c31309-b7e2-448f-abb5-2e0934862a7f service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:47.129182 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2390/4761] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:46 2022] DELETE /identity/v3/users/b766ab7d168745a4bb8b07fb8e5df3ed => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:47.192191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:47.192681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:47.193393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:47.239411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:47.243540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:47.293632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:47.294947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:47.294947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:47.294947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] PATH_INFO: `/v3/projects/c1130c1b899f497a99b217e6fadfa4e9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:47.296162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1130c1b899f497a99b217e6fadfa4e9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:47.299719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:47.321348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:47.355919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:47.392875 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2391/4762] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:47.418752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1130c1b899f497a99b217e6fadfa4e9', 'request_id': 'req-319b46e4-9ef7-4357-acb2-35f5ca77b894'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.433509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1130c1b899f497a99b217e6fadfa4e9', 'request_id': 'req-319b46e4-9ef7-4357-acb2-35f5ca77b894'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.452716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1130c1b899f497a99b217e6fadfa4e9', 'request_id': 'req-319b46e4-9ef7-4357-acb2-35f5ca77b894'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.492879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] The token cache is being invalidate because project c1130c1b899f497a99b217e6fadfa4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:47.493389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.494316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.497211 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2372/4763] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:47 2022] DELETE /identity/v3/projects/c1130c1b899f497a99b217e6fadfa4e9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:47.501699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:47.505445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:47.553342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cfb30-f672-44ea-8beb-d77e5965cbd3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:47.661283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:47.662121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:47.662277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:47.663411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] PATH_INFO: `/v3/users/4283c4cfbe3844c2a5c3023bbd901049` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:47.663828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4283c4cfbe3844c2a5c3023bbd901049)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:47.666964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:47.838757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4283c4cfbe3844c2a5c3023bbd901049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.876003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4283c4cfbe3844c2a5c3023bbd901049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.892042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4283c4cfbe3844c2a5c3023bbd901049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.906279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4283c4cfbe3844c2a5c3023bbd901049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.906928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4283c4cfbe3844c2a5c3023bbd901049'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:47.918639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2392/4764] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:47 2022] DELETE /identity/v3/users/4283c4cfbe3844c2a5c3023bbd901049 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:48.121162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:48.127496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:48.149449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b46e4-9ef7-4357-acb2-35f5ca77b894 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:48.241462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:48.242847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:48.243494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:48.243919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] PATH_INFO: `/v3/projects/a31cdf32235d44b1bbd691ebea38663b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:48.246108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31cdf32235d44b1bbd691ebea38663b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:48.249804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:48.392254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31cdf32235d44b1bbd691ebea38663b', 'request_id': 'req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.400906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31cdf32235d44b1bbd691ebea38663b', 'request_id': 'req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.423468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31cdf32235d44b1bbd691ebea38663b', 'request_id': 'req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.458629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] The token cache is being invalidate because project a31cdf32235d44b1bbd691ebea38663b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:48.460436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.461675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.466432 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2373/4765] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:48 2022] DELETE /identity/v3/projects/a31cdf32235d44b1bbd691ebea38663b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:48.468237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:48.470682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:48.488860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2d37a-cd96-4966-a05e-08d1860593c3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:48.592116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:48.625980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:48.627108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:48.627543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:48.628202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] PATH_INFO: `/v3/users/173f60c866c8453a97d2701fee66bfb5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:48.629921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=173f60c866c8453a97d2701fee66bfb5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:48.632424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:48.757368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173f60c866c8453a97d2701fee66bfb5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.786837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173f60c866c8453a97d2701fee66bfb5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.829484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173f60c866c8453a97d2701fee66bfb5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.842898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173f60c866c8453a97d2701fee66bfb5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.843638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19c641a7-16b7-474b-85b7-4a9c57d5c0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173f60c866c8453a97d2701fee66bfb5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:48.855178 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2393/4766] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:48 2022] DELETE /identity/v3/users/173f60c866c8453a97d2701fee66bfb5 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:48.921559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:48.923147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:48.940991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb829f1b-f712-4ea4-8cf1-ab4e93070be6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:49.068686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:49.094949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:49.096268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.097152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.097439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] PATH_INFO: `/v3/projects/545a3b69459f455ba54696c057815ae3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.098653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=545a3b69459f455ba54696c057815ae3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:49.108104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:49.228617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545a3b69459f455ba54696c057815ae3', 'request_id': 'req-15915625-4022-4bb8-a9f5-f5402afea7cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.256233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545a3b69459f455ba54696c057815ae3', 'request_id': 'req-15915625-4022-4bb8-a9f5-f5402afea7cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.275304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545a3b69459f455ba54696c057815ae3', 'request_id': 'req-15915625-4022-4bb8-a9f5-f5402afea7cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.291048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] The token cache is being invalidate because project 545a3b69459f455ba54696c057815ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:49.291767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.292559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15915625-4022-4bb8-a9f5-f5402afea7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:49.294614 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2374/4767] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:48 2022] DELETE /identity/v3/projects/545a3b69459f455ba54696c057815ae3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:49.299762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d8d6-5517-408f-80f4-d890f7238d26 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.300209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d8d6-5517-408f-80f4-d890f7238d26 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.300427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d8d6-5517-408f-80f4-d890f7238d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.347417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2c00d8d6-5517-408f-80f4-d890f7238d26 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:49.349226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c00d8d6-5517-408f-80f4-d890f7238d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:49.519700 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2394/4768] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:49.521371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96d7d4-4fb0-43c6-a9a9-130678f461d2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.521819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96d7d4-4fb0-43c6-a9a9-130678f461d2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.522046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96d7d4-4fb0-43c6-a9a9-130678f461d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.571824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cd96d7d4-4fb0-43c6-a9a9-130678f461d2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:49.573087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cd96d7d4-4fb0-43c6-a9a9-130678f461d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:49.599760 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2375/4769] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:49.602105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d8d6-5517-408f-80f4-d890f7238d26 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.629877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d8d6-5517-408f-80f4-d890f7238d26 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:49.645123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:49.645754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.651345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.652772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.653920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:49.654253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:49.658531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:49.691734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96d7d4-4fb0-43c6-a9a9-130678f461d2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.694672 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2395/4770] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:49.713328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96d7d4-4fb0-43c6-a9a9-130678f461d2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:49.746257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:49.747017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:49.747376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:49.747693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:49.748326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:49.760918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:49.836596 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2376/4771] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:49.841343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.849047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:49.858875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:49.873658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa436-31e8-4d5c-9f54-044c9d0318b6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.040322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.041305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.041728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.042216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.043456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.052565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.057880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71763-7efb-438e-9f63-d172b631b1c0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.070930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.073194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.074145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.074736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] PATH_INFO: `/v3/projects/385194bd6b1143edb0d72548ec736c88` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.077246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] RBAC: Authorizing `identity:delete_project(project_id=385194bd6b1143edb0d72548ec736c88)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.081857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.089564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:50.124813 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2396/4772] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:49 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:50.130666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.151195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed0b3-b6a9-445c-b7ef-5d57b1c28c47 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.165949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.166782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.167081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.167356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.168598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.170441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.197002 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2397/4773] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:50.198243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.214594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:50.216816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6233c4-9863-49a5-9e7b-bbad8cba524e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.228891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385194bd6b1143edb0d72548ec736c88', 'request_id': 'req-7a9e3d57-0613-470b-8414-13a2aae5654f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:50.249460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385194bd6b1143edb0d72548ec736c88', 'request_id': 'req-7a9e3d57-0613-470b-8414-13a2aae5654f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:50.267792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385194bd6b1143edb0d72548ec736c88', 'request_id': 'req-7a9e3d57-0613-470b-8414-13a2aae5654f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:50.315102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] The token cache is being invalidate because project 385194bd6b1143edb0d72548ec736c88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:50.316749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:50.318538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7a9e3d57-0613-470b-8414-13a2aae5654f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:50.321982 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2377/4774] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:49 2022] DELETE /identity/v3/projects/385194bd6b1143edb0d72548ec736c88 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:50.326930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c4ce-cb1e-4072-9f9b-0199d122b90f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.327548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c4ce-cb1e-4072-9f9b-0199d122b90f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.328342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c4ce-cb1e-4072-9f9b-0199d122b90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.362500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.363121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.363579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.364861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.366189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.379640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.380699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a859c4ce-cb1e-4072-9f9b-0199d122b90f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:50.382846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a859c4ce-cb1e-4072-9f9b-0199d122b90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:50.393416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2378/4775] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:05:50.403551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e7137-1f2e-4dbc-98ae-6f79cb6372b0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.404633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e7137-1f2e-4dbc-98ae-6f79cb6372b0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.405170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e7137-1f2e-4dbc-98ae-6f79cb6372b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.411361 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2398/4776] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:50.413857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.419613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:50.464799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade33295-dc9f-4950-a4f8-52b49d7d6ad2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.480670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b59076-9b93-4931-ab5b-5c13a5e9b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.482757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b59076-9b93-4931-ab5b-5c13a5e9b9e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.483197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b59076-9b93-4931-ab5b-5c13a5e9b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.484934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-415e7137-1f2e-4dbc-98ae-6f79cb6372b0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:50.490267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-415e7137-1f2e-4dbc-98ae-6f79cb6372b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:50.492893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b59076-9b93-4931-ab5b-5c13a5e9b9e2 admin admin] PATH_INFO: `/v3/projects/1f0b60aeb69544368b486012b41c7ab5/users/5cc5fe28677445ad8e2bb6b74a458738/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.521902 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2379/4777] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:50.524086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b59076-9b93-4931-ab5b-5c13a5e9b9e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0b60aeb69544368b486012b41c7ab5, user_id=5cc5fe28677445ad8e2bb6b74a458738, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.527426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e7137-1f2e-4dbc-98ae-6f79cb6372b0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.528606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b59076-9b93-4931-ab5b-5c13a5e9b9e2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.546096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e7137-1f2e-4dbc-98ae-6f79cb6372b0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.565268 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2399/4778] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:50 2022] PUT /identity/v3/projects/1f0b60aeb69544368b486012b41c7ab5/users/5cc5fe28677445ad8e2bb6b74a458738/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:50.579601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614778f-38bb-455d-a752-96c23ea70c26 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.579902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614778f-38bb-455d-a752-96c23ea70c26 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.580317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614778f-38bb-455d-a752-96c23ea70c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.628387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d614778f-38bb-455d-a752-96c23ea70c26 None None] MFA Rules not processed for user `5cc5fe28677445ad8e2bb6b74a458738`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:50.630519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d614778f-38bb-455d-a752-96c23ea70c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:50.714001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.714644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.714932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.715212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.715875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:50.716185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.717817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.752743 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2380/4779] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:50.753879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.766561 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2400/4780] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:05:50.772996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bc71a-6208-4b05-b431-aed5a839415e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.774494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614778f-38bb-455d-a752-96c23ea70c26 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.778863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.779617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.779738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.780023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.780722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.783061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.821675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614778f-38bb-455d-a752-96c23ea70c26 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.828724 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2381/4781] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:50.830410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:50.913019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:50.957661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.958615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.960017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.960456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.962945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:50.965950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:50.988662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d48b-b995-4c92-a097-634bcb6b6126 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:50.995963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:50.996729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:50.997076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:50.997541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:50.998314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.000368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.006613 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2401/4782] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:51.010520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.030971 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2382/4783] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:50 2022] POST /identity/v3/users => generated 282 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:51.034983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.048504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0add8-5ba1-4df0-88d2-a9ded45e3acf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.071712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:51.100921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b4961-5012-4b9b-826e-2572a12dc61a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.108287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.108833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.109119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.109397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.110951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.112908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.147633 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2402/4784] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:51 2022] POST /identity/v3/users => generated 295 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:51.152150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.160456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.162577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.162577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.162779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.163605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.166675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.172497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffead2-fc94-4edb-82bc-c8335686386d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.184326 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2383/4785] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:51.189369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.193631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.194355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.194643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.196463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.197263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.199066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.218536 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2403/4786] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:51.222911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.227666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455be3ca-6b2b-4721-a2e6-687e28e2902d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.235235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88372c84-6b1f-4091-b428-e9baccf2f800 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.242978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.243998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.245120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.245878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.246247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.246610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.246901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.247234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] PATH_INFO: `/v3/projects/19ca75db834243588309524674d9a3b8/users/c119aa18f8f249a59fc2fd28754133b7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.248142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.252439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.266584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ca75db834243588309524674d9a3b8, user_id=c119aa18f8f249a59fc2fd28754133b7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.269486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.281798 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2384/4787] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:51.286680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.287421 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2404/4788] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:51 2022] PUT /identity/v3/projects/19ca75db834243588309524674d9a3b8/users/c119aa18f8f249a59fc2fd28754133b7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:51.292522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.306456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c53af0-e566-4a98-b07f-21cc87a8611f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.389167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7fb54-f2f5-4e56-b9bb-fbdf3ba7f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.390376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7fb54-f2f5-4e56-b9bb-fbdf3ba7f5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.390803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7fb54-f2f5-4e56-b9bb-fbdf3ba7f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.391223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7fb54-f2f5-4e56-b9bb-fbdf3ba7f5ef admin admin] PATH_INFO: `/v3/projects/910afa67dcbc4f46a9403c5c9dd3a15e/users/97f9ec826b694534aea8259cd817ac10/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.410139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7fb54-f2f5-4e56-b9bb-fbdf3ba7f5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=910afa67dcbc4f46a9403c5c9dd3a15e, user_id=97f9ec826b694534aea8259cd817ac10, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.413242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7fb54-f2f5-4e56-b9bb-fbdf3ba7f5ef admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.454285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2385/4789] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:51 2022] PUT /identity/v3/projects/910afa67dcbc4f46a9403c5c9dd3a15e/users/97f9ec826b694534aea8259cd817ac10/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:51.454285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade473-40ec-4420-a2a8-8c0376fb99d7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.454285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade473-40ec-4420-a2a8-8c0376fb99d7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.454285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade473-40ec-4420-a2a8-8c0376fb99d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.460551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a449b96-58c1-4256-8d1c-5ad9e8f54e89 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.486955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4aade473-40ec-4420-a2a8-8c0376fb99d7 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:51.489455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4aade473-40ec-4420-a2a8-8c0376fb99d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:51.503635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:51.546850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea308ad-d09d-48de-9819-c1af7577d129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.547476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea308ad-d09d-48de-9819-c1af7577d129 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.547760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea308ad-d09d-48de-9819-c1af7577d129 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.548035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea308ad-d09d-48de-9819-c1af7577d129 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.548694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea308ad-d09d-48de-9819-c1af7577d129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.550519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea308ad-d09d-48de-9819-c1af7577d129 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.564302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2405/4790] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:51.565779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c28e4-07df-4e6b-9fb6-053d9c180f46 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.566221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c28e4-07df-4e6b-9fb6-053d9c180f46 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.566507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c28e4-07df-4e6b-9fb6-053d9c180f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.580535 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2386/4791] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:51.582087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade473-40ec-4420-a2a8-8c0376fb99d7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.599198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade473-40ec-4420-a2a8-8c0376fb99d7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.607099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.608034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.608536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.609152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] PATH_INFO: `/v3/projects/19ca75db834243588309524674d9a3b8/users/c119aa18f8f249a59fc2fd28754133b7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.612552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ca75db834243588309524674d9a3b8, user_id=c119aa18f8f249a59fc2fd28754133b7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.613700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a32c28e4-07df-4e6b-9fb6-053d9c180f46 None None] MFA Rules not processed for user `97f9ec826b694534aea8259cd817ac10`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:51.615006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a32c28e4-07df-4e6b-9fb6-053d9c180f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:51.616322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.637444 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2387/4792] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:51 2022] PUT /identity/v3/projects/19ca75db834243588309524674d9a3b8/users/c119aa18f8f249a59fc2fd28754133b7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:51.639087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.641529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:51.685362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:51.703548 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2406/4793] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:51.705149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.705591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.706250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.740963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] MFA Rules not processed for user `c119aa18f8f249a59fc2fd28754133b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:51.742813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:51.783138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939f6c-3a91-4e79-9c57-c7812f0f7e46 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:51.792820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:51.793724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:51.794990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:51.795275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:51.859643 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2407/4794] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:51.863606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:51.863606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:51.915543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:51.917429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:51.929324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:51.946255 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2388/4795] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:05:51.947339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:52.018803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bc22f-afd9-4cb6-92e3-abafc804d2a1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:52.019795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61aab1-bd43-4c9c-9cbf-332779036f67 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:52.029295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:52.029956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.030251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.030535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.031187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:52.045570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:52.047300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.048419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.048419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.052051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:52.115880 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2389/4796] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:51 2022] POST /identity/v3/projects => generated 333 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:52.126759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:52.169995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:52.172537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:52.190684 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2408/4797] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:52.269463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b70688-b0e4-405a-92a0-2957dc289c63 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:52.277285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:52.278431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.278944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.279451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.280193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:52.282310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:52.333024 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2390/4798] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:05:52 2022] POST /identity/v3/projects => generated 332 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:52.333675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:52.351971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048ab35-76a7-4015-9799-e1a53843cdce service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:52.411398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:52.461580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:52.486391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:52.488246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.488946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.489390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.490203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:52.492771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:52.542072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17d84d-5eab-4de9-b940-4c1817f84254 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:52.557527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:52.559239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.559971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.560139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] PATH_INFO: `/v3/users/97f9ec826b694534aea8259cd817ac10` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.561940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f9ec826b694534aea8259cd817ac10)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:52.564574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:52.582793 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2409/4799] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:52 2022] POST /identity/v3/projects => generated 358 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:05:52.587986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:52.663106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f9ec826b694534aea8259cd817ac10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:52.679535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f9ec826b694534aea8259cd817ac10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:52.681257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f9ec826b694534aea8259cd817ac10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:52.709512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f9ec826b694534aea8259cd817ac10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:52.712370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9b987c8c-938e-4f53-8d33-020cdcf5f325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f9ec826b694534aea8259cd817ac10'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:52.733586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:52.765555 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2391/4800] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:52 2022] DELETE /identity/v3/users/97f9ec826b694534aea8259cd817ac10 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:52.766624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336b074-351d-4ef4-a886-f302e8ec117d tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:52.770809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32359e-787c-4418-a709-54984a0b510c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.771339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32359e-787c-4418-a709-54984a0b510c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.771822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32359e-787c-4418-a709-54984a0b510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.803493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fd32359e-787c-4418-a709-54984a0b510c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:52.805510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fd32359e-787c-4418-a709-54984a0b510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:52.895382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:52.896055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.898115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.898731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.899341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:52.902269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:52.920502 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2392/4801] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:05:52.938414 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2410/4802] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:05:52 2022] GET /identity/v3/projects => generated 7541 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:52.943738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32359e-787c-4418-a709-54984a0b510c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:52.944048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:52.946148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fd32359e-787c-4418-a709-54984a0b510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:52.966464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:52.973028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32359e-787c-4418-a709-54984a0b510c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:52.989246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:52.989959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:52.990262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:52.990558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:52.991294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:52.991529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:52.994160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:53.022465 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2393/4803] 10.222.0.33 () {58 vars in 1158 bytes} [Sat Sep 10 21:05:52 2022] GET /identity/v3/projects?enabled=False => generated 444 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:53.029286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:53.047163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56024f-c591-4951-bba6-e07eb03a7bdb tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:53.063929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:53.064786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:53.065177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:53.065558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:53.066562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Adding query filter params (parent_id=c31a8113283841eca809d8bae44dd95a) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:53.074167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:53.079479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:53.109788 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2394/4804] 10.222.0.33 () {58 vars in 1216 bytes} [Sat Sep 10 21:05:53 2022] GET /identity/v3/projects?parent_id=c31a8113283841eca809d8bae44dd95a => generated 498 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:05:53.114737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:53.138310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468237-6131-481b-88e6-90bfad8e2052 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:53.140205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdce50c-0a89-4809-963e-9a3137f0f352 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:53.146848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:53.147411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:53.148809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:53.149526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:53.152174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:53.152841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:53.153195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:53.153517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects/4221fbec2111459d892424dd6eb1cd7e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:53.156564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4221fbec2111459d892424dd6eb1cd7e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:53.158707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:53.168391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:53.174509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:53.191902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:53.204004 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2411/4805] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:53.300002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4221fbec2111459d892424dd6eb1cd7e', 'request_id': 'req-25e92ed6-00fb-4879-9693-40c5930bb456'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.311602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4221fbec2111459d892424dd6eb1cd7e', 'request_id': 'req-25e92ed6-00fb-4879-9693-40c5930bb456'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.324577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4221fbec2111459d892424dd6eb1cd7e', 'request_id': 'req-25e92ed6-00fb-4879-9693-40c5930bb456'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.348650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] The token cache is being invalidate because project 4221fbec2111459d892424dd6eb1cd7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:53.348650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.354738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.360823 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2395/4806] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:53 2022] DELETE /identity/v3/projects/4221fbec2111459d892424dd6eb1cd7e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:53.361986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:53.397310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503f6dd-bb7e-41bc-96df-47fee47916e0 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:53.535691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:53.543519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:53.544072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:53.544582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects/c31a8113283841eca809d8bae44dd95a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:53.556284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c31a8113283841eca809d8bae44dd95a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:53.558582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:53.734745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:53.753134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31a8113283841eca809d8bae44dd95a', 'request_id': 'req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.775398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31a8113283841eca809d8bae44dd95a', 'request_id': 'req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.795126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31a8113283841eca809d8bae44dd95a', 'request_id': 'req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.823363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] The token cache is being invalidate because project c31a8113283841eca809d8bae44dd95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:53.824201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.825684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:53.834805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:53.836168 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2412/4807] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:53 2022] DELETE /identity/v3/projects/c31a8113283841eca809d8bae44dd95a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:53.837445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:53.858673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e92ed6-00fb-4879-9693-40c5930bb456 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:54.001101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c119aa18f8f249a59fc2fd28754133b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ca75db834243588309524674d9a3b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:54.001776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:54.002098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:54.002507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] PATH_INFO: `/v3/projects/b35cf1a9628f4f61af87b9a72be5f3c1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:54.005896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b35cf1a9628f4f61af87b9a72be5f3c1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:54.008305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:54.158610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35cf1a9628f4f61af87b9a72be5f3c1', 'request_id': 'req-2df334d1-00c2-421e-aae1-d252831c037b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.166631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35cf1a9628f4f61af87b9a72be5f3c1', 'request_id': 'req-2df334d1-00c2-421e-aae1-d252831c037b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.178235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35cf1a9628f4f61af87b9a72be5f3c1', 'request_id': 'req-2df334d1-00c2-421e-aae1-d252831c037b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.200085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] The token cache is being invalidate because project b35cf1a9628f4f61af87b9a72be5f3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:54.200706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.201746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2df334d1-00c2-421e-aae1-d252831c037b tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.203825 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2396/4808] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:53 2022] DELETE /identity/v3/projects/b35cf1a9628f4f61af87b9a72be5f3c1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:54.208993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:54.210511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:54.346235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66ca0c-8fa9-4f3d-9530-a0011244ac19 tempest-ListProjectsTestJSON-1636128458 tempest-ListProjectsTestJSON-1636128458-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:54.410590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:54.411150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:54.411491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:54.411773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] PATH_INFO: `/v3/users/5cc5fe28677445ad8e2bb6b74a458738` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:54.412949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc5fe28677445ad8e2bb6b74a458738)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:54.415784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:54.484998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc5fe28677445ad8e2bb6b74a458738'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.502826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc5fe28677445ad8e2bb6b74a458738'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.513490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc5fe28677445ad8e2bb6b74a458738'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.517011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:54.527019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc5fe28677445ad8e2bb6b74a458738'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.529357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc5fe28677445ad8e2bb6b74a458738'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.539706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2413/4809] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:54 2022] DELETE /identity/v3/users/5cc5fe28677445ad8e2bb6b74a458738 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:54.544935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:54.545513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:54.545513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:54.556627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:54.558231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:54.588381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:54.589636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:54.662157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42691c-31cc-450c-9072-8f4597142539 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:54.665975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:54.674693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:54.678067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:54.678479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:54.679087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] PATH_INFO: `/v3/projects/910afa67dcbc4f46a9403c5c9dd3a15e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:54.685813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=910afa67dcbc4f46a9403c5c9dd3a15e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:54.688512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:54.737618 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2397/4810] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:54.751959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:54.753675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:54.792988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '910afa67dcbc4f46a9403c5c9dd3a15e', 'request_id': 'req-8aebb585-4f6a-4020-a65c-6f763cc62ae8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.803380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '910afa67dcbc4f46a9403c5c9dd3a15e', 'request_id': 'req-8aebb585-4f6a-4020-a65c-6f763cc62ae8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.816923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '910afa67dcbc4f46a9403c5c9dd3a15e', 'request_id': 'req-8aebb585-4f6a-4020-a65c-6f763cc62ae8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.853563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] The token cache is being invalidate because project 910afa67dcbc4f46a9403c5c9dd3a15e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:54.853896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.854969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8aebb585-4f6a-4020-a65c-6f763cc62ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:54.858283 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2414/4811] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:54 2022] DELETE /identity/v3/projects/910afa67dcbc4f46a9403c5c9dd3a15e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:54.863755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be031678-6ba7-4ab3-b29d-b45ffd2b0cfb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:54.864231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be031678-6ba7-4ab3-b29d-b45ffd2b0cfb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:54.865148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be031678-6ba7-4ab3-b29d-b45ffd2b0cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:54.902404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840792c-0fbb-4323-9139-b3ef860d3275 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:54.908319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-be031678-6ba7-4ab3-b29d-b45ffd2b0cfb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:54.909357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-be031678-6ba7-4ab3-b29d-b45ffd2b0cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:55.047889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.048617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.048798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.049077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.052035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:55.108184 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2415/4812] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:55.113966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c7c6-a284-48d6-9c96-33803dbbbcd1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.113966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c7c6-a284-48d6-9c96-33803dbbbcd1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.113966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609c7c6-a284-48d6-9c96-33803dbbbcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.179331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e609c7c6-a284-48d6-9c96-33803dbbbcd1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:55.181196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e609c7c6-a284-48d6-9c96-33803dbbbcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:55.227038 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2416/4813] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:55.231418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609c7c6-a284-48d6-9c96-33803dbbbcd1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:55.244598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609c7c6-a284-48d6-9c96-33803dbbbcd1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:55.250723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.252289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.252679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.252963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.254038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:55.254352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.256282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.266862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.268328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.276035 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2417/4814] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:05:55.277619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:55.282331 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2398/4815] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:05:55.284683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:55.291187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dda296-ab1a-48d3-9320-091e23dbf1f0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:55.298129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.299384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.299871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.300189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.300791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.303037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.343515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2418/4816] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:55 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:55.348974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:55.369138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b76d6-fb69-4b37-ae44-23e10575e71d service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:55.413116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:55.445307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.446281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.446527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.446982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.447737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.449953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.478678 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2399/4817] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:55 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:55.482679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:55.494972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf9ae2-55e1-413d-af31-06b0f78f23c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:55.500019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.500816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.502079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.502079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.502322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.504147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.519652 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2419/4818] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:05:55.521252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:55.531854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0fdcf-3fc0-429f-bc30-2bcd9d631546 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:55.537521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e968b3-3871-4a2f-895f-c8987939d2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.538327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e968b3-3871-4a2f-895f-c8987939d2ca admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.539078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e968b3-3871-4a2f-895f-c8987939d2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.539078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e968b3-3871-4a2f-895f-c8987939d2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.539945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e968b3-3871-4a2f-895f-c8987939d2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.541677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e968b3-3871-4a2f-895f-c8987939d2ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.558719 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2400/4819] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:55.560038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:55.585235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817077f8-955d-4202-b512-b50738901ca0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:55.593872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.597207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.597678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.598093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] PATH_INFO: `/v3/projects/9b196d42c2d6416c81818922af07b8a3/users/13e5077421c44eb7abcda40582c7ea9a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.619256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b196d42c2d6416c81818922af07b8a3, user_id=13e5077421c44eb7abcda40582c7ea9a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.622257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.637895 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2420/4820] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:55 2022] PUT /identity/v3/projects/9b196d42c2d6416c81818922af07b8a3/users/13e5077421c44eb7abcda40582c7ea9a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:55.642015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.643368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.643872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.666991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:55.668636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:55.754338 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2401/4821] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:55.780096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:55.782549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:55.901033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc55c-bb97-4ba6-8deb-6db82dbd0045 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:55.908296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:55.908296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:55.908296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:55.908296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:55.921387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:55.923834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:55.935314 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2421/4822] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:05:56.593163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:56.594927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:56.614848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7a486-2b57-4e86-9ab7-96720f2c4399 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:56.622338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:56.623111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:56.623695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:56.624066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] PATH_INFO: `/v3/projects/1f0b60aeb69544368b486012b41c7ab5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:56.625434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0b60aeb69544368b486012b41c7ab5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:56.628166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:56.739609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0b60aeb69544368b486012b41c7ab5', 'request_id': 'req-50b72e2c-d405-453c-8b0f-eb9f065f3aec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:56.752671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0b60aeb69544368b486012b41c7ab5', 'request_id': 'req-50b72e2c-d405-453c-8b0f-eb9f065f3aec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:56.768902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0b60aeb69544368b486012b41c7ab5', 'request_id': 'req-50b72e2c-d405-453c-8b0f-eb9f065f3aec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:56.801767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] The token cache is being invalidate because project 1f0b60aeb69544368b486012b41c7ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:56.802358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:56.803046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:56.806399 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2402/4823] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:56 2022] DELETE /identity/v3/projects/1f0b60aeb69544368b486012b41c7ab5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:05:56.810626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:56.824383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd5450-92a9-4d0f-9160-e78b30a99c3e service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:56.957339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:56.958068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:56.959370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:56.959657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] PATH_INFO: `/v3/users/c119aa18f8f249a59fc2fd28754133b7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:56.965278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c119aa18f8f249a59fc2fd28754133b7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:56.967516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:57.078242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c119aa18f8f249a59fc2fd28754133b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:57.083709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:57.101774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c119aa18f8f249a59fc2fd28754133b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:57.120585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c119aa18f8f249a59fc2fd28754133b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:57.133549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c119aa18f8f249a59fc2fd28754133b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:57.134717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e836a858-0b81-404f-8c11-ebfc357c1639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c119aa18f8f249a59fc2fd28754133b7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:57.152100 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2422/4824] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:05:56 2022] DELETE /identity/v3/users/c119aa18f8f249a59fc2fd28754133b7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:58.317643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:58.319706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:58.344811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b72e2c-d405-453c-8b0f-eb9f065f3aec admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:58.481177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:58.538476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:58.539850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:58.540596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:58.541359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] PATH_INFO: `/v3/projects/19ca75db834243588309524674d9a3b8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:58.542808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ca75db834243588309524674d9a3b8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:58.545393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:58.666199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ca75db834243588309524674d9a3b8', 'request_id': 'req-40307a4c-2d2f-450a-9127-e2b0b33ec114'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:58.684664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ca75db834243588309524674d9a3b8', 'request_id': 'req-40307a4c-2d2f-450a-9127-e2b0b33ec114'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:58.702760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ca75db834243588309524674d9a3b8', 'request_id': 'req-40307a4c-2d2f-450a-9127-e2b0b33ec114'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:58.742810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] The token cache is being invalidate because project 19ca75db834243588309524674d9a3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:05:58.742810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:58.748885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40307a4c-2d2f-450a-9127-e2b0b33ec114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:05:58.754007 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2403/4825] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:05:58 2022] DELETE /identity/v3/projects/19ca75db834243588309524674d9a3b8 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:58.756720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637ebe-1861-4c5d-b6be-1372c16d0695 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:58.758143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637ebe-1861-4c5d-b6be-1372c16d0695 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:58.758916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637ebe-1861-4c5d-b6be-1372c16d0695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:58.808340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-20637ebe-1861-4c5d-b6be-1372c16d0695 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:58.812300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20637ebe-1861-4c5d-b6be-1372c16d0695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:58.970138 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2423/4826] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:58.977767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42781f-1cf1-45ec-ab80-558f2ed71695 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:58.979341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42781f-1cf1-45ec-ab80-558f2ed71695 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:58.979855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42781f-1cf1-45ec-ab80-558f2ed71695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.013645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0d42781f-1cf1-45ec-ab80-558f2ed71695 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:59.015112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0d42781f-1cf1-45ec-ab80-558f2ed71695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:59.034719 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2404/4827] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:59.035532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637ebe-1861-4c5d-b6be-1372c16d0695 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:59.048240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637ebe-1861-4c5d-b6be-1372c16d0695 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:59.065332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:59.066385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.066889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.067400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.068346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:05:59.068871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:59.071945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:59.100847 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2424/4828] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:05:59.105036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42781f-1cf1-45ec-ab80-558f2ed71695 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:59.116441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42781f-1cf1-45ec-ab80-558f2ed71695 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:59.130741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:59.131379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.131661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.131942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.132637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:59.134811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:59.201268 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2405/4829] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:59 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:59.205498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:59.221464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac5f4e-9112-4e68-afcd-42d3b333c319 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:59.343555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:59.344269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.344629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.344976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.345691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:59.348728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:59.406347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2425/4830] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:05:59 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:05:59.410281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:59.421530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf63cc-53cb-46bc-bd82-a62365201829 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:59.431637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:59.432529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.432969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.434382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.435303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:59.437913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:59.456595 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2406/4831] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:59.460756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:59.478795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44fd42-d45a-496d-bb2c-12d4b7ebd3c4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:59.492310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd06a52-e923-4c0d-a5d7-9412422695ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:59.492910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd06a52-e923-4c0d-a5d7-9412422695ee admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.493016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd06a52-e923-4c0d-a5d7-9412422695ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.493282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd06a52-e923-4c0d-a5d7-9412422695ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.493980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd06a52-e923-4c0d-a5d7-9412422695ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:59.496632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd06a52-e923-4c0d-a5d7-9412422695ee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:59.516052 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2426/4832] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:05:59.517512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:59.529391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c8022-d698-4f27-b481-5502ea67b79d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:59.535250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:59.536141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.536582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.537484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] PATH_INFO: `/v3/projects/99f1de7ee98c4cc5941b456648845554/users/59a1313275c2436994ceaac5265925a2/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.560801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f1de7ee98c4cc5941b456648845554, user_id=59a1313275c2436994ceaac5265925a2, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:05:59.564359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:05:59.588088 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2407/4833] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:05:59 2022] PUT /identity/v3/projects/99f1de7ee98c4cc5941b456648845554/users/59a1313275c2436994ceaac5265925a2/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:05:59.603211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.603500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.603893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.647122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] MFA Rules not processed for user `59a1313275c2436994ceaac5265925a2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:05:59.648364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:05:59.691008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:05:59.802946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:05:59.806291 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2427/4834] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:05:59.981792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf5b88-0110-4086-9503-eab11b6c03ca admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:05:59.994930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:05:59.995582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:05:59.995871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:05:59.996154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:05:59.996806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.002698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:00.031820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:00.050487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:00.052533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:00.076344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2408/4835] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:05:59 2022] POST /identity/v3/projects => generated 365 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:00.077337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:00.190909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4c381-d61b-4614-b51d-92bc3df02ddd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:00.197655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:00.198510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.198801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.199147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.199918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.209078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:00.279510 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2409/4836] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:00 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:00.282637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:00.296345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36ae7b-a568-4cdd-ad62-9cc3ee4b6363 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:00.324307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7416-773e-403a-ae75-3b6268d840e6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:00.338404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:00.339032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.342314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.342631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.343326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.345475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:00.375278 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2410/4837] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:00.376561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:00.390331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cd03a-60a0-40ff-993f-c4f90fb73911 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:00.401306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:00.402115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.402399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.402669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] PATH_INFO: `/v3/projects/399da206101243b7ae45576f63cb65a6/users/3ab655fef5df4f78a862946bb1cb0732/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.410599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:00.414099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.418228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.418806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.423045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399da206101243b7ae45576f63cb65a6, user_id=3ab655fef5df4f78a862946bb1cb0732, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.423968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:00.427818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:00.471329 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2411/4838] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:00 2022] PUT /identity/v3/projects/399da206101243b7ae45576f63cb65a6/users/3ab655fef5df4f78a862946bb1cb0732/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:00.475975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:00.482712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.484594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:00.531750 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2428/4839] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:00.610876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbe8c-753b-4827-962a-4ccf57a0fa74 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:00.617803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae1f1-d371-4bfd-8cf2-5195112d379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:00.619906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae1f1-d371-4bfd-8cf2-5195112d379f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.620408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae1f1-d371-4bfd-8cf2-5195112d379f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.620890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae1f1-d371-4bfd-8cf2-5195112d379f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.622757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae1f1-d371-4bfd-8cf2-5195112d379f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.634099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae1f1-d371-4bfd-8cf2-5195112d379f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:00.653981 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2412/4840] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:00.658089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:00.680330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c83033-1b1d-4e8e-9d82-d46bb158769a service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:00.687289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:00.687960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.688253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.688695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] PATH_INFO: `/v3/projects/399da206101243b7ae45576f63cb65a6/users/3ab655fef5df4f78a862946bb1cb0732/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.691420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=399da206101243b7ae45576f63cb65a6, user_id=3ab655fef5df4f78a862946bb1cb0732, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.694694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:00.709423 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2429/4841] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:00 2022] PUT /identity/v3/projects/399da206101243b7ae45576f63cb65a6/users/3ab655fef5df4f78a862946bb1cb0732/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:00.720654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de359f-b047-4b5d-97b1-a88c4647ee05 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.720966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de359f-b047-4b5d-97b1-a88c4647ee05 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.721554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de359f-b047-4b5d-97b1-a88c4647ee05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.818259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e6de359f-b047-4b5d-97b1-a88c4647ee05 None None] MFA Rules not processed for user `3ab655fef5df4f78a862946bb1cb0732`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:00.818259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e6de359f-b047-4b5d-97b1-a88c4647ee05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:00.876481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:00.942065 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2413/4842] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:00.946926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:00.964410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb4865-a340-4c8a-b7bf-8925b1f9b98f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:00.972136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:00.973008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:00.973480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:00.974011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:00.974864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:00.977651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.033325 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2430/4843] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:06:00 2022] POST /identity/v3/domains => generated 277 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:01.037859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de359f-b047-4b5d-97b1-a88c4647ee05 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.068930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de359f-b047-4b5d-97b1-a88c4647ee05 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.184455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.185533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.186033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.186495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.187630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.192297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.287584 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2414/4844] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:01 2022] POST /identity/v3/users => generated 347 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:01.292177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.311575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f51325-9ad9-4823-846c-29bd29be9bc8 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.331828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.333074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.333528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.334544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.337649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.345713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.439322 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2431/4845] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:01 2022] POST /identity/v3/users => generated 322 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:01.443880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.458088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d493d-5347-4104-a0bf-1e1ea3c87041 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.468780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.469669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.469971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.470888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users/a5f9524535ee41eb852bfde55a4b4633` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.514850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:get_user(user_id=a5f9524535ee41eb852bfde55a4b4633)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.519102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.525370 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2415/4846] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:06:01 2022] GET /identity/v3/users/a5f9524535ee41eb852bfde55a4b4633 => generated 347 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:01.530712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.536963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:01.547893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1c18d-061a-4b22-a81b-0d8632f70444 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.573120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.573758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.574073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.574353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.575088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Adding query filter params (domain_id=e9ad15d7450f4347afd0a7afa7e447a5) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:01.575366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.577547 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:06:01.577547 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:06:01.577547 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:06:01.578476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.594841 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2432/4847] 10.222.0.33 () {58 vars in 1207 bytes} [Sat Sep 10 21:06:01 2022] GET /identity/v3/users?domain_id=e9ad15d7450f4347afd0a7afa7e447a5 => generated 484 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:01.599503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.613052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c9f02-7f7b-4d54-ab1f-2236d0d15877 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.625743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.628715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.629046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.630012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.632138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.633260 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:06:01.633260 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:06:01.633260 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:06:01.634447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.685386 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2416/4848] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:01 2022] GET /identity/v3/users => generated 8855 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:01.690684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.705915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e43a-aa6a-42aa-9a7f-af12c0532de3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.712592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.713421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.713856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.714303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.715165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Adding query filter params (name=tempest-test_user-1427468231) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:01.715592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.718376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.776404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2433/4849] 10.222.0.33 () {58 vars in 1189 bytes} [Sat Sep 10 21:06:01 2022] GET /identity/v3/users?name=tempest-test_user-1427468231 => generated 475 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:01.778094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.795413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b10c-c48f-4920-aa75-f9504ef0bbd3 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.810750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.811389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.811675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.811954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.812843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:01.813198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.819870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:01.877787 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2417/4850] 10.222.0.33 () {58 vars in 1149 bytes} [Sat Sep 10 21:06:01 2022] GET /identity/v3/users?enabled=False => generated 430 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:01.881348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:01.903309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f4ff8-84f0-48a4-b7fd-0c136b384b47 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:01.923743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:01.924084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:01.924332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:01.924593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users/2f1df7eb2f2340d0998e2ad892937674` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:01.950793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:delete_user(user_id=2f1df7eb2f2340d0998e2ad892937674)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:01.952750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:02.045198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1df7eb2f2340d0998e2ad892937674'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.069850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1df7eb2f2340d0998e2ad892937674'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.084940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1df7eb2f2340d0998e2ad892937674'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.098137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1df7eb2f2340d0998e2ad892937674'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.098900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1df7eb2f2340d0998e2ad892937674'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.114620 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2434/4851] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:01 2022] DELETE /identity/v3/users/2f1df7eb2f2340d0998e2ad892937674 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:02.115595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:02.118667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:02.146750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16650a91-0d1e-4812-b55c-aeaf68251bb9 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:02.288979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:02.291042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:02.291500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:02.292117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/users/a5f9524535ee41eb852bfde55a4b4633` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:02.293752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a5f9524535ee41eb852bfde55a4b4633)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:02.297996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:02.322665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:02.413993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f9524535ee41eb852bfde55a4b4633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.446215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f9524535ee41eb852bfde55a4b4633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.447558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f9524535ee41eb852bfde55a4b4633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.461725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f9524535ee41eb852bfde55a4b4633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.465286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f9524535ee41eb852bfde55a4b4633'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.506130 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2418/4852] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:02 2022] DELETE /identity/v3/users/a5f9524535ee41eb852bfde55a4b4633 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:02.509989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:02.538058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0500e7-9264-4a25-8759-ecb5a5f285b6 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:02.672469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:02.673694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:02.674687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:02.675066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/domains/e9ad15d7450f4347afd0a7afa7e447a5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:02.676552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=e9ad15d7450f4347afd0a7afa7e447a5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:02.678557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:02.712810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.713988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.828068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.830757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:02.845135 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2435/4853] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:06:02 2022] PATCH /identity/v3/domains/e9ad15d7450f4347afd0a7afa7e447a5 => generated 278 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:02.848634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:02.851643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:02.876618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be630535-fe59-4c42-be59-2712746d4cad tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:03.046423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab655fef5df4f78a862946bb1cb0732', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399da206101243b7ae45576f63cb65a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:03.047086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:03.047367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:03.047644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] PATH_INFO: `/v3/domains/e9ad15d7450f4347afd0a7afa7e447a5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:03.065902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=e9ad15d7450f4347afd0a7afa7e447a5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:03.067676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:03.097414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.102706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.161767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5', 'request_id': 'req-8280789b-8439-4e24-ac92-b2a6b0d74b30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.180111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5', 'request_id': 'req-8280789b-8439-4e24-ac92-b2a6b0d74b30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.208236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:03.225468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5', 'request_id': 'req-8280789b-8439-4e24-ac92-b2a6b0d74b30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.244437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:03.246110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:03.252151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:03.254357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] The token cache is being invalidate because project e9ad15d7450f4347afd0a7afa7e447a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:03.254658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.255580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.297409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5', 'request_id': 'req-8280789b-8439-4e24-ac92-b2a6b0d74b30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.298946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5', 'request_id': 'req-8280789b-8439-4e24-ac92-b2a6b0d74b30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.300606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'e9ad15d7450f4347afd0a7afa7e447a5', 'request_id': 'req-8280789b-8439-4e24-ac92-b2a6b0d74b30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.325171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2419/4854] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:02 2022] DELETE /identity/v3/domains/e9ad15d7450f4347afd0a7afa7e447a5 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:03.326218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:03.329239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:03.403345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ed79-5754-4c64-a5d2-e6547ec191ea tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:03.482121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280789b-8439-4e24-ac92-b2a6b0d74b30 tempest-UsersV3TestJSON-414884655 tempest-UsersV3TestJSON-414884655-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:03.489988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:03.491032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:03.491487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:03.492158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] PATH_INFO: `/v3/users/59a1313275c2436994ceaac5265925a2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:03.493994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a1313275c2436994ceaac5265925a2)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:03.496414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:03.497200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:03.497592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:03.498205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] PATH_INFO: `/v3/users/5215babeaff64f97914a58db6dc33db1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:03.498427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:03.499707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5215babeaff64f97914a58db6dc33db1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:03.502919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:03.577525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5215babeaff64f97914a58db6dc33db1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.581171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a1313275c2436994ceaac5265925a2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.591116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a1313275c2436994ceaac5265925a2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.592198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a1313275c2436994ceaac5265925a2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.601471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5215babeaff64f97914a58db6dc33db1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.604016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a1313275c2436994ceaac5265925a2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.605138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9844790-8e87-418b-ab21-bdd87b9741d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a1313275c2436994ceaac5265925a2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.611034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5215babeaff64f97914a58db6dc33db1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.624078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5215babeaff64f97914a58db6dc33db1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.624920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-119a7ee0-3aa3-4561-80bd-af2fa3add494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5215babeaff64f97914a58db6dc33db1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:03.642140 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2436/4855] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:03 2022] DELETE /identity/v3/users/5215babeaff64f97914a58db6dc33db1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:03.644740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:03.645535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:03.646181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:03.666964 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2420/4856] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:03 2022] DELETE /identity/v3/users/59a1313275c2436994ceaac5265925a2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:03.671461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:03.671868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:03.672341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:03.678339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:03.680467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:03.701779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:03.704792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:03.727566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:03.787029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:03.799408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2437/4857] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:03.819562 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2421/4858] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:03.822682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:03.824799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:03.838075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:03.841216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:03.919065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efc85c-20f4-47fb-88f9-3d38cd758d56 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:03.924227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:03.925250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:03.925991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:03.926410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:03.948191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:03.949767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:03.968636 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2438/4859] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:03.968783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eec334-45e1-410c-ae7e-0de2e0a7fae7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:03.974250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc3172-8982-4617-9944-b9c2dcb8ad1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:03.974995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc3172-8982-4617-9944-b9c2dcb8ad1c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:03.975645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc3172-8982-4617-9944-b9c2dcb8ad1c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:03.976639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc3172-8982-4617-9944-b9c2dcb8ad1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:03.990654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc3172-8982-4617-9944-b9c2dcb8ad1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:03.992350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc3172-8982-4617-9944-b9c2dcb8ad1c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:04.004110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2422/4860] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:04.547935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:04.550769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:04.566377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929d3c0-c095-4aab-8659-ac49b4c52ab1 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:04.573624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:04.574346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:04.574865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:04.582068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] PATH_INFO: `/v3/projects/4a4f0dbb2e68458ab1e50702fc0e9dfa` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:04.583770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4f0dbb2e68458ab1e50702fc0e9dfa)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:04.586831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:04.662722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4f0dbb2e68458ab1e50702fc0e9dfa', 'request_id': 'req-70fe927e-f452-49b0-acb3-3aa8268dc6d5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:04.671114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4f0dbb2e68458ab1e50702fc0e9dfa', 'request_id': 'req-70fe927e-f452-49b0-acb3-3aa8268dc6d5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:04.681389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4f0dbb2e68458ab1e50702fc0e9dfa', 'request_id': 'req-70fe927e-f452-49b0-acb3-3aa8268dc6d5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:04.699144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] The token cache is being invalidate because project 4a4f0dbb2e68458ab1e50702fc0e9dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:04.699491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:04.700162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:04.702295 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2439/4861] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:04 2022] DELETE /identity/v3/projects/4a4f0dbb2e68458ab1e50702fc0e9dfa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:04.707324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229e552-1fe3-47c5-a33a-fb7e5517f642 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:04.707821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229e552-1fe3-47c5-a33a-fb7e5517f642 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:04.708098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229e552-1fe3-47c5-a33a-fb7e5517f642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:04.782226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9229e552-1fe3-47c5-a33a-fb7e5517f642 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:04.784623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9229e552-1fe3-47c5-a33a-fb7e5517f642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:04.838963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:04.840815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:04.941757 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2423/4862] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:04.979328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9673-337d-4670-a6df-78d0b052a9e7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:04.979593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9673-337d-4670-a6df-78d0b052a9e7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:04.979876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9673-337d-4670-a6df-78d0b052a9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:04.986354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:04.996289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe927e-f452-49b0-acb3-3aa8268dc6d5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.006440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.009387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.010247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.010863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] PATH_INFO: `/v3/projects/99f1de7ee98c4cc5941b456648845554` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.015700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f1de7ee98c4cc5941b456648845554)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.020340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f81c9673-337d-4670-a6df-78d0b052a9e7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:05.020533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.034874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f81c9673-337d-4670-a6df-78d0b052a9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:05.058016 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2424/4863] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:05.063202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c9673-337d-4670-a6df-78d0b052a9e7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.083870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c9673-337d-4670-a6df-78d0b052a9e7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.095958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f1de7ee98c4cc5941b456648845554', 'request_id': 'req-1c229981-90c7-443e-b5ce-a49d902c12df'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.104186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f1de7ee98c4cc5941b456648845554', 'request_id': 'req-1c229981-90c7-443e-b5ce-a49d902c12df'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.112937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f1de7ee98c4cc5941b456648845554', 'request_id': 'req-1c229981-90c7-443e-b5ce-a49d902c12df'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.136903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] The token cache is being invalidate because project 99f1de7ee98c4cc5941b456648845554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:05.137356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.139787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.146754 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2440/4864] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:04 2022] DELETE /identity/v3/projects/99f1de7ee98c4cc5941b456648845554 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:05.147662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.151630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:05.176857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.178256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.178613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.178997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.180899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:05.181190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.183762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.203765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2425/4865] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:05.208245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.211690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:05.240727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d825d4-ac1d-425c-a0e1-a5c4cdd17b7d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.249739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.250797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.251461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.251957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.252790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.256037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.267437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c229981-90c7-443e-b5ce-a49d902c12df admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.274043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.275345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.275825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.276231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] PATH_INFO: `/v3/users/3ab655fef5df4f78a862946bb1cb0732` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.277531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab655fef5df4f78a862946bb1cb0732)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.279808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.288611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:05.318829 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2426/4866] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:05 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:05.325756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.354255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab655fef5df4f78a862946bb1cb0732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.378813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab655fef5df4f78a862946bb1cb0732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.393145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab655fef5df4f78a862946bb1cb0732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.406741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab655fef5df4f78a862946bb1cb0732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.407693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab655fef5df4f78a862946bb1cb0732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:05.416642 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2441/4867] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:05 2022] DELETE /identity/v3/users/3ab655fef5df4f78a862946bb1cb0732 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:05.421303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72093f-e735-4ab3-b6a7-2644e7d9f3f6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.500170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.500884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.501232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.501567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.502316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.504336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.538263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2427/4868] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:05 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:05.540171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.541615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:05.552212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:05.557000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778cbbb-e1f1-40e6-96ff-02c2a86fd733 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.565864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.566819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.567716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.568614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.570183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.571858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.584593 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2442/4869] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:05.589213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.605138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7139-8b5a-43ed-b7fe-f4ab9182755b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.611154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84071a41-1ab3-4dde-97c4-503d9b21c637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.611932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84071a41-1ab3-4dde-97c4-503d9b21c637 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.612392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84071a41-1ab3-4dde-97c4-503d9b21c637 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.612896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84071a41-1ab3-4dde-97c4-503d9b21c637 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.613658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84071a41-1ab3-4dde-97c4-503d9b21c637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.615687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84071a41-1ab3-4dde-97c4-503d9b21c637 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.630041 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2428/4870] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:05.635015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.646570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dc7ba-7d16-49f3-957d-d5d1f586b735 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:05.652711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:05.653435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.653790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.654170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] PATH_INFO: `/v3/projects/839c3d9e368b42f78a055d529e7c5bfc/users/21e68efa994b4dd3b0dde8e160544267/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.672805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=839c3d9e368b42f78a055d529e7c5bfc, user_id=21e68efa994b4dd3b0dde8e160544267, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:05.675641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:05.704890 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2443/4871] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:05 2022] PUT /identity/v3/projects/839c3d9e368b42f78a055d529e7c5bfc/users/21e68efa994b4dd3b0dde8e160544267/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:05.706691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:05.707116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:05.707606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:05.748989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] MFA Rules not processed for user `21e68efa994b4dd3b0dde8e160544267`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:05.748989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:05.880163 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2429/4872] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:05.911329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.915247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:05.997587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:05.999563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:06.006196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f7088-19a9-4f2b-8ee6-55a77d48f47a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.010627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.011396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.012564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.012564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.022684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.025453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.027336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:06.037189 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2444/4873] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:06.038224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:06.040122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:06.192348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbc9bc-92fa-4676-8e41-57cfac412c14 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.197191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2310845-0284-4b80-8194-0fb26196a2f5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.199398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.200639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.200933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.201215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] PATH_INFO: `/v3/projects/399da206101243b7ae45576f63cb65a6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.202551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399da206101243b7ae45576f63cb65a6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.204500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.207236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3667187-b4a5-4c1a-9c91-74a0941c64fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.209952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3667187-b4a5-4c1a-9c91-74a0941c64fa service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.210275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3667187-b4a5-4c1a-9c91-74a0941c64fa service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.210858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3667187-b4a5-4c1a-9c91-74a0941c64fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.214333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e3667187-b4a5-4c1a-9c91-74a0941c64fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:06.290556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399da206101243b7ae45576f63cb65a6', 'request_id': 'req-b0c483a8-f889-460d-97a0-f6477e02ae00'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.323095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3667187-b4a5-4c1a-9c91-74a0941c64fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.330199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3667187-b4a5-4c1a-9c91-74a0941c64fa service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.331626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399da206101243b7ae45576f63cb65a6', 'request_id': 'req-b0c483a8-f889-460d-97a0-f6477e02ae00'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.342918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399da206101243b7ae45576f63cb65a6', 'request_id': 'req-b0c483a8-f889-460d-97a0-f6477e02ae00'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.365418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] The token cache is being invalidate because project 399da206101243b7ae45576f63cb65a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:06.366098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.368937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b0c483a8-f889-460d-97a0-f6477e02ae00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:06.369194 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2430/4874] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:06.371646 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2445/4875] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:06 2022] DELETE /identity/v3/projects/399da206101243b7ae45576f63cb65a6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:06.376737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec739-0d0a-4b6a-9ae9-ac4bb548da87 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.377065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec739-0d0a-4b6a-9ae9-ac4bb548da87 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.377375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bec739-0d0a-4b6a-9ae9-ac4bb548da87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.404254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b9bec739-0d0a-4b6a-9ae9-ac4bb548da87 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:06.405661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b9bec739-0d0a-4b6a-9ae9-ac4bb548da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:06.499732 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2431/4876] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:06.509632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bbb2-398c-41e3-9f0b-7fa3a7e55dcf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.509945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bbb2-398c-41e3-9f0b-7fa3a7e55dcf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.510212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bbb2-398c-41e3-9f0b-7fa3a7e55dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.525648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:06.556460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e392bbb2-398c-41e3-9f0b-7fa3a7e55dcf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:06.557958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e392bbb2-398c-41e3-9f0b-7fa3a7e55dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:06.587340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2446/4877] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:06.588310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bec739-0d0a-4b6a-9ae9-ac4bb548da87 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:06.606269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bec739-0d0a-4b6a-9ae9-ac4bb548da87 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.649475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.651303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.654180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.656538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.657324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:06.657715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.660053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.691058 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2432/4878] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:06.697117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bbb2-398c-41e3-9f0b-7fa3a7e55dcf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:06.711101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bbb2-398c-41e3-9f0b-7fa3a7e55dcf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.718833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.719935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.720221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.720498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.721274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.723863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.779521 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2447/4879] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:06 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:06.783713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:06.797711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e254e-9128-40ea-98f7-0781d9d700bc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:06.847403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:06.937345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:06.938286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:06.938783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:06.939248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:06.940004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:06.942060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:06.982932 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2433/4880] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:06 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:06.986987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.003239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dea4b-4b80-4f64-9eb1-7464f9c851fe admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.013662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.014304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.014585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.014860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.015498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.017184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.042290 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2448/4881] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:07.046345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.060932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31452970-69ac-45c3-99de-590c4adfb2a6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.073183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f1bce-1fcd-4eda-a40e-725aa8d0222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.073818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1bce-1fcd-4eda-a40e-725aa8d0222d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.074119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1bce-1fcd-4eda-a40e-725aa8d0222d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.074390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1bce-1fcd-4eda-a40e-725aa8d0222d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.075039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f1bce-1fcd-4eda-a40e-725aa8d0222d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.076637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f1bce-1fcd-4eda-a40e-725aa8d0222d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.105738 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2434/4882] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:07.109880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.123143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37726-29e3-4881-bad8-2b4365fcc303 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.136971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.138141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.139070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.139568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] PATH_INFO: `/v3/projects/ce3f758adba946139e588eb84eadd2ee/users/269dd863f5094d22829e0e46786af67c/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.180816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce3f758adba946139e588eb84eadd2ee, user_id=269dd863f5094d22829e0e46786af67c, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.183941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.222724 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2449/4883] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:07 2022] PUT /identity/v3/projects/ce3f758adba946139e588eb84eadd2ee/users/269dd863f5094d22829e0e46786af67c/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:07.227491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeb9de-ec70-4973-a6d5-723cacabf9e2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.228036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeb9de-ec70-4973-a6d5-723cacabf9e2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.228532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eeb9de-ec70-4973-a6d5-723cacabf9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.285914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-17eeb9de-ec70-4973-a6d5-723cacabf9e2 None None] MFA Rules not processed for user `269dd863f5094d22829e0e46786af67c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:07.290249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-17eeb9de-ec70-4973-a6d5-723cacabf9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:07.437800 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2435/4884] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:07.442297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.549927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3498f-ac27-432f-ab79-c96c72db3abe admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.556549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.558010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.558585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.559108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.561027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.563671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.615530 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2450/4885] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:07 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:07.620657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeb9de-ec70-4973-a6d5-723cacabf9e2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.638157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eeb9de-ec70-4973-a6d5-723cacabf9e2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.748871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.749614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.750022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.750494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.751403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.753279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.806915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2436/4886] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:07 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:07.809971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.825417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3caed-54a4-4439-a3d4-4c92976173c4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.839380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.840121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.840480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.840942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.841674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.846081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.875311 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2451/4887] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:07.876699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.891715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238a162-1aa2-469d-a66b-3eedcf33347b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:07.905632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:07.906464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:07.906812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:07.909754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] PATH_INFO: `/v3/projects/b5ab2028c1af4cf39e885ae7e15c4a0c/users/dd9f7def6c584abd8c16a1515ee239ec/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:07.935722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ab2028c1af4cf39e885ae7e15c4a0c, user_id=dd9f7def6c584abd8c16a1515ee239ec, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:07.938968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:07.957081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:07.968021 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2437/4888] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:07 2022] PUT /identity/v3/projects/b5ab2028c1af4cf39e885ae7e15c4a0c/users/dd9f7def6c584abd8c16a1515ee239ec/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:07.970885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:07.994643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0444c77-27eb-4a3b-a9c1-dd4389faa984 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.142661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15326bf3-fba3-486b-99de-233c66745b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.144383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15326bf3-fba3-486b-99de-233c66745b40 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.144869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15326bf3-fba3-486b-99de-233c66745b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.145320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15326bf3-fba3-486b-99de-233c66745b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.146208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15326bf3-fba3-486b-99de-233c66745b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.150106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15326bf3-fba3-486b-99de-233c66745b40 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.167840 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2452/4889] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:08.168030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.185723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a79464-5906-4d44-bf14-08ff41e15a84 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.200187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.201267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.201267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.201473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] PATH_INFO: `/v3/projects/b5ab2028c1af4cf39e885ae7e15c4a0c/users/dd9f7def6c584abd8c16a1515ee239ec/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.208935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ab2028c1af4cf39e885ae7e15c4a0c, user_id=dd9f7def6c584abd8c16a1515ee239ec, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.212248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.237846 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2438/4890] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:08 2022] PUT /identity/v3/projects/b5ab2028c1af4cf39e885ae7e15c4a0c/users/dd9f7def6c584abd8c16a1515ee239ec/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:08.250768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aefb8a-2d68-4f74-90b8-12e840b66b3e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.251143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aefb8a-2d68-4f74-90b8-12e840b66b3e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.251346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aefb8a-2d68-4f74-90b8-12e840b66b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.301958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:08.322639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-91aefb8a-2d68-4f74-90b8-12e840b66b3e None None] MFA Rules not processed for user `dd9f7def6c584abd8c16a1515ee239ec`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:08.323869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91aefb8a-2d68-4f74-90b8-12e840b66b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:08.490631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2453/4891] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:08.492588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.510124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ff90f-3e73-4c23-a61b-7882bd428ab2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.516969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.517906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.518377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.518799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.519762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.522865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.584312 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2439/4892] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/projects => generated 359 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:08.584518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aefb8a-2d68-4f74-90b8-12e840b66b3e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.604080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aefb8a-2d68-4f74-90b8-12e840b66b3e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.715149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.716019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.716303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.716583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.717407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.720011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.776681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.789340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee61f8-3d21-43e5-bb2c-eefdec6b2aac tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.790185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2454/4893] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:08 2022] POST /identity/v3/users => generated 413 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:08.809915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.811193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.811734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.812237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/users/3e3f710879464e74bd022acb72846003` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.853557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_user(user_id=3e3f710879464e74bd022acb72846003)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.856043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:08.864131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:08.864890 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2440/4894] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:06:08 2022] GET /identity/v3/users/3e3f710879464e74bd022acb72846003 => generated 413 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:08.887515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7678dc-1e1a-4803-aea6-04edc7d6a62c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:08.909938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:08.910610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:08.911013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:08.911297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/users/3e3f710879464e74bd022acb72846003` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:08.912631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3e3f710879464e74bd022acb72846003)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:08.923466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:09.013693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e3f710879464e74bd022acb72846003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.046904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e3f710879464e74bd022acb72846003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.057734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e3f710879464e74bd022acb72846003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.063365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:09.071488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e3f710879464e74bd022acb72846003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.073481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e3f710879464e74bd022acb72846003'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.086382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2455/4895] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:08 2022] DELETE /identity/v3/users/3e3f710879464e74bd022acb72846003 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:09.090313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:09.097459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:09.126060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f2a34-8b78-45f5-b28f-d2dbfe9d9ac0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:09.236959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:09.238370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:09.238659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:09.238944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/8dc2c62dc3f745e482314feb7b631ddc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:09.243210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8dc2c62dc3f745e482314feb7b631ddc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:09.245302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:09.328370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:09.332058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:09.345289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc2c62dc3f745e482314feb7b631ddc', 'request_id': 'req-eee56829-e711-4847-9e03-5a81193ceb03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.353346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc2c62dc3f745e482314feb7b631ddc', 'request_id': 'req-eee56829-e711-4847-9e03-5a81193ceb03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.363819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc2c62dc3f745e482314feb7b631ddc', 'request_id': 'req-eee56829-e711-4847-9e03-5a81193ceb03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.384365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 8dc2c62dc3f745e482314feb7b631ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:09.387498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.389671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.403566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2441/4896] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:09 2022] DELETE /identity/v3/projects/8dc2c62dc3f745e482314feb7b631ddc => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:09.408400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:09.411866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:09.458394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662a7e0-1be6-4a8a-91a6-fa07a4ef80de tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:09.493448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:09.538165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56829-e711-4847-9e03-5a81193ceb03 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:09.547076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:09.547745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:09.548035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:09.548316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:09.548956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:09.552252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:09.571432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:09.574130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:09.574571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:09.575439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] PATH_INFO: `/v3/users/c3a02193633a44dda1885d363fcd5361` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:09.577118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a02193633a44dda1885d363fcd5361)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:09.581783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:09.592110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2442/4897] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:09 2022] POST /identity/v3/projects => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:09.596474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:09.695758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a02193633a44dda1885d363fcd5361'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.724737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a02193633a44dda1885d363fcd5361'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.742987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a02193633a44dda1885d363fcd5361'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.748565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98951ce-f3c1-4487-bfa3-4c992e5d80dd tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:09.754930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:09.755561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:09.755838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:09.756115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:09.757851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:09.758716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:09.759936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a02193633a44dda1885d363fcd5361'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.760414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-157cea0b-4adf-4fea-80e9-09ef9558e091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a02193633a44dda1885d363fcd5361'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:09.768481 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2456/4898] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:09 2022] DELETE /identity/v3/users/c3a02193633a44dda1885d363fcd5361 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:09.769922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:09.773637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:09.773985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:09.774271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:09.806303 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2443/4899] 10.222.0.33 () {58 vars in 1160 bytes} [Sat Sep 10 21:06:09 2022] GET /identity/v3/projects?is_domain=True => generated 1021 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:09.811603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:09.818911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:09.844378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:09.846596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:09.855079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7218cee-20d2-4533-9959-7666500a6ce7 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:09.885744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:10.011694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:10.012337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:10.012742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:10.012970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:10.016044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:10.020914 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2457/4900] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:10.021132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:10.058327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:10.060101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:10.087360 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2444/4901] 10.222.0.33 () {58 vars in 1128 bytes} [Sat Sep 10 21:06:09 2022] GET /identity/v3/domains => generated 830 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:10.088614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:10.124185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff3477-d309-45bb-851f-d1ae7d72b5a0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:10.140456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:10.141377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:10.141863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:10.142367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/c4cc3b067bf14cd4895ea6c325666bb7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:10.149454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:update_project(project_id=c4cc3b067bf14cd4895ea6c325666bb7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:10.154596 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:06:10.154596 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:06:10.154596 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:06:10.156301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:10.164569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28505c0-a371-47bc-8845-21020f55bcec None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:10.169634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:10.170614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:10.171127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:10.171856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:10.210798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:10.212492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:10.224480 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2458/4902] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:10.226881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.237336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.344591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.345529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.353960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2445/4903] 10.222.0.33 () {60 vars in 1252 bytes} [Sat Sep 10 21:06:10 2022] PATCH /identity/v3/projects/c4cc3b067bf14cd4895ea6c325666bb7 => generated 363 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:10.355287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:10.380469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c0952-8d3e-4f85-94d7-4e973deb9c40 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:10.529942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:10.531155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:10.531741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:10.532252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/c4cc3b067bf14cd4895ea6c325666bb7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:10.547207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c4cc3b067bf14cd4895ea6c325666bb7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:10.549080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:10.580006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.590537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.659246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:10.677552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7', 'request_id': 'req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.691777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7', 'request_id': 'req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.706261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7', 'request_id': 'req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.734117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project c4cc3b067bf14cd4895ea6c325666bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:10.734983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.735383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.746867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7', 'request_id': 'req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.747671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7', 'request_id': 'req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.751582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c4cc3b067bf14cd4895ea6c325666bb7', 'request_id': 'req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:10.763630 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2459/4904] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:10 2022] DELETE /identity/v3/projects/c4cc3b067bf14cd4895ea6c325666bb7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:10.765221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:10.767109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:10.822054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f93387-4a5b-4050-ac6f-0583b3f8f69f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:10.958070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:10.971201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:10.972419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:11.087239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:11.091746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.092061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.092345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.096057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:11.105623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:11.211752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad45b6b-fd10-41e1-bf24-04920d48b1c9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:11.214352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2446/4905] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:10 2022] POST /identity/v3/projects => generated 359 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:11.217970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:11.223708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:11.314667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:11.315707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.315992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.316808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] PATH_INFO: `/v3/projects/0980532ffcf34fcba842779e24616001` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.326992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0980532ffcf34fcba842779e24616001)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:11.329118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:11.384969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb88332-1cd3-4d37-80e2-d1723d43cd2a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:11.405722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0980532ffcf34fcba842779e24616001', 'request_id': 'req-e9dc1182-582e-42b2-ae59-351248198f6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.408201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:11.416735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0980532ffcf34fcba842779e24616001', 'request_id': 'req-e9dc1182-582e-42b2-ae59-351248198f6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.428237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0980532ffcf34fcba842779e24616001', 'request_id': 'req-e9dc1182-582e-42b2-ae59-351248198f6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.453707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] The token cache is being invalidate because project 0980532ffcf34fcba842779e24616001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:11.454125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.454848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9dc1182-582e-42b2-ae59-351248198f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.462018 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2460/4906] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:10 2022] DELETE /identity/v3/projects/0980532ffcf34fcba842779e24616001 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:11.464335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01cd0e-794f-4073-880e-bbe8c2e5b8bc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.464702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01cd0e-794f-4073-880e-bbe8c2e5b8bc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.465045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01cd0e-794f-4073-880e-bbe8c2e5b8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.496627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6b01cd0e-794f-4073-880e-bbe8c2e5b8bc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:11.498494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b01cd0e-794f-4073-880e-bbe8c2e5b8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:11.540133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:11.541058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.541497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.541953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/cdbc9841852f4ca9a94b7ecc0a7a6780` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.543658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=cdbc9841852f4ca9a94b7ecc0a7a6780)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:11.550452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:11.553218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2447/4907] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:11 2022] GET /identity/v3/projects/cdbc9841852f4ca9a94b7ecc0a7a6780 => generated 359 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:11.558837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:11.562683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:11.613712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2d230-dcb0-43fe-859a-dc79ffbc763f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:11.633990 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2461/4908] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:11.641786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0946b3a-ee20-47fd-aa26-c4aafc84e314 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.642376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0946b3a-ee20-47fd-aa26-c4aafc84e314 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.642893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0946b3a-ee20-47fd-aa26-c4aafc84e314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.655883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:11.655883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.655883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.656353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/cdbc9841852f4ca9a94b7ecc0a7a6780` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.657987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cdbc9841852f4ca9a94b7ecc0a7a6780)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:11.660654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:11.689607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a0946b3a-ee20-47fd-aa26-c4aafc84e314 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:11.714574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a0946b3a-ee20-47fd-aa26-c4aafc84e314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:11.731688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2462/4909] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:11.735550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0946b3a-ee20-47fd-aa26-c4aafc84e314 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:11.754461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0946b3a-ee20-47fd-aa26-c4aafc84e314 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:11.766577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:11.767278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:11.767628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:11.767971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:11.768808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:11.769179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:11.771065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:11.794174 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2463/4910] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:11.798188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:11.818966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61468731-db18-47b6-863d-5e61b948e994 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:11.824228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbc9841852f4ca9a94b7ecc0a7a6780', 'request_id': 'req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.840181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbc9841852f4ca9a94b7ecc0a7a6780', 'request_id': 'req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.851385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:11.865445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbc9841852f4ca9a94b7ecc0a7a6780', 'request_id': 'req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.895270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project cdbc9841852f4ca9a94b7ecc0a7a6780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:11.895568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.896878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:11.904846 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2448/4911] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:11 2022] DELETE /identity/v3/projects/cdbc9841852f4ca9a94b7ecc0a7a6780 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:11.912557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:11.915327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:12.002998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.003845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.004231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.006553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.007724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.009996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.082681 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2464/4912] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:11 2022] POST /identity/v3/projects => generated 403 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:12.084371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:12.087015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:12.093497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a0f9f-8bbc-4317-bfb4-bf3b1316f059 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.206817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:12.236122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.237621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.238405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.238846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.239723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.240447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f14f-32d3-4241-9c8f-5a27936ba7a1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.243958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.254639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d75b30-224b-400a-afdb-3f07ec7ec90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.255286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d75b30-224b-400a-afdb-3f07ec7ec90e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.255576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d75b30-224b-400a-afdb-3f07ec7ec90e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.256270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d75b30-224b-400a-afdb-3f07ec7ec90e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.256488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d75b30-224b-400a-afdb-3f07ec7ec90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.259241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d75b30-224b-400a-afdb-3f07ec7ec90e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.299275 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2465/4913] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:12 2022] POST /identity/v3/users => generated 302 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:12.300953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86359-5f6a-428a-9aad-4ef802796bca None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.301474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86359-5f6a-428a-9aad-4ef802796bca None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.301583 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2449/4914] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:11 2022] POST /identity/v3/projects => generated 360 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:12.302059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86359-5f6a-428a-9aad-4ef802796bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.307081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:12.385800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bda86359-5f6a-428a-9aad-4ef802796bca None None] MFA Rules not processed for user `13e5077421c44eb7abcda40582c7ea9a`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:12.389765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bda86359-5f6a-428a-9aad-4ef802796bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:12.465378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052832e0-2d81-4208-9d9e-f77a113e46c5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.477216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.477873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.478305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.478416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.479256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.481073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.512762 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2450/4915] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:12.518851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:12.565717 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2466/4916] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:12.573977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86359-5f6a-428a-9aad-4ef802796bca None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:12.596917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86359-5f6a-428a-9aad-4ef802796bca None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.598339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:12.605501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.606377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.606722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.607445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.607767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.611818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.634708 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2467/4917] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:12.636071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:12.639028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:12.730737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848faf8-5e15-44e2-830d-0877e1137730 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.739738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.740534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.740982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.742001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/f7dae288cf144d9eb33565e616a02128` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.743626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=f7dae288cf144d9eb33565e616a02128)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.746020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.751848 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2451/4918] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:12 2022] GET /identity/v3/projects/f7dae288cf144d9eb33565e616a02128 => generated 360 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:12.753265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:12.766320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f1444-1ea1-44db-9706-1834cc749453 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.777439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.778866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.779358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.779848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] PATH_INFO: `/v3/projects/32d5b3a1520647e9820d945b9175d252/users/016f090062c74ab3bf2497f4a7b12636/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.799375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d21c39-ccbd-484f-a3c0-7e05fcdb71de admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.799934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d5b3a1520647e9820d945b9175d252, user_id=016f090062c74ab3bf2497f4a7b12636, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.802909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.818231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ec04e-8f48-4cef-ae55-e4a960bfc740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.821251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec04e-8f48-4cef-ae55-e4a960bfc740 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.821606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec04e-8f48-4cef-ae55-e4a960bfc740 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.822009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec04e-8f48-4cef-ae55-e4a960bfc740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.823002 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2452/4919] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:12 2022] PUT /identity/v3/projects/32d5b3a1520647e9820d945b9175d252/users/016f090062c74ab3bf2497f4a7b12636/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:12.825328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:12.854526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ec04e-8f48-4cef-ae55-e4a960bfc740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.857252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ec04e-8f48-4cef-ae55-e4a960bfc740 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:12.885370 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2468/4920] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:12.889959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.891356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.892975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.955761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad25ce-2e41-42b6-97db-448cd3bc0fb6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:12.962382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:12.967090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:12.986002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:12.986709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:12.987002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:12.987282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/f7dae288cf144d9eb33565e616a02128` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:12.994599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f7dae288cf144d9eb33565e616a02128)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:12.999076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:13.067866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:13.103674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7dae288cf144d9eb33565e616a02128', 'request_id': 'req-10187c4c-16b6-46e3-8f29-828137977c9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.116134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7dae288cf144d9eb33565e616a02128', 'request_id': 'req-10187c4c-16b6-46e3-8f29-828137977c9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.122839 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2469/4921] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:13.124951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:13.127641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:13.137197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7dae288cf144d9eb33565e616a02128', 'request_id': 'req-10187c4c-16b6-46e3-8f29-828137977c9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.155682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219843a3-7ef0-41a1-885e-d862c7b050c8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:13.172456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project f7dae288cf144d9eb33565e616a02128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:13.173567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.174990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.179155 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2453/4922] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:12 2022] DELETE /identity/v3/projects/f7dae288cf144d9eb33565e616a02128 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:13.180127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:13.185410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:13.298529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:13.299247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:13.299536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:13.299814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] PATH_INFO: `/v3/users/bbbfb72cf13347ce901cad42460e7dce` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:13.301027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbfb72cf13347ce901cad42460e7dce)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:13.303039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:13.318296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:13.408100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbfb72cf13347ce901cad42460e7dce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.408874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10187c4c-16b6-46e3-8f29-828137977c9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:13.420217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:13.424028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:13.424322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:13.424605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:13.429628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:13.445266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbfb72cf13347ce901cad42460e7dce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.458691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:13.465735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:13.467197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbfb72cf13347ce901cad42460e7dce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.496807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbfb72cf13347ce901cad42460e7dce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.498660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbfb72cf13347ce901cad42460e7dce'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:13.509272 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2470/4924] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:13 2022] DELETE /identity/v3/users/bbbfb72cf13347ce901cad42460e7dce => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:13.512985 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2454/4924] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:13.513461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:13.515215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:13.680010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65864f8e-3470-4227-917c-86c744a9ee9a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:13.687022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:13.687984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:13.688424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:13.688855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:13.689864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:13.692323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:13.699345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:13.768628 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2471/4925] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:06:13 2022] POST /identity/v3/projects => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:13.773638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:13.775880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:13.914842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0aee6-519d-44d5-8203-b853cf492e05 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.016399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.017424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.017880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.018313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/d52b404327674c4c8a7ecc2082433d5f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.019555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=d52b404327674c4c8a7ecc2082433d5f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.023174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.025436 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2455/4926] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:13 2022] GET /identity/v3/projects/d52b404327674c4c8a7ecc2082433d5f => generated 346 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:14.029647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.045271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac23d0-4393-4db6-b54c-06de243f45b5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.061229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.062043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.062336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.062672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/d52b404327674c4c8a7ecc2082433d5f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.068057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d52b404327674c4c8a7ecc2082433d5f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.074408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.248918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd52b404327674c4c8a7ecc2082433d5f', 'request_id': 'req-47371d4a-4dac-4379-938e-357aefe7a504'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.265262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd52b404327674c4c8a7ecc2082433d5f', 'request_id': 'req-47371d4a-4dac-4379-938e-357aefe7a504'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.289793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd52b404327674c4c8a7ecc2082433d5f', 'request_id': 'req-47371d4a-4dac-4379-938e-357aefe7a504'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.339675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project d52b404327674c4c8a7ecc2082433d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:14.339986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.344969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:14.351328 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2472/4927] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:14 2022] DELETE /identity/v3/projects/d52b404327674c4c8a7ecc2082433d5f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:14.355877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.360086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:14.385197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0839f-7ef1-49ec-bb91-baca8ac34776 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.581112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.581939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.582236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.582523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.583854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.586106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.631356 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2456/4928] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:06:14 2022] POST /identity/v3/domains => generated 274 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:14.635096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.650439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47371d4a-4dac-4379-938e-357aefe7a504 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:14.738387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.741967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:14.745191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:14.746019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:14.746434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:14.747334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:14.750298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:14.753555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:14.825435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2473/4929] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:14 2022] POST /identity/v3/projects => generated 404 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:14.829912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:14.833464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:14.951006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:15.009914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af74eb-349d-4445-b944-66b5b5b92d9f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.018024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16906d51-d647-41a9-a3b9-724a9c6ba917 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.026719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.027576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.027962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.028472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/8f326e48ea32444097faf4cf33dff1c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.031409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=8f326e48ea32444097faf4cf33dff1c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.034625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.042751 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2474/4930] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:14 2022] GET /identity/v3/projects/8f326e48ea32444097faf4cf33dff1c8 => generated 404 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:15.045323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.071659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b11d8-df0e-46f7-b63d-b8ff785a8af4 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.084218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.086699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.088404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.090526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/8f326e48ea32444097faf4cf33dff1c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.092258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8f326e48ea32444097faf4cf33dff1c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.101230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.135773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.136745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.138598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.139135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] PATH_INFO: `/v3/projects/e9df0b572a1844bda38471874d2afe92` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.140566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9df0b572a1844bda38471874d2afe92)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.144124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.209779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f326e48ea32444097faf4cf33dff1c8', 'request_id': 'req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.226139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f326e48ea32444097faf4cf33dff1c8', 'request_id': 'req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.243766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9df0b572a1844bda38471874d2afe92', 'request_id': 'req-a95d49c8-b830-49ae-9567-3a87c098e42d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.250801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f326e48ea32444097faf4cf33dff1c8', 'request_id': 'req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.279792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9df0b572a1844bda38471874d2afe92', 'request_id': 'req-a95d49c8-b830-49ae-9567-3a87c098e42d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.301671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 8f326e48ea32444097faf4cf33dff1c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:15.302838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.304052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.304236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9df0b572a1844bda38471874d2afe92', 'request_id': 'req-a95d49c8-b830-49ae-9567-3a87c098e42d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.311158 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2475/4931] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:15 2022] DELETE /identity/v3/projects/8f326e48ea32444097faf4cf33dff1c8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:15.312261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.314427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.327914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] The token cache is being invalidate because project e9df0b572a1844bda38471874d2afe92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:15.328519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.329564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a95d49c8-b830-49ae-9567-3a87c098e42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.333303 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2457/4932] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:14 2022] DELETE /identity/v3/projects/e9df0b572a1844bda38471874d2afe92 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:15.338327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0865a-adbc-41d7-a4bb-ab5ce1272919 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.338833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0865a-adbc-41d7-a4bb-ab5ce1272919 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.339208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0865a-adbc-41d7-a4bb-ab5ce1272919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.375543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7aa0865a-adbc-41d7-a4bb-ab5ce1272919 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:15.377172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7aa0865a-adbc-41d7-a4bb-ab5ce1272919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.461241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f9b0-a664-43da-9a4c-5cc7c8d560d2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.479936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:15.510098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2458/4933] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:15.517771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c93d1-0dbc-4e17-b922-9031c59d9095 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.518093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c93d1-0dbc-4e17-b922-9031c59d9095 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.518379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c93d1-0dbc-4e17-b922-9031c59d9095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.568385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.569432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.569908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.570210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-991c93d1-0dbc-4e17-b922-9031c59d9095 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:15.570505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains/2761b9bd865d4fa5862c31234a41579c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.571469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-991c93d1-0dbc-4e17-b922-9031c59d9095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.572265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=2761b9bd865d4fa5862c31234a41579c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.574721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.593659 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2459/4934] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:15.610276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c93d1-0dbc-4e17-b922-9031c59d9095 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.624841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c93d1-0dbc-4e17-b922-9031c59d9095 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.633302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '2761b9bd865d4fa5862c31234a41579c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.634188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '2761b9bd865d4fa5862c31234a41579c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.640188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:15.645490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.646255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.646448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.646718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.656208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:15.656551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.658216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.695615 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2460/4935] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:15.699838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '2761b9bd865d4fa5862c31234a41579c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.700191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.702611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '2761b9bd865d4fa5862c31234a41579c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:15.702765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.708668 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2476/4936] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:06:15 2022] PATCH /identity/v3/domains/2761b9bd865d4fa5862c31234a41579c => generated 275 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:15.709994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:15.713252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:15.743341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298e3c-1012-4b16-99df-e59671ea2daf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.922086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.922859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.923144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.923416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.924054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.926013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:15.936547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418f14e-5b8b-4e50-bd11-b51104477342 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:15.952458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:15.953360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:15.953842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:15.954269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains/2761b9bd865d4fa5862c31234a41579c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:15.991925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=2761b9bd865d4fa5862c31234a41579c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:15.995639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.006129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2461/4937] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:15 2022] POST /identity/v3/projects => generated 365 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:16.011825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.016619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.084077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '2761b9bd865d4fa5862c31234a41579c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.095738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '2761b9bd865d4fa5862c31234a41579c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.219196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040828d-155d-436e-9a22-85e7008c632c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.275446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2761b9bd865d4fa5862c31234a41579c', 'request_id': 'req-30b700be-6ee3-4d36-a3dc-9deb46eaa756'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.302337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2761b9bd865d4fa5862c31234a41579c', 'request_id': 'req-30b700be-6ee3-4d36-a3dc-9deb46eaa756'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.325718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2761b9bd865d4fa5862c31234a41579c', 'request_id': 'req-30b700be-6ee3-4d36-a3dc-9deb46eaa756'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.406249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 2761b9bd865d4fa5862c31234a41579c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:16.406739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.408220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.431257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '2761b9bd865d4fa5862c31234a41579c', 'request_id': 'req-30b700be-6ee3-4d36-a3dc-9deb46eaa756'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.432999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '2761b9bd865d4fa5862c31234a41579c', 'request_id': 'req-30b700be-6ee3-4d36-a3dc-9deb46eaa756'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.441718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '2761b9bd865d4fa5862c31234a41579c', 'request_id': 'req-30b700be-6ee3-4d36-a3dc-9deb46eaa756'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.454569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.455450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.455879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.456296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.457242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.460904 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2477/4938] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:15 2022] DELETE /identity/v3/domains/2761b9bd865d4fa5862c31234a41579c => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:16.462590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.465043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.467970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.488575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:16.502977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b700be-6ee3-4d36-a3dc-9deb46eaa756 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.519754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.520415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.520698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.520976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] PATH_INFO: `/v3/users/96372af27d29482491d613b5a783a0d1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.522222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96372af27d29482491d613b5a783a0d1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.525112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.543688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2462/4939] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:16 2022] POST /identity/v3/users => generated 283 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:16.544952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.551903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.568051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:UPDATE=1,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:16.670078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96372af27d29482491d613b5a783a0d1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.690801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96372af27d29482491d613b5a783a0d1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.701481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96372af27d29482491d613b5a783a0d1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.719579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96372af27d29482491d613b5a783a0d1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.734541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96372af27d29482491d613b5a783a0d1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:16.745473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ae40-2140-43fb-9f77-c4283befd69f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:16.748340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2478/4940] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:16 2022] DELETE /identity/v3/users/96372af27d29482491d613b5a783a0d1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:16.749558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.751228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.873973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:16.875782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:16.876581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:16.877068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:16.879032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:16.881714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:16.944915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2463/4941] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:06:16 2022] POST /identity/v3/domains => generated 277 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:16.946775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:16.948901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:16.954844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f39a73-4024-4993-bf26-466bf57b1693 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.064004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97fdd-d828-49ba-a1e1-bfd078d45d1a tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.071199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.071830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.072300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.072506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.073227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.075519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.084054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.084784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.085904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.086132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.086986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.089298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.103720 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2479/4942] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:17.108349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.126556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f1ada-87e7-4956-87bb-ff22dad676be admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.133739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.134573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.135566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.136009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.137561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.139615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.153543 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2464/4943] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:16 2022] POST /identity/v3/projects => generated 409 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:17.154730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.168977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:17.171887 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2480/4944] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:17.176363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.210707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ae82b-a7bb-4386-86d6-4f676a9d990d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.249314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:17.302564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf1433-6b2e-4fa2-9d68-ad66387d0cf9 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.315301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.315958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.316247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.318632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.319367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.325009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.346444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd695c-0f91-40ea-837f-0972f73206dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.348463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd695c-0f91-40ea-837f-0972f73206dc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.348881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd695c-0f91-40ea-837f-0972f73206dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.349274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd695c-0f91-40ea-837f-0972f73206dc admin admin] PATH_INFO: `/v3/projects/27acb763ffd24512afe7ca5e1d06f031/users/6bfa3af0a7284bc6854f2c9c58392935/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.391715 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2465/4945] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:17 2022] POST /identity/v3/projects => generated 405 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:17.394770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd695c-0f91-40ea-837f-0972f73206dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=27acb763ffd24512afe7ca5e1d06f031, user_id=6bfa3af0a7284bc6854f2c9c58392935, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.396011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.403340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd695c-0f91-40ea-837f-0972f73206dc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.430137 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2481/4946] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:17 2022] PUT /identity/v3/projects/27acb763ffd24512afe7ca5e1d06f031/users/6bfa3af0a7284bc6854f2c9c58392935/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:17.436692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.436992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.437261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.488497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:17.491048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:17.554296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c8744-6378-477f-96d4-7b2a9ab403da tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.642755 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2482/4947] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:17.643584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.644916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:17.673508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913d276-b4ef-4447-a82a-f070d5edfbc9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:17.679532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.680405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.680838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.681439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] PATH_INFO: `/v3/projects/1da09fdc751e406f9c3bad4bf0e1b067` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.682378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:17.683185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:17.683541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:17.683812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da09fdc751e406f9c3bad4bf0e1b067)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.683922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/720f349c7ed7410696495786275da3fb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:17.685525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=720f349c7ed7410696495786275da3fb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:17.686363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.687451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:17.761479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da09fdc751e406f9c3bad4bf0e1b067', 'request_id': 'req-ffeced8f-b3ed-4d3c-8616-7c17ef005858'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.761812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:17.783110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720f349c7ed7410696495786275da3fb', 'request_id': 'req-db4588a8-55da-4e86-b497-0f99e1ced7e0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.792419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720f349c7ed7410696495786275da3fb', 'request_id': 'req-db4588a8-55da-4e86-b497-0f99e1ced7e0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.809812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da09fdc751e406f9c3bad4bf0e1b067', 'request_id': 'req-ffeced8f-b3ed-4d3c-8616-7c17ef005858'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.820468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da09fdc751e406f9c3bad4bf0e1b067', 'request_id': 'req-ffeced8f-b3ed-4d3c-8616-7c17ef005858'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.826789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720f349c7ed7410696495786275da3fb', 'request_id': 'req-db4588a8-55da-4e86-b497-0f99e1ced7e0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.840326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] The token cache is being invalidate because project 1da09fdc751e406f9c3bad4bf0e1b067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:17.840924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.842141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.844593 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2483/4948] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:17 2022] DELETE /identity/v3/projects/1da09fdc751e406f9c3bad4bf0e1b067 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:17.845698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.849228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:17.854380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 720f349c7ed7410696495786275da3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:17.854681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.855934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:17.860499 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2466/4949] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:17 2022] DELETE /identity/v3/projects/720f349c7ed7410696495786275da3fb => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:17.861343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:17.867023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:17.995749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4588a8-55da-4e86-b497-0f99e1ced7e0 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.000758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:18.005141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.005797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.006121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.006612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] PATH_INFO: `/v3/users/52973c98d6374eec88f4ebd5077c74bd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.010296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=52973c98d6374eec88f4ebd5077c74bd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.012366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.052084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeced8f-b3ed-4d3c-8616-7c17ef005858 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.103539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52973c98d6374eec88f4ebd5077c74bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.117851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52973c98d6374eec88f4ebd5077c74bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.118639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52973c98d6374eec88f4ebd5077c74bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.136408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52973c98d6374eec88f4ebd5077c74bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.137126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52973c98d6374eec88f4ebd5077c74bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.137953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.139292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.139659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.139936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.142203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.171376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.174187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.179831 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2467/4950] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:17 2022] DELETE /identity/v3/users/52973c98d6374eec88f4ebd5077c74bd => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:18.179831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.185462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.199738 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2484/4951] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:18.298323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:18.347447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a1a7f-d758-4292-a883-9dd432316e8f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.367522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.368208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.368480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.368757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/25d41e94eebf4cbe8e339eccfcf0aa54` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.371681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=25d41e94eebf4cbe8e339eccfcf0aa54)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.373224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.483828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d41e94eebf4cbe8e339eccfcf0aa54', 'request_id': 'req-f79c356a-3820-4f3c-94a4-573fa869e53f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.494777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d41e94eebf4cbe8e339eccfcf0aa54', 'request_id': 'req-f79c356a-3820-4f3c-94a4-573fa869e53f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.511005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d41e94eebf4cbe8e339eccfcf0aa54', 'request_id': 'req-f79c356a-3820-4f3c-94a4-573fa869e53f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.540592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 25d41e94eebf4cbe8e339eccfcf0aa54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:18.541315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.543320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f79c356a-3820-4f3c-94a4-573fa869e53f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.546386 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2468/4952] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:18 2022] DELETE /identity/v3/projects/25d41e94eebf4cbe8e339eccfcf0aa54 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:18.561926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.576133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.576828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.577268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.645371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] MFA Rules not processed for user `016f090062c74ab3bf2497f4a7b12636`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:18.650722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.682323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37eabe-1aca-4b6d-8878-2ea0ae3ec65b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:18.767714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:18.768642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:18.768769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:18.769040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains/465c3409a3024ce19f99e1c58f827bbf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:18.770568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=465c3409a3024ce19f99e1c58f827bbf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:18.772971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:18.808557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.809515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.818302 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2469/4953] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:18.819197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.822969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.882008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.882676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:18.889546 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2485/4954] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:06:18 2022] PATCH /identity/v3/domains/465c3409a3024ce19f99e1c58f827bbf => generated 278 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:18.889546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:18.889973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:18.899011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:19.029670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dee4c-3777-486f-9a1b-0c468e84ea75 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.035306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.036820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.037374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.037734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.039256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.041670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.060773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddf21a-2a03-4228-a13b-0ff3cb2f250c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.066481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.067141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.067427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.067707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] PATH_INFO: `/v3/projects/57907b81d51d40488a5c5a6129a702ff` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.069770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57907b81d51d40488a5c5a6129a702ff)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.071928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.080436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:19.090776 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2486/4955] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:18 2022] POST /identity/v3/projects => generated 401 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:19.092587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.094921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.189657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57907b81d51d40488a5c5a6129a702ff', 'request_id': 'req-1734b838-c48d-472a-a527-2a75f10561c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.201607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57907b81d51d40488a5c5a6129a702ff', 'request_id': 'req-1734b838-c48d-472a-a527-2a75f10561c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.224606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57907b81d51d40488a5c5a6129a702ff', 'request_id': 'req-1734b838-c48d-472a-a527-2a75f10561c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.233040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd9e8a-a3ce-43f8-a4f2-fc22b4f15564 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.254547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] The token cache is being invalidate because project 57907b81d51d40488a5c5a6129a702ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:19.254547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.267852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.270022 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2470/4956] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:18 2022] DELETE /identity/v3/projects/57907b81d51d40488a5c5a6129a702ff => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:19.271264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.282126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.339285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.340283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.340768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.341290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains/465c3409a3024ce19f99e1c58f827bbf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.381887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=465c3409a3024ce19f99e1c58f827bbf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.382823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734b838-c48d-472a-a527-2a75f10561c4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.386474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.395685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e381e74-048c-43cb-a713-af5d2cdad7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.396447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e381e74-048c-43cb-a713-af5d2cdad7ab admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.396809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e381e74-048c-43cb-a713-af5d2cdad7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.397205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e381e74-048c-43cb-a713-af5d2cdad7ab admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.399595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e381e74-048c-43cb-a713-af5d2cdad7ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.403058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e381e74-048c-43cb-a713-af5d2cdad7ab admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.438163 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2471/4957] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:19 2022] POST /identity/v3/users => generated 307 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:19.438395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.438491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f8ce9-1dd4-45ac-9a03-8a470b593125 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.438491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f8ce9-1dd4-45ac-9a03-8a470b593125 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.438491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f8ce9-1dd4-45ac-9a03-8a470b593125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.450914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.454837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:19.489285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-277f8ce9-1dd4-45ac-9a03-8a470b593125 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:19.492521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-277f8ce9-1dd4-45ac-9a03-8a470b593125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.505049 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2472/4958] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:19.507482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f8ce9-1dd4-45ac-9a03-8a470b593125 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.541487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f8ce9-1dd4-45ac-9a03-8a470b593125 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.595104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf', 'request_id': 'req-5b194833-7445-49f2-b663-51748c077ae5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.615901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf', 'request_id': 'req-5b194833-7445-49f2-b663-51748c077ae5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.626598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf', 'request_id': 'req-5b194833-7445-49f2-b663-51748c077ae5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.652898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 465c3409a3024ce19f99e1c58f827bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:19.653200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.655040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.677356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf', 'request_id': 'req-5b194833-7445-49f2-b663-51748c077ae5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.678238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf', 'request_id': 'req-5b194833-7445-49f2-b663-51748c077ae5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.686655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5b194833-7445-49f2-b663-51748c077ae5 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '465c3409a3024ce19f99e1c58f827bbf', 'request_id': 'req-5b194833-7445-49f2-b663-51748c077ae5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:19.701769 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2487/4959] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:19 2022] DELETE /identity/v3/domains/465c3409a3024ce19f99e1c58f827bbf => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:19.703273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.703890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.705854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.708462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.709090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.709375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.709654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.711110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.712721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.740061 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2473/4960] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:19.742177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.746910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.773237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:19.774985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.783613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:19.790368 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2488/4961] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:19.791346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.792783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:19.812992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f4f0b-fa08-4cf4-83bf-2fb7adee1891 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.822343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.823080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.823928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:19.824271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] PATH_INFO: `/v3/projects/e1e4eccf74b240b2b2dd86d909762636/users/cb12c49d2e284eb7a5fde22647bd045c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:19.847553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e4eccf74b240b2b2dd86d909762636, user_id=cb12c49d2e284eb7a5fde22647bd045c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:19.852223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:19.876096 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2489/4962] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:19 2022] PUT /identity/v3/projects/e1e4eccf74b240b2b2dd86d909762636/users/cb12c49d2e284eb7a5fde22647bd045c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:19.882293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:19.895204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22009c-d768-4498-b591-834c0978e6db admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:19.995937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa0a1a-8a0f-40df-81c8-898ae0f09bf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:19.999434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0a1a-8a0f-40df-81c8-898ae0f09bf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:19.999726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0a1a-8a0f-40df-81c8-898ae0f09bf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.000008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0a1a-8a0f-40df-81c8-898ae0f09bf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.000666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa0a1a-8a0f-40df-81c8-898ae0f09bf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.007814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa0a1a-8a0f-40df-81c8-898ae0f09bf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.018184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1b533-e417-4b97-bcfb-010e95c8bdef admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.033971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.035001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.035359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.035647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.036314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:20.039103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.043388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.098050 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2490/4963] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:20.100039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.121743 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2474/4964] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:19 2022] POST /identity/v3/projects => generated 378 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:20.125275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0ba3-f198-4678-8a39-27046c9fdb92 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.125575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0ba3-f198-4678-8a39-27046c9fdb92 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.125880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0ba3-f198-4678-8a39-27046c9fdb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.144211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e1f5-23a1-4d19-b803-8f5106a6f6b6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.181714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-801d0ba3-f198-4678-8a39-27046c9fdb92 None None] MFA Rules not processed for user `13e5077421c44eb7abcda40582c7ea9a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:20.190954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-801d0ba3-f198-4678-8a39-27046c9fdb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:20.208048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:20.267652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.267652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.268437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.268437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.269388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.274324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.291350 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2491/4965] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:20.292426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.309031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c560a2d-27f1-4744-b9b5-3baf54f02958 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.321060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.322206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.322840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.323326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.324217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.326954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.363779 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2475/4966] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:20.365432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0ba3-f198-4678-8a39-27046c9fdb92 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.391569 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2492/4967] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:20 2022] POST /identity/v3/projects => generated 397 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:20.393177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.510693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:20.529311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0ba3-f198-4678-8a39-27046c9fdb92 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.564057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa665aa-f0ab-447c-ac76-8a31b27043d0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.573003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.573003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.573352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.573563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] PATH_INFO: `/v3/projects/e1e4eccf74b240b2b2dd86d909762636/users/cb12c49d2e284eb7a5fde22647bd045c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.580299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e4eccf74b240b2b2dd86d909762636, user_id=cb12c49d2e284eb7a5fde22647bd045c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.590285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.613804 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2493/4968] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:20 2022] PUT /identity/v3/projects/e1e4eccf74b240b2b2dd86d909762636/users/cb12c49d2e284eb7a5fde22647bd045c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:20.617756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.625842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.627838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.628346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.628829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/4d06f33fda6c42299116b9c04b76ae47` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.632985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=4d06f33fda6c42299116b9c04b76ae47)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.636300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.639292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2476/4969] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:20 2022] GET /identity/v3/projects/4d06f33fda6c42299116b9c04b76ae47 => generated 378 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:20.643357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.669070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf84cf0-090b-46ff-80fe-4d97a902233c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.764560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f97bd-f773-4b1e-90dc-1668447474c5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.781174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.781802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.782145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.782457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.783084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.791123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.796178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.797448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.798125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.799225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.800848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.805218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.840070 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2494/4970] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:20 2022] POST /identity/v3/users => generated 299 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:20.843052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.851756 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2477/4971] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:06:20 2022] GET /identity/v3/projects => generated 6982 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:20.862929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.866983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b602f-a6a6-49f4-9617-7d07ae5022da admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.882922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9550d5f-c5a0-4231-8be7-ccb17714f429 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.883998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.884647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.885053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.885406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.886189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.888223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.896440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.897820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.898235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.898613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/4d06f33fda6c42299116b9c04b76ae47` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.900494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4d06f33fda6c42299116b9c04b76ae47)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.905620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:20.916180 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2495/4972] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:20.920372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:20.949085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093f4a1-f38c-4e86-a3f4-2c99818362ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:20.969770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:20.970803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:20.971100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:20.971382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:20.972097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:20.976388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.002210 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2496/4973] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:21.008374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.054520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55386f-d8a6-47d7-a133-4809c71a5024 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.072480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d06f33fda6c42299116b9c04b76ae47', 'request_id': 'req-92bcbe66-7088-48e0-a542-2734da119dfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.083794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.091800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d06f33fda6c42299116b9c04b76ae47', 'request_id': 'req-92bcbe66-7088-48e0-a542-2734da119dfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.125022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d06f33fda6c42299116b9c04b76ae47', 'request_id': 'req-92bcbe66-7088-48e0-a542-2734da119dfb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.156765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 4d06f33fda6c42299116b9c04b76ae47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:21.157029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.159369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:21.170666 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2478/4974] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:20 2022] DELETE /identity/v3/projects/4d06f33fda6c42299116b9c04b76ae47 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:21.172140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.174333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.194989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.202922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f84ce-a8bb-4809-84b5-a99b4366366b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.203749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f84ce-a8bb-4809-84b5-a99b4366366b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.204040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f84ce-a8bb-4809-84b5-a99b4366366b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.204317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f84ce-a8bb-4809-84b5-a99b4366366b admin admin] PATH_INFO: `/v3/projects/ceebf89c39014750ae279e0741a0bb79/users/9546792be5a640e2a59fb2e8481c6c4b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.239339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f84ce-a8bb-4809-84b5-a99b4366366b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceebf89c39014750ae279e0741a0bb79, user_id=9546792be5a640e2a59fb2e8481c6c4b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.243302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f84ce-a8bb-4809-84b5-a99b4366366b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.275624 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2497/4975] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:21 2022] PUT /identity/v3/projects/ceebf89c39014750ae279e0741a0bb79/users/9546792be5a640e2a59fb2e8481c6c4b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:21.280625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.280912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.281187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.303945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcbe66-7088-48e0-a542-2734da119dfb tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.328246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:21.329654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.481228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.481731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.482049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.482390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.483198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.485340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.511584 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2498/4976] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:21.527976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.534912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.557859 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2479/4977] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:06:21 2022] POST /identity/v3/projects => generated 347 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:21.571148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.699024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dade2-aff2-411b-be30-089bbcb25410 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.707150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347de05-6966-4a5d-9cd0-242b7b78ee12 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.714184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.714840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.715199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.715389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/296270ebd0e34c3588fae37e15d0b766` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.716623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:update_project(project_id=296270ebd0e34c3588fae37e15d0b766)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.724397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.797736 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2480/4978] 10.222.0.33 () {60 vars in 1252 bytes} [Sat Sep 10 21:06:21 2022] PATCH /identity/v3/projects/296270ebd0e34c3588fae37e15d0b766 => generated 361 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:21.799324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.821468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066946a-04d3-4f20-ab80-668937570a47 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.824267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.824895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.825178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.825453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.834033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.835683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.836514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.836800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.837083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/296270ebd0e34c3588fae37e15d0b766` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.871522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=296270ebd0e34c3588fae37e15d0b766)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.873709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.878788 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2481/4979] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/projects/296270ebd0e34c3588fae37e15d0b766 => generated 348 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:21.878788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.883424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:21.915882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:21.926281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:21.930200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:21.942638 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2499/4980] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:21.943802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:21.970898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002f4d5-7cb3-44be-af59-ed66767fc40d service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:21.994422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:21.995084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:21.995373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:21.995658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/296270ebd0e34c3588fae37e15d0b766` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:21.999092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=296270ebd0e34c3588fae37e15d0b766)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.005208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.078236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6fc99-c6f1-4c5b-bed1-8638bf4dbaef tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.085619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.086796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.087181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.087559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.089643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.119726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.121658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.146498 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2482/4981] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:22.159285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296270ebd0e34c3588fae37e15d0b766', 'request_id': 'req-97037e7b-e241-444d-bf1a-c2cc58c4681f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.175372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296270ebd0e34c3588fae37e15d0b766', 'request_id': 'req-97037e7b-e241-444d-bf1a-c2cc58c4681f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.182740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296270ebd0e34c3588fae37e15d0b766', 'request_id': 'req-97037e7b-e241-444d-bf1a-c2cc58c4681f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.208127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 296270ebd0e34c3588fae37e15d0b766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:22.208435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.209106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:22.217100 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2500/4982] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:21 2022] DELETE /identity/v3/projects/296270ebd0e34c3588fae37e15d0b766 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:22.219890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.239158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff867711-03b9-40c6-bde0-d2dcb377a815 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.378321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.379148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.379436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.379765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.382271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.384374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.427676 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2483/4983] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:22 2022] POST /identity/v3/projects => generated 360 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:22.432053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.433631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:22.464152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97037e7b-e241-444d-bf1a-c2cc58c4681f tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.671220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.676060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.680731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.681188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/1ba86bb20c1e471a97c03e974fc4c4c7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.688874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:update_project(project_id=1ba86bb20c1e471a97c03e974fc4c4c7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.690160 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:06:22.690160 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:06:22.690160 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:06:22.696451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.784633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:22.818676 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2501/4984] 10.222.0.33 () {60 vars in 1252 bytes} [Sat Sep 10 21:06:22 2022] PATCH /identity/v3/projects/1ba86bb20c1e471a97c03e974fc4c4c7 => generated 372 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:22.824002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.870670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2934cec-53e6-458d-8cb2-bec5eea7b128 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.890883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.894154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.894493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.894774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/1ba86bb20c1e471a97c03e974fc4c4c7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.924752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=1ba86bb20c1e471a97c03e974fc4c4c7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.927081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:22.928994 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2484/4985] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:22 2022] GET /identity/v3/projects/1ba86bb20c1e471a97c03e974fc4c4c7 => generated 359 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:22.933331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:22.957285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94db37-ba7b-4deb-96ee-4c13be423cf2 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:22.971132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:22.972060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:22.972531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:22.972963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/1ba86bb20c1e471a97c03e974fc4c4c7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:22.975017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1ba86bb20c1e471a97c03e974fc4c4c7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:22.980818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.112060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba86bb20c1e471a97c03e974fc4c4c7', 'request_id': 'req-631101df-b8f0-49f8-8ab8-cf33aa07d506'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:23.122491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba86bb20c1e471a97c03e974fc4c4c7', 'request_id': 'req-631101df-b8f0-49f8-8ab8-cf33aa07d506'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:23.134405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba86bb20c1e471a97c03e974fc4c4c7', 'request_id': 'req-631101df-b8f0-49f8-8ab8-cf33aa07d506'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:23.177776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 1ba86bb20c1e471a97c03e974fc4c4c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:23.181385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:23.182884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:23.188764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2502/4986] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:22 2022] DELETE /identity/v3/projects/1ba86bb20c1e471a97c03e974fc4c4c7 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:23.193726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.196245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:23.217937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a3387-b51f-478f-97ef-64b9d8caef29 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.301745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.303436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.303436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.303436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.304698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.307073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.359863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:23.403892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2485/4987] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:23 2022] POST /identity/v3/projects => generated 355 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:23.404542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.436866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631101df-b8f0-49f8-8ab8-cf33aa07d506 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.576415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.577348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.577962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.578510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/3ba38885f4b64dd0a11efd50922710c3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.580036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:update_project(project_id=3ba38885f4b64dd0a11efd50922710c3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.582512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.696345 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2503/4988] 10.222.0.33 () {60 vars in 1252 bytes} [Sat Sep 10 21:06:23 2022] PATCH /identity/v3/projects/3ba38885f4b64dd0a11efd50922710c3 => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:23.701584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.721522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42056-ec0d-4cda-a28a-60df5f2eb800 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.729533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.730529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.730984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.731502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/3ba38885f4b64dd0a11efd50922710c3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.769188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:get_project(project_id=3ba38885f4b64dd0a11efd50922710c3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.774835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.801873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:23.811716 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2486/4989] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:06:23 2022] GET /identity/v3/projects/3ba38885f4b64dd0a11efd50922710c3 => generated 355 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:23.823571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7fc90-edea-4453-9863-5e24c0c27e05 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:23.840789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:23.842676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:23.842968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:23.843249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/projects/3ba38885f4b64dd0a11efd50922710c3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:23.847306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3ba38885f4b64dd0a11efd50922710c3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:23.852462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:23.974950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:UPDATE=2,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:23.985542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba38885f4b64dd0a11efd50922710c3', 'request_id': 'req-335f4422-8173-4b15-8071-a0e1c793d77c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:23.999793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba38885f4b64dd0a11efd50922710c3', 'request_id': 'req-335f4422-8173-4b15-8071-a0e1c793d77c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.013243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba38885f4b64dd0a11efd50922710c3', 'request_id': 'req-335f4422-8173-4b15-8071-a0e1c793d77c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.039247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] The token cache is being invalidate because project 3ba38885f4b64dd0a11efd50922710c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:24.039551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.041734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.046432 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2504/4990] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:23 2022] DELETE /identity/v3/projects/3ba38885f4b64dd0a11efd50922710c3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:24.051458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:24.054141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:24.109552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411da1-436e-4bd4-af4f-8e5a1834fefc tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:24.278679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:24.279336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:24.279627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:24.280110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains/465c3409a3024ce19f99e1c58f827bbf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:24.292010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Could not find domain: 465c3409a3024ce19f99e1c58f827bbf.: keystone.exception.DomainNotFound: Could not find domain: 465c3409a3024ce19f99e1c58f827bbf. Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 465c3409a3024ce19f99e1c58f827bbf. Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:06:24.299480 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:24.300588 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 465c3409a3024ce19f99e1c58f827bbf. Sep 10 21:06:24.301487 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:06:24.307198 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2487/4991] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:06:24 2022] PATCH /identity/v3/domains/465c3409a3024ce19f99e1c58f827bbf => generated 112 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:06:24.307347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:24.324600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f4422-8173-4b15-8071-a0e1c793d77c tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:24.336120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75411f-23b7-4d46-813d-86efc8635c14 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd9f7def6c584abd8c16a1515ee239ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:24.337982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75411f-23b7-4d46-813d-86efc8635c14 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:24.346581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75411f-23b7-4d46-813d-86efc8635c14 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:24.347119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75411f-23b7-4d46-813d-86efc8635c14 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] PATH_INFO: `/v3/domains/2761b9bd865d4fa5862c31234a41579c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:24.357177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75411f-23b7-4d46-813d-86efc8635c14 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-5a75411f-23b7-4d46-813d-86efc8635c14 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Could not find domain: 2761b9bd865d4fa5862c31234a41579c.: keystone.exception.DomainNotFound: Could not find domain: 2761b9bd865d4fa5862c31234a41579c. Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2761b9bd865d4fa5862c31234a41579c. Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:06:24.367061 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:24.369536 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 2761b9bd865d4fa5862c31234a41579c. Sep 10 21:06:24.370813 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:06:24.376279 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2505/4992] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:06:24 2022] PATCH /identity/v3/domains/2761b9bd865d4fa5862c31234a41579c => generated 112 bytes in 70 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:06:24.380642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:24.386794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:24.389024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0149ac9-515a-405a-bf70-aca023423461 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:24.389562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0149ac9-515a-405a-bf70-aca023423461 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:24.390232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0149ac9-515a-405a-bf70-aca023423461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:24.465317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d0149ac9-515a-405a-bf70-aca023423461 None None] MFA Rules not processed for user `cb12c49d2e284eb7a5fde22647bd045c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:24.467871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d0149ac9-515a-405a-bf70-aca023423461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:24.551414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a47cd-5f2e-4a5f-9580-8fd9e95c8349 tempest-ProjectsTestJSON-273257579 tempest-ProjectsTestJSON-273257579-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:24.562617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:24.563279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:24.565562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:24.565879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] PATH_INFO: `/v3/users/269dd863f5094d22829e0e46786af67c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:24.570542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269dd863f5094d22829e0e46786af67c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:24.576743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:24.637281 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2506/4993] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:24.674399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269dd863f5094d22829e0e46786af67c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.684745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269dd863f5094d22829e0e46786af67c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.686303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269dd863f5094d22829e0e46786af67c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.699214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269dd863f5094d22829e0e46786af67c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.700486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269dd863f5094d22829e0e46786af67c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:24.739003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:24.739460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:24.739888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:24.745413 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2488/4994] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:24 2022] DELETE /identity/v3/users/269dd863f5094d22829e0e46786af67c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:24.769790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:24.771248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:24.864347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2507/4995] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:24.878817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:24.880789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:24.888648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:25.011160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f7aa4-f4fa-462f-9628-85cb1ed21b25 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:25.021045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:25.021973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:25.022108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:25.022426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:25.047735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:25.049677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:25.078887 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2489/4996] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:26.526798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:26.528953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:26.556022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36cbd-ae97-4ef9-8c61-b32a9fc17549 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:26.563201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:26.564235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:26.566295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:26.566827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] PATH_INFO: `/v3/projects/ce3f758adba946139e588eb84eadd2ee` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:26.568338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce3f758adba946139e588eb84eadd2ee)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:26.571232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:26.629177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:26.639500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce3f758adba946139e588eb84eadd2ee', 'request_id': 'req-6568622c-30ef-4a46-a7ed-fd188823dcad'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.650801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce3f758adba946139e588eb84eadd2ee', 'request_id': 'req-6568622c-30ef-4a46-a7ed-fd188823dcad'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.675445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce3f758adba946139e588eb84eadd2ee', 'request_id': 'req-6568622c-30ef-4a46-a7ed-fd188823dcad'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.721972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] The token cache is being invalidate because project ce3f758adba946139e588eb84eadd2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:26.723058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.724417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.729025 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2508/4997] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:26 2022] DELETE /identity/v3/projects/ce3f758adba946139e588eb84eadd2ee => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:26.733411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:26.746728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d52d-ff9d-45bf-8b0a-7230735e8469 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:26.828453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:26.830329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:26.832892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:26.834097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] PATH_INFO: `/v3/users/dd9f7def6c584abd8c16a1515ee239ec` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:26.837708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9f7def6c584abd8c16a1515ee239ec)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:26.842544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:26.922168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9f7def6c584abd8c16a1515ee239ec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.936940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9f7def6c584abd8c16a1515ee239ec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.937898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9f7def6c584abd8c16a1515ee239ec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.948428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9f7def6c584abd8c16a1515ee239ec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.949111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9f7def6c584abd8c16a1515ee239ec'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:26.995327 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2490/4998] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:26 2022] DELETE /identity/v3/users/dd9f7def6c584abd8c16a1515ee239ec => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:28.710277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.711915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:28.733189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568622c-30ef-4a46-a7ed-fd188823dcad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.788502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:28.790911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:28.838636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:28.906807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1e0bc-356d-4fdd-b203-a6025333d7cc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:28.915245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.916556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.917033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.917519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] PATH_INFO: `/v3/projects/b5ab2028c1af4cf39e885ae7e15c4a0c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.919271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ab2028c1af4cf39e885ae7e15c4a0c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.920941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:28.921704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:28.922641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:28.923310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:28.923548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] PATH_INFO: `/v3/users/016f090062c74ab3bf2497f4a7b12636` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:28.926969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016f090062c74ab3bf2497f4a7b12636)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:28.932946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.041486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'request_id': 'req-1b731d67-fc22-48b3-9c8c-92b988afcb78'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.053272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'request_id': 'req-1b731d67-fc22-48b3-9c8c-92b988afcb78'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.059006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016f090062c74ab3bf2497f4a7b12636'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.067863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ab2028c1af4cf39e885ae7e15c4a0c', 'request_id': 'req-1b731d67-fc22-48b3-9c8c-92b988afcb78'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.073714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016f090062c74ab3bf2497f4a7b12636'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.082411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016f090062c74ab3bf2497f4a7b12636'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.088505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] The token cache is being invalidate because project b5ab2028c1af4cf39e885ae7e15c4a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:29.088807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.090059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1b731d67-fc22-48b3-9c8c-92b988afcb78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.092784 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2491/4999] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:28 2022] DELETE /identity/v3/projects/b5ab2028c1af4cf39e885ae7e15c4a0c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:29.096674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016f090062c74ab3bf2497f4a7b12636'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.098212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f47dd099-867a-43f2-a0f3-401ced397860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016f090062c74ab3bf2497f4a7b12636'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:29.099497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f0d37-b29a-476d-b17e-780dee12b4a3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.099952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f0d37-b29a-476d-b17e-780dee12b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.101246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f0d37-b29a-476d-b17e-780dee12b4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.113878 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2509/5000] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:28 2022] DELETE /identity/v3/users/016f090062c74ab3bf2497f4a7b12636 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:29.168929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b71f0d37-b29a-476d-b17e-780dee12b4a3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:29.170600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b71f0d37-b29a-476d-b17e-780dee12b4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:29.336319 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2492/5001] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:29.346286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525480-d786-4f26-af91-89ccefd17959 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.346711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525480-d786-4f26-af91-89ccefd17959 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.347095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525480-d786-4f26-af91-89ccefd17959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.404390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b6525480-d786-4f26-af91-89ccefd17959 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:29.405787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b6525480-d786-4f26-af91-89ccefd17959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:29.432115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2510/5002] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:29.433707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f0d37-b29a-476d-b17e-780dee12b4a3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.448271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f0d37-b29a-476d-b17e-780dee12b4a3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.479719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.480265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.480549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.480828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.481497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:29.481804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.486291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.531232 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2493/5003] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:29.532434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525480-d786-4f26-af91-89ccefd17959 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.547204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525480-d786-4f26-af91-89ccefd17959 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.567021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.567667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.567957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.568238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.568897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.572184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.643067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2511/5004] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:29 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:29.644259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.663809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aab5e-5e99-4612-aaf1-2e449db29756 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.710536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:29.822712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.823362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.823610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.824076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.825044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.827191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.872462 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2494/5005] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:29 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:29.876702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.894794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d2e72-07ea-440a-911f-410989ad3409 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.902391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.903649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.904333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.905131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.906381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.911320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:29.925665 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2512/5006] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:29.929948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:29.948760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a8cc8-d8c2-4523-9537-b7ed7709e3a0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:29.961633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9bf6e-9c12-4d8f-a80e-ae81811a88ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:29.962530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9bf6e-9c12-4d8f-a80e-ae81811a88ca admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:29.962989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9bf6e-9c12-4d8f-a80e-ae81811a88ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:29.963402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9bf6e-9c12-4d8f-a80e-ae81811a88ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:29.964200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9bf6e-9c12-4d8f-a80e-ae81811a88ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:29.965997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9bf6e-9c12-4d8f-a80e-ae81811a88ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:30.000919 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2495/5007] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:30.005057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:30.025513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f372-5fca-47d8-a05a-8cbe29cfc0c4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:30.032130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:30.039197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.039492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.039774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] PATH_INFO: `/v3/projects/b72b644fc5184f8ba6087ba73d4cdf4c/users/f5986165cb874ceb8e0810863a81c406/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.064723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:30.087764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72b644fc5184f8ba6087ba73d4cdf4c, user_id=f5986165cb874ceb8e0810863a81c406, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:30.090722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:30.126201 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2513/5008] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:30 2022] PUT /identity/v3/projects/b72b644fc5184f8ba6087ba73d4cdf4c/users/f5986165cb874ceb8e0810863a81c406/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:30.127500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cac84b-18e2-4aa7-b59b-72d9b1714fe0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.127809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cac84b-18e2-4aa7-b59b-72d9b1714fe0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.128089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cac84b-18e2-4aa7-b59b-72d9b1714fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.196443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-36cac84b-18e2-4aa7-b59b-72d9b1714fe0 None None] MFA Rules not processed for user `f5986165cb874ceb8e0810863a81c406`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:30.214851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36cac84b-18e2-4aa7-b59b-72d9b1714fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:30.349672 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2496/5009] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:30.355353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:30.370123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988ac6b-98ae-4371-9aaa-7cc3110481c9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:30.380851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ba0e9-1f2e-453f-9c64-7f36859492d8 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:30.382591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ba0e9-1f2e-453f-9c64-7f36859492d8 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.383247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ba0e9-1f2e-453f-9c64-7f36859492d8 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.384410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ba0e9-1f2e-453f-9c64-7f36859492d8 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.386723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ba0e9-1f2e-453f-9c64-7f36859492d8 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:create_application_credential(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:30.388902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ba0e9-1f2e-453f-9c64-7f36859492d8 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:30.502142 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2514/5010] 10.222.0.33 () {60 vars in 1314 bytes} [Sat Sep 10 21:06:30 2022] POST /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 789 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:30.507246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cac84b-18e2-4aa7-b59b-72d9b1714fe0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:30.527124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cac84b-18e2-4aa7-b59b-72d9b1714fe0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:30.540624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:30.541422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.541846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.542248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/318c9daae4574c53950960761decc7ec` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.592262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:get_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=318c9daae4574c53950960761decc7ec)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:30.594089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:30.601133 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2497/5011] 10.222.0.33 () {58 vars in 1392 bytes} [Sat Sep 10 21:06:30 2022] GET /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/318c9daae4574c53950960761decc7ec => generated 689 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:30.605397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d3750-cab1-4878-9ecc-d6d3bbbb504e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.606000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d3750-cab1-4878-9ecc-d6d3bbbb504e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.606400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d3750-cab1-4878-9ecc-d6d3bbbb504e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.646591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d91d3750-cab1-4878-9ecc-d6d3bbbb504e None None] MFA Rules not processed for user `f5986165cb874ceb8e0810863a81c406`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:30.651141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d91d3750-cab1-4878-9ecc-d6d3bbbb504e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:30.942272 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2515/5012] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3425 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 10 21:06:30.943238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:30.956278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f8546-8e15-4f90-9f0b-d4a757fa6a69 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:30.981239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:30.982248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:30.983326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:30.984684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/318c9daae4574c53950960761decc7ec` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:30.986710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:delete_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=318c9daae4574c53950960761decc7ec)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:30.988972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.031684 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2498/5013] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:06:30 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/318c9daae4574c53950960761decc7ec => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:31.036881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d3750-cab1-4878-9ecc-d6d3bbbb504e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:31.051526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d3750-cab1-4878-9ecc-d6d3bbbb504e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:31.065401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:31.066362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:31.067037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:31.067310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:31.067580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:31.068255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:create_application_credential(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:31.069713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.071291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:31.186474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:31.238795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e331cb2-5cea-4272-9f42-3da5d3c8fcc1 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:31.254551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:31.255219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:31.255499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:31.255772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] PATH_INFO: `/v3/projects/32d5b3a1520647e9820d945b9175d252` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:31.258214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d5b3a1520647e9820d945b9175d252)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:31.260133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.261811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:31.312030 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2516/5014] 10.222.0.33 () {60 vars in 1315 bytes} [Sat Sep 10 21:06:31 2022] POST /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 920 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:31.313313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:31.344220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3967b08-f96a-45ce-ae0c-7508291907da tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:31.356414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:31.357854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:31.358265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:31.360229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/78006fac19c6464ead191cf9a46c7a8f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:31.394301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:delete_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=78006fac19c6464ead191cf9a46c7a8f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:31.394525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d5b3a1520647e9820d945b9175d252', 'request_id': 'req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.397711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.409373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d5b3a1520647e9820d945b9175d252', 'request_id': 'req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.439862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d5b3a1520647e9820d945b9175d252', 'request_id': 'req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.452436 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2517/5015] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:06:31 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/78006fac19c6464ead191cf9a46c7a8f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:31.458104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:31.468887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] The token cache is being invalidate because project 32d5b3a1520647e9820d945b9175d252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:31.469247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.470115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.474190 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2499/5016] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:31 2022] DELETE /identity/v3/projects/32d5b3a1520647e9820d945b9175d252 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:31.479280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:31.482492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5b21a-aee3-4437-9fc7-2cd2b70bf1f0 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:31.483006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:31.614823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:31.615598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:31.618269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:31.618678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:31.619485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:create_application_credential(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:31.620807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.654926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e345c83-3627-48e5-b7c0-2e4f5a25f80a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:31.660815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:31.662587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:31.666513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:31.667391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] PATH_INFO: `/v3/users/cb12c49d2e284eb7a5fde22647bd045c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:31.669433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb12c49d2e284eb7a5fde22647bd045c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:31.677571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.769646 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2518/5017] 10.222.0.33 () {60 vars in 1315 bytes} [Sat Sep 10 21:06:31 2022] POST /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 812 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:31.774544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:31.779400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:31.806083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b760d-4605-4e6e-ad90-12de893116ba tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:31.809156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb12c49d2e284eb7a5fde22647bd045c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.816546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:31.817233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:31.818639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:31.820328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/c5311d96fef942ccbc394ea101bbe808` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:31.830682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb12c49d2e284eb7a5fde22647bd045c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.831807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb12c49d2e284eb7a5fde22647bd045c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.843022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb12c49d2e284eb7a5fde22647bd045c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.844111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb12c49d2e284eb7a5fde22647bd045c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:31.844922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:delete_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=c5311d96fef942ccbc394ea101bbe808)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:31.846690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:31.885024 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2500/5018] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:31 2022] DELETE /identity/v3/users/cb12c49d2e284eb7a5fde22647bd045c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:31.890836 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2519/5019] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:06:31 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/c5311d96fef942ccbc394ea101bbe808 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:31.899502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:31.899502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:31.921443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:32.036717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6aa35-aec3-451e-a641-94c974fd2806 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.042334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.043149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.043701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.043993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.044666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:create_application_credential(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.045886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.172223 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2501/5020] 10.222.0.33 () {60 vars in 1314 bytes} [Sat Sep 10 21:06:31 2022] POST /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 789 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:32.176734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.184426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:32.191840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1cb3d-60a6-47e8-adba-3d657add596c tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.198069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.199386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.199889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.200439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.201334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:create_application_credential(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.202828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.321796 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2520/5021] 10.222.0.33 () {60 vars in 1314 bytes} [Sat Sep 10 21:06:32 2022] POST /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 788 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:32.325766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.358292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88ae03-412b-4881-91df-128c27d789ca tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.384630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.385766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.386245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.386753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.387566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:list_application_credentials(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.390968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.448182 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2502/5022] 10.222.0.33 () {58 vars in 1293 bytes} [Sat Sep 10 21:06:32 2022] GET /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 1501 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:32.449702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.471125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501dd92-5cb5-404a-9c1f-f24f85ceaebb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.495089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.495914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.496308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.496797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/8559bd5f757c4d339734ee2a22a81b42` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.517855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:delete_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=8559bd5f757c4d339734ee2a22a81b42)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.519573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.572977 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2521/5023] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:06:32 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/8559bd5f757c4d339734ee2a22a81b42 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:32.574615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.589044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bac06a-69ad-428c-930c-25ec3363b1eb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.596496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.602858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.603396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.603835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/d9b72299941b4f0b914fa1916dbfaff1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.625216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:delete_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=d9b72299941b4f0b914fa1916dbfaff1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.627113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.681871 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2503/5024] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:06:32 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/d9b72299941b4f0b914fa1916dbfaff1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:32.687795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.702809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c12dd-4ef2-476f-befe-c77aac0b97be tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.726517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.727878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.728559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.729066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.730064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:create_application_credential(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.731749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.802869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.805916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:32.875965 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2522/5025] 10.222.0.33 () {60 vars in 1314 bytes} [Sat Sep 10 21:06:32 2022] POST /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 789 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:32.875965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:32.876234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:32.954009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4624388-ada2-471e-9807-647f10361783 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:32.960452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:32.961999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:32.962456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:32.963133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] PATH_INFO: `/v3/users/21e68efa994b4dd3b0dde8e160544267` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:32.966576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e68efa994b4dd3b0dde8e160544267)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:32.972566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:32.994021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:33.027677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743928dc-7545-410f-822c-188fa7eca49b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.035407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.036696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.037228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.038625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] PATH_INFO: `/v3/projects/e1e4eccf74b240b2b2dd86d909762636` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.041651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e4eccf74b240b2b2dd86d909762636)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.046613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.054751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e68efa994b4dd3b0dde8e160544267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.060250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:33.067760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e68efa994b4dd3b0dde8e160544267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.069275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e68efa994b4dd3b0dde8e160544267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.082807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e68efa994b4dd3b0dde8e160544267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.083892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e68efa994b4dd3b0dde8e160544267'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.115385 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2504/5026] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:32 2022] DELETE /identity/v3/users/21e68efa994b4dd3b0dde8e160544267 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:33.117667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.119383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.236982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20fe4-9398-45c5-9b56-4c6ad38a2fb0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.243576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599a980-9339-4776-ae1e-2f72b3b24439 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.244832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599a980-9339-4776-ae1e-2f72b3b24439 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.245212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599a980-9339-4776-ae1e-2f72b3b24439 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.245545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599a980-9339-4776-ae1e-2f72b3b24439 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.246288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599a980-9339-4776-ae1e-2f72b3b24439 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:create_application_credential(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.247525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599a980-9339-4776-ae1e-2f72b3b24439 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.339499 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2505/5027] 10.222.0.33 () {60 vars in 1314 bytes} [Sat Sep 10 21:06:33 2022] POST /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials => generated 788 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:33.340911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9a4c7-5450-4427-a007-83659da8e020 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.341339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9a4c7-5450-4427-a007-83659da8e020 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.341700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9a4c7-5450-4427-a007-83659da8e020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.353752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e4eccf74b240b2b2dd86d909762636', 'request_id': 'req-b2ae5528-dcfe-4956-bae9-9841c719bab2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.374775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-29c9a4c7-5450-4427-a007-83659da8e020 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:33.376030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-29c9a4c7-5450-4427-a007-83659da8e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.389320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e4eccf74b240b2b2dd86d909762636', 'request_id': 'req-b2ae5528-dcfe-4956-bae9-9841c719bab2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.400631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e4eccf74b240b2b2dd86d909762636', 'request_id': 'req-b2ae5528-dcfe-4956-bae9-9841c719bab2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.420375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:33.465609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] The token cache is being invalidate because project e1e4eccf74b240b2b2dd86d909762636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:33.465952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.469558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b2ae5528-dcfe-4956-bae9-9841c719bab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:33.469719 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2506/5028] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:33.469719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9a4c7-5450-4427-a007-83659da8e020 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.469911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2523/5029] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:32 2022] DELETE /identity/v3/projects/e1e4eccf74b240b2b2dd86d909762636 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:33.490414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.490717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.491538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.523438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:33.525127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.548228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9a4c7-5450-4427-a007-83659da8e020 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.558023 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2524/5030] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:33.558023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.558023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.560037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955e670-4b9c-4872-878b-1b60f492eceb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.560708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955e670-4b9c-4872-878b-1b60f492eceb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.560996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955e670-4b9c-4872-878b-1b60f492eceb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.561278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955e670-4b9c-4872-878b-1b60f492eceb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.561952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955e670-4b9c-4872-878b-1b60f492eceb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Adding query filter params (name=tempest-application_credential-930539014) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:33.562270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955e670-4b9c-4872-878b-1b60f492eceb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:list_application_credentials(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.564664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955e670-4b9c-4872-878b-1b60f492eceb tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.583949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2507/5031] 10.222.0.33 () {58 vars in 1384 bytes} [Sat Sep 10 21:06:33 2022] GET /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials?name=tempest-application_credential-930539014 => generated 885 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:33.585642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.585977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.586632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.624227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:33.625433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.638643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2508/5032] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:33.639610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.641072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.661401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3769a7-ddce-4e8d-9e0a-6a201b45c193 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.671551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.671691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b9d90-9045-4238-aad1-7055e29ac650 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.672412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.672766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.673218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/97a573a731d04feb8b9cf758c847fb26` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.693160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:delete_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=97a573a731d04feb8b9cf758c847fb26)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.695286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.703712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.706528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.708363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.708780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.713747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.735408 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2509/5033] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:06:33 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/97a573a731d04feb8b9cf758c847fb26 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:33.736516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.744218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.745973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.758733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca8f5d-2091-4480-be72-6bc64c30631b tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.764469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.765246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.765592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.766210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.767744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:33.768126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.769977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.778537 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2525/5034] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:33.785235 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2510/5035] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:33.786814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.789748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.802923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8472fd8-e64e-4823-9996-2878da64e227 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.810093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.810810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.811321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ede73-8a5f-4c00-8a27-46bc19003c88 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:33.811438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.811755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.812475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.815037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.828428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5986165cb874ceb8e0810863a81c406', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:33.829352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:33.829562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:33.829981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/5952f5e844334fefb8ea44cc698cd5ec` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:33.873500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorizing `identity:delete_application_credential(user_id=f5986165cb874ceb8e0810863a81c406, application_credential_id=5952f5e844334fefb8ea44cc698cd5ec)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:33.875830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:33.881275 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2511/5036] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:33 2022] POST /identity/v3/projects => generated 363 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:33.883589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.924986 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2526/5037] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:06:33 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406/application_credentials/5952f5e844334fefb8ea44cc698cd5ec => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:33.930071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:33.935967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:33.977763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e5c44-3819-4026-8f24-fc0a91b2d549 tempest-ApplicationCredentialsV3Test-107175780 tempest-ApplicationCredentialsV3Test-107175780-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:34.023136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcd4a3-4093-4da7-9357-d768eed9fdfe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:34.034924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:34.035584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.035873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.036152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.036902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:34.039687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:34.050632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:34.116960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2512/5038] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:33 2022] POST /identity/v3/users => generated 282 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:34.119694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:34.147569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:34.148186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.148477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.148751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] PATH_INFO: `/v3/users/f5986165cb874ceb8e0810863a81c406` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.150133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5986165cb874ceb8e0810863a81c406)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:34.152221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:34.156039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:34.171507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63fd82-4a36-4a50-9c89-798758ec7e2d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:34.199847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:34.206943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.207162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.209036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.211274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:34.215416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:34.253665 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2513/5039] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:34.259659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5986165cb874ceb8e0810863a81c406'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:34.261101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:34.297704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5986165cb874ceb8e0810863a81c406'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:34.299211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e942-e657-4d5b-bbf0-67741f1fe4bd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:34.307825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:34.309063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.309382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5986165cb874ceb8e0810863a81c406'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:34.309505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.309931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.310678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:34.312589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:34.322117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5986165cb874ceb8e0810863a81c406'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:34.323768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4935996b-5649-4164-be42-47aef6b39c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5986165cb874ceb8e0810863a81c406'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:34.332287 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2527/5040] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:33 2022] DELETE /identity/v3/users/f5986165cb874ceb8e0810863a81c406 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:34.339393 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2514/5041] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:34.358993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.359743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:34.359982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.360459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.364119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:34.394613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31e49c-1e30-43d0-8028-7340d2341ebb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:34.395497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:34.396833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:34.531586 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2528/5042] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:34.545555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:34.547453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:34.593670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:34.595306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.595777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.596367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] PATH_INFO: `/v3/projects/c06beec165b94160b79fb689bb7e0e0d/users/c8b015ae87f44d35be4d9e3a0f4d07d9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.641242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06beec165b94160b79fb689bb7e0e0d, user_id=c8b015ae87f44d35be4d9e3a0f4d07d9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:34.647847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:34.680598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e1796-3cb8-4bcf-a73b-95776ac64be3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:34.682130 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2515/5043] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:34 2022] PUT /identity/v3/projects/c06beec165b94160b79fb689bb7e0e0d/users/c8b015ae87f44d35be4d9e3a0f4d07d9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:34.690095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:34.690095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:34.760758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:34.800788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bb615-58e5-489d-8ce0-afb19fda6193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:34.801401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bb615-58e5-489d-8ce0-afb19fda6193 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.801876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bb615-58e5-489d-8ce0-afb19fda6193 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.802157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bb615-58e5-489d-8ce0-afb19fda6193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.838162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb5d5a-b6e1-4d2f-8aa1-400911840b5b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:34.853671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:34.855097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.855450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.856248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] PATH_INFO: `/v3/projects/839c3d9e368b42f78a055d529e7c5bfc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.859019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] RBAC: Authorizing `identity:delete_project(project_id=839c3d9e368b42f78a055d529e7c5bfc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:34.863071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:34.891404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bb615-58e5-489d-8ce0-afb19fda6193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:34.893293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bb615-58e5-489d-8ce0-afb19fda6193 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:34.911859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2529/5044] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:34.913970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:34.914538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:34.914968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:34.951225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] MFA Rules not processed for user `c8b015ae87f44d35be4d9e3a0f4d07d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:34.952752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:34.997763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839c3d9e368b42f78a055d529e7c5bfc', 'request_id': 'req-6336fca5-2fc0-4564-8908-4d54cd61ebce'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:35.010656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839c3d9e368b42f78a055d529e7c5bfc', 'request_id': 'req-6336fca5-2fc0-4564-8908-4d54cd61ebce'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:35.013153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:35.025672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839c3d9e368b42f78a055d529e7c5bfc', 'request_id': 'req-6336fca5-2fc0-4564-8908-4d54cd61ebce'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:35.051876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] The token cache is being invalidate because project 839c3d9e368b42f78a055d529e7c5bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:35.054671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:35.057015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6336fca5-2fc0-4564-8908-4d54cd61ebce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:35.060384 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2516/5045] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:34 2022] DELETE /identity/v3/projects/839c3d9e368b42f78a055d529e7c5bfc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:35.068719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645c9e9-1f19-4e8c-9f01-2babff4e1f4e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.069103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645c9e9-1f19-4e8c-9f01-2babff4e1f4e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.069645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645c9e9-1f19-4e8c-9f01-2babff4e1f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.091258 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2530/5046] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:35.095612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.098514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:35.129210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5645c9e9-1f19-4e8c-9f01-2babff4e1f4e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:35.130473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5645c9e9-1f19-4e8c-9f01-2babff4e1f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:35.137859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a858ca0-06b7-4a17-a01d-7d2db30b0198 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.248811 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2517/5047] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:35.258348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd363624-d7d2-4459-8c90-b7bf743114b4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.258641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd363624-d7d2-4459-8c90-b7bf743114b4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.258910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd363624-d7d2-4459-8c90-b7bf743114b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.295404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cd363624-d7d2-4459-8c90-b7bf743114b4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:35.296970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cd363624-d7d2-4459-8c90-b7bf743114b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:35.298197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.298968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.299378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.299652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.302665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.305592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.307008 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2518/5048] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:35.310931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd363624-d7d2-4459-8c90-b7bf743114b4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.328361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd363624-d7d2-4459-8c90-b7bf743114b4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.336459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.338389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.338857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.339284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.340096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:35.340623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.344727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.349333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:35.370834 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2531/5049] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:35 2022] POST /identity/v3/projects => generated 363 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:35.372000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.372912 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2519/5050] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:35.380004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.399037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ee565-7d0d-4c17-9102-a338f398916a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.503443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e894a5d-424c-4323-9691-382fc1372634 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.523719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.525909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.526366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.527622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.528491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.532522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.568961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.569769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.570238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.570647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.571495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.574877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.579729 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2532/5051] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:35 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:35.584195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.598381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685164-ea7b-4502-97d7-ea77f805934a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.610501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.611203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.611490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.611904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.612557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.614882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.637897 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2533/5052] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:35.644782 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2520/5053] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:35 2022] POST /identity/v3/projects => generated 377 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:35.645634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.646469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.670556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00a341-8eb7-4e42-8a50-912ccbc42d4b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.807189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:35.840486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.841565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.841865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.842160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] PATH_INFO: `/v3/projects/9086698824a3457b8bdbd9f1cc4ddaaf/users/72827b24892a420fab1be776a91f3b66/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.843618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409afc80-bbd1-4ea9-af4a-d5b5dbb321cf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.852322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.853515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.854638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.855134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.856011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.858533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.884240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9086698824a3457b8bdbd9f1cc4ddaaf, user_id=72827b24892a420fab1be776a91f3b66, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.893984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2521/5054] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:35 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:35.899230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.901120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.914882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79269e-1394-421b-8ed6-6e10def8fd7b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:35.922413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:35.923459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:35.924276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:35.924754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:35.925602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:35.927758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:35.928049 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2534/5055] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:35 2022] PUT /identity/v3/projects/9086698824a3457b8bdbd9f1cc4ddaaf/users/72827b24892a420fab1be776a91f3b66/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:35.935575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.945714 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2522/5056] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:35.949931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:35.964546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f396241-fe1a-4a33-8583-3bee804cc46e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:36.115192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:36.119794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.120154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.120438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.121107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:36.123567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:36.138607 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2523/5057] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:36.142296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1012c-2818-41fe-8927-96ff622b0bca admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:36.142727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:36.157563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:36.158250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.158529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.158799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.159445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:36.161037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:36.162732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:36.171679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c650d0-cb62-42c6-a6f8-66f816e2f11b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:36.177463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:36.179463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.179894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.180179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] PATH_INFO: `/v3/projects/44cac27a013c491199ea5392ceb9639b/users/7d2478d458b14f2a9bbc0005886bf4cf/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.190706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2535/5058] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:36.191656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:36.198526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cac27a013c491199ea5392ceb9639b, user_id=7d2478d458b14f2a9bbc0005886bf4cf, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:36.201327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:36.203225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:36.218557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c6453-463e-45e0-bf72-517fb626ddfa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:36.223611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2524/5059] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:36 2022] PUT /identity/v3/projects/44cac27a013c491199ea5392ceb9639b/users/7d2478d458b14f2a9bbc0005886bf4cf/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:36.224178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:36.308211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:36.309141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.309599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.310124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] PATH_INFO: `/v3/projects/b72b644fc5184f8ba6087ba73d4cdf4c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.311495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72b644fc5184f8ba6087ba73d4cdf4c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:36.313716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:36.417305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'request_id': 'req-0f66e9ea-e768-4a70-a408-3c08279c1c15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:36.420079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a519144-5434-48f6-bcf5-62d2102fb1ff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:36.438995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'request_id': 'req-0f66e9ea-e768-4a70-a408-3c08279c1c15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:36.460522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72b644fc5184f8ba6087ba73d4cdf4c', 'request_id': 'req-0f66e9ea-e768-4a70-a408-3c08279c1c15'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:36.489333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] The token cache is being invalidate because project b72b644fc5184f8ba6087ba73d4cdf4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:36.489631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:36.492137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f66e9ea-e768-4a70-a408-3c08279c1c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:36.496557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2536/5060] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:36 2022] DELETE /identity/v3/projects/b72b644fc5184f8ba6087ba73d4cdf4c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:36.503115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90abb5-37e7-4a29-be70-a34950fb910b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.503407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90abb5-37e7-4a29-be70-a34950fb910b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.503687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90abb5-37e7-4a29-be70-a34950fb910b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.508176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:36.549192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b275c60-182c-494c-872e-ce4f4fcb5b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:36.550117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b275c60-182c-494c-872e-ce4f4fcb5b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.550403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b275c60-182c-494c-872e-ce4f4fcb5b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.551406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b275c60-182c-494c-872e-ce4f4fcb5b1c admin admin] PATH_INFO: `/v3/projects/9086698824a3457b8bdbd9f1cc4ddaaf/users/72827b24892a420fab1be776a91f3b66/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.557057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b275c60-182c-494c-872e-ce4f4fcb5b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9086698824a3457b8bdbd9f1cc4ddaaf, user_id=72827b24892a420fab1be776a91f3b66, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:36.560993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b275c60-182c-494c-872e-ce4f4fcb5b1c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:36.579053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7b90abb5-37e7-4a29-be70-a34950fb910b None None] MFA Rules not processed for user `7d2478d458b14f2a9bbc0005886bf4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:36.581090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b90abb5-37e7-4a29-be70-a34950fb910b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:36.596980 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2525/5061] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:36 2022] PUT /identity/v3/projects/9086698824a3457b8bdbd9f1cc4ddaaf/users/72827b24892a420fab1be776a91f3b66/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:36.608913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2eff2-9113-4d4e-9a93-272865f1588e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.609210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2eff2-9113-4d4e-9a93-272865f1588e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.609490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2eff2-9113-4d4e-9a93-272865f1588e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.660079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4bb2eff2-9113-4d4e-9a93-272865f1588e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:36.662583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4bb2eff2-9113-4d4e-9a93-272865f1588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:36.679121 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2537/5062] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:36.701846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755d87-b64d-42fa-b0f4-69605018a757 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.702438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755d87-b64d-42fa-b0f4-69605018a757 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.702735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755d87-b64d-42fa-b0f4-69605018a757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.744880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-95755d87-b64d-42fa-b0f4-69605018a757 None None] MFA Rules not processed for user `72827b24892a420fab1be776a91f3b66`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:36.746415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-95755d87-b64d-42fa-b0f4-69605018a757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:36.751737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:36.832306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2526/5063] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:36.834311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d68cb-8c8b-4972-97d4-6986b57bdc91 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.836477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d68cb-8c8b-4972-97d4-6986b57bdc91 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.836781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d68cb-8c8b-4972-97d4-6986b57bdc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.868883 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2538/5064] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:36.871777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:36.872248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:36.872670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:36.908221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bf8d68cb-8c8b-4972-97d4-6986b57bdc91 None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:36.909463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bf8d68cb-8c8b-4972-97d4-6986b57bdc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:36.911705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] MFA Rules not processed for user `9546792be5a640e2a59fb2e8481c6c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:36.913707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.021121 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2539/5065] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:37.034153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:37.039558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.069937 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2527/5066] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:37.069937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.069937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.069937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.127081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:37.127081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.146005 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2528/5067] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:37.150741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:37.153315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.175314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a6cbb-c14e-4b9c-b636-872480d7e6ef None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:37.181691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd80e8-bcab-4da1-81f6-cf44ea32cf9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:37.182479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd80e8-bcab-4da1-81f6-cf44ea32cf9a service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.182856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd80e8-bcab-4da1-81f6-cf44ea32cf9a service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.184002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd80e8-bcab-4da1-81f6-cf44ea32cf9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.219620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd80e8-bcab-4da1-81f6-cf44ea32cf9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:37.222021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd80e8-bcab-4da1-81f6-cf44ea32cf9a service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:37.247490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2540/5068] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:37.251131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee79a-61d1-4250-806d-accda52bdb9a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.251420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee79a-61d1-4250-806d-accda52bdb9a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.251700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee79a-61d1-4250-806d-accda52bdb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.308396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ad5ee79a-61d1-4250-806d-accda52bdb9a None None] MFA Rules not processed for user `9546792be5a640e2a59fb2e8481c6c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:37.312355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ad5ee79a-61d1-4250-806d-accda52bdb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.336170 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2541/5069] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:37.341977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63030486-05f8-4caa-959f-d409894beb1b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.342318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63030486-05f8-4caa-959f-d409894beb1b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.342598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63030486-05f8-4caa-959f-d409894beb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.357914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7927000-3b2b-48f6-9227-13057718e317 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:37.379182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:37.379872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.380172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.382849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.411064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:37.412288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-63030486-05f8-4caa-959f-d409894beb1b None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:37.415389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-63030486-05f8-4caa-959f-d409894beb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.416264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:37.425135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:37.440086 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2542/5070] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:37.440810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63030486-05f8-4caa-959f-d409894beb1b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:37.443223 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2529/5071] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:37.447518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:37.451766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.480466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63030486-05f8-4caa-959f-d409894beb1b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:37.497010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:37.499400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.500791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.502224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.505862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:37.506036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:37.512741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:37.557494 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2543/5072] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:37.558538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:37.566292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.606259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f85c-527a-4b22-aeec-2aa7f4c08f6c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:37.612821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:37.613468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.613754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.614085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] PATH_INFO: `/v3/users/7d2478d458b14f2a9bbc0005886bf4cf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.615260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2478d458b14f2a9bbc0005886bf4cf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:37.617784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:37.619499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:37.714637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd33638-33c4-40fc-8d30-c7c2d5eeb9a8 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:37.720017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2478d458b14f2a9bbc0005886bf4cf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:37.721116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:37.722010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.722393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.722902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.756443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:37.760136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:37.765858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2478d458b14f2a9bbc0005886bf4cf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:37.779133 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2530/5073] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:37.779950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2478d458b14f2a9bbc0005886bf4cf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:37.783613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:37.795499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2478d458b14f2a9bbc0005886bf4cf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:37.796055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2478d458b14f2a9bbc0005886bf4cf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:37.810161 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2544/5074] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:37 2022] DELETE /identity/v3/users/7d2478d458b14f2a9bbc0005886bf4cf => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:37.811369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:37.813926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:37.814989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efa58d-2c0b-4e73-8062-3e3d0bdbba2d service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:37.954902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff549fad-cdbe-4999-a904-dbdc7a5c85d6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:37.957050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b2227-e0e0-4e34-8dfa-b99de7841c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:37.958492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2227-e0e0-4e34-8dfa-b99de7841c72 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.980195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2227-e0e0-4e34-8dfa-b99de7841c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.980904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2227-e0e0-4e34-8dfa-b99de7841c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.982183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b2227-e0e0-4e34-8dfa-b99de7841c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:37.984969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b2227-e0e0-4e34-8dfa-b99de7841c72 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:37.985790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:37.988711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:37.989108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:37.990124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:37.996261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:38.065921 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2531/5075] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:37 2022] POST /identity/v3/projects => generated 359 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:38.070104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.070562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.070939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.119868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:38.123219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:38.135050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:38.137570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:38.162092 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2545/5076] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:38.165657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:38.170653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:38.201926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:38.226050 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2532/5077] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:38.227504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:38.229147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:38.245464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c19e9-ae05-4fb7-8a9c-5e997404257b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:38.251528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:38.252315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.252693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.253063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.253794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:38.256009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:38.264483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:38.291378 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2533/5078] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:38 2022] POST /identity/v3/users => generated 280 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:38.293331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:38.304257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:38.356020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f6ecb-555d-4a72-8506-ad8d84e070e6 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:38.476645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e074a6-3566-49dc-a1e6-325143b09acb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:38.486803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:38.487611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.488907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.489261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.494879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:38.496471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.497313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.498172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.503657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:38.515409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:38.517042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:38.546616 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2534/5079] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:38.547961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:38.577009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe64b1-d2ee-402a-a347-d853e53649bb service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:38.588564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:38.589194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.589480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.592992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.594969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:38.597128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:38.625028 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2535/5080] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:38.629885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:38.645947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:38.645947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:38.661782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702ed39-10ec-4e14-bd42-e08f1bee4bdd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:38.675787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9808c4-853e-4abd-8891-2b1d8971c95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:38.676454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9808c4-853e-4abd-8891-2b1d8971c95e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.676733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9808c4-853e-4abd-8891-2b1d8971c95e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.677057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9808c4-853e-4abd-8891-2b1d8971c95e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.680209 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2546/5081] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:38.681257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9808c4-853e-4abd-8891-2b1d8971c95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:38.684310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9808c4-853e-4abd-8891-2b1d8971c95e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:38.714536 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2536/5082] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:38.716703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:38.740310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55b6b0-0283-41c8-9acf-5aadc25c9882 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:38.754724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd10034-85f1-4c42-9df4-976b10065ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:38.755598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10034-85f1-4c42-9df4-976b10065ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.755882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10034-85f1-4c42-9df4-976b10065ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.756490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10034-85f1-4c42-9df4-976b10065ddc admin admin] PATH_INFO: `/v3/projects/c787113bd8ed404b9e45f6ed5ed55882/users/870dcfc0b54842a8b9c6e3f657481fd8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.789870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd10034-85f1-4c42-9df4-976b10065ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c787113bd8ed404b9e45f6ed5ed55882, user_id=870dcfc0b54842a8b9c6e3f657481fd8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:38.793024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd10034-85f1-4c42-9df4-976b10065ddc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:38.824844 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2547/5083] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:38 2022] PUT /identity/v3/projects/c787113bd8ed404b9e45f6ed5ed55882/users/870dcfc0b54842a8b9c6e3f657481fd8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:38.827450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0af78-5c45-4e27-8d9c-84bfe2c5fc33 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:38.827743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0af78-5c45-4e27-8d9c-84bfe2c5fc33 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:38.828072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0af78-5c45-4e27-8d9c-84bfe2c5fc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:38.877937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-70a0af78-5c45-4e27-8d9c-84bfe2c5fc33 None None] MFA Rules not processed for user `870dcfc0b54842a8b9c6e3f657481fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:38.881563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70a0af78-5c45-4e27-8d9c-84bfe2c5fc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.009097 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2537/5084] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:39.014435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64837870-89e6-4f30-9286-f905f668747a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.014727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64837870-89e6-4f30-9286-f905f668747a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.015004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64837870-89e6-4f30-9286-f905f668747a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.050987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-64837870-89e6-4f30-9286-f905f668747a None None] MFA Rules not processed for user `870dcfc0b54842a8b9c6e3f657481fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:39.056977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-64837870-89e6-4f30-9286-f905f668747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.071910 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2548/5085] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:39 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:06:39.079448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ec37-849f-4c51-97c2-ea964b1998e6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.079448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ec37-849f-4c51-97c2-ea964b1998e6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.079717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ec37-849f-4c51-97c2-ea964b1998e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.104488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-52d5ec37-849f-4c51-97c2-ea964b1998e6 None None] MFA Rules not processed for user `870dcfc0b54842a8b9c6e3f657481fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:39.105786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-52d5ec37-849f-4c51-97c2-ea964b1998e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.109900 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2538/5086] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:39 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:06:39.113782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64837870-89e6-4f30-9286-f905f668747a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.128710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64837870-89e6-4f30-9286-f905f668747a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.133804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '870dcfc0b54842a8b9c6e3f657481fd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c787113bd8ed404b9e45f6ed5ed55882', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.134336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.134771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.135153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.142155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorizing `identity:check_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:39.144412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:39.159821 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2549/5087] 10.222.0.33 () {56 vars in 1264 bytes} [Sat Sep 10 21:06:39 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Sep 10 21:06:39.163180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5ec37-849f-4c51-97c2-ea964b1998e6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.179610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5ec37-849f-4c51-97c2-ea964b1998e6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.192110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '870dcfc0b54842a8b9c6e3f657481fd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c787113bd8ed404b9e45f6ed5ed55882', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.192952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.193480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.195004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.210937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:39.213814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:39.232380 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2539/5088] 10.222.0.33 () {56 vars in 1263 bytes} [Sat Sep 10 21:06:39 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 21:06:39.236557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.250330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0b1eb-a8ef-46fb-ab38-abbd2c2795a3 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.267751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cce319-5f43-46de-928c-5c4c3ff13517 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '870dcfc0b54842a8b9c6e3f657481fd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c787113bd8ed404b9e45f6ed5ed55882', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.268381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce319-5f43-46de-928c-5c4c3ff13517 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.268660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce319-5f43-46de-928c-5c4c3ff13517 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.268934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce319-5f43-46de-928c-5c4c3ff13517 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.284511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cce319-5f43-46de-928c-5c4c3ff13517 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:39.286069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cce319-5f43-46de-928c-5c4c3ff13517 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:39.338829 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2550/5089] 10.222.0.33 () {56 vars in 1266 bytes} [Sat Sep 10 21:06:39 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:39.342539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.355599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985bdc0-e931-4bb0-be51-563239b65622 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.366452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '870dcfc0b54842a8b9c6e3f657481fd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c787113bd8ed404b9e45f6ed5ed55882', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.367315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] REQUEST_METHOD: `HEAD` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.367707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.368112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.370388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Sep 10 21:06:39.424287 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 10 21:06:39.425385 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 10 21:06:39.428215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.428215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.428215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 10 21:06:39.428215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 10 21:06:39.428215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 10 21:06:39.428215 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:06:39.428215 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2540/5090] 10.222.0.33 () {56 vars in 1264 bytes} [Sat Sep 10 21:06:39 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 86 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) Sep 10 21:06:39.435353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fc95c-1004-472c-bcd8-3a6ec0fc6d21 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.436695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fc95c-1004-472c-bcd8-3a6ec0fc6d21 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.437201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fc95c-1004-472c-bcd8-3a6ec0fc6d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.456180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:39.499422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f51fc95c-1004-472c-bcd8-3a6ec0fc6d21 None None] MFA Rules not processed for user `870dcfc0b54842a8b9c6e3f657481fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:39.501936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f51fc95c-1004-472c-bcd8-3a6ec0fc6d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.516010 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2551/5091] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:39 2022] POST /identity/v3/auth/tokens => generated 345 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:06:39.520169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.544418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662440a-ba87-40f8-aec1-7893297ccd0c tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.558028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '870dcfc0b54842a8b9c6e3f657481fd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c787113bd8ed404b9e45f6ed5ed55882', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.558845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.560023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.563272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.583651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:39.587318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:39.624393 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2541/5092] 10.222.0.33 () {56 vars in 1263 bytes} [Sat Sep 10 21:06:39 2022] GET /identity/v3/auth/tokens => generated 345 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 21:06:39.627997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fc95c-1004-472c-bcd8-3a6ec0fc6d21 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.651603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fc95c-1004-472c-bcd8-3a6ec0fc6d21 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.671385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '870dcfc0b54842a8b9c6e3f657481fd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c787113bd8ed404b9e45f6ed5ed55882', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.671385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.671385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.671727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.694316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:39.696370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:39.735101 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2552/5093] 10.222.0.33 () {56 vars in 1266 bytes} [Sat Sep 10 21:06:39 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:39.739077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.744963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:39.759611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3433a-5ec5-4695-88f4-63923614d98a tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:39.774951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '870dcfc0b54842a8b9c6e3f657481fd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c787113bd8ed404b9e45f6ed5ed55882', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:39.778127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:39.782005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:39.782600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:39.783468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:39.783696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.789674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 10 21:06:39.824096 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 10 21:06:39.825257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 10 21:06:39.826642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:06:39.826642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:06:39.826642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 10 21:06:39.826642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 10 21:06:39.826642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 10 21:06:39.826642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:06:39.831564 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2542/5094] 10.222.0.33 () {56 vars in 1263 bytes} [Sat Sep 10 21:06:39 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 93 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 21:06:39.838839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:40.006004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce628b-6ec6-46e0-86e5-d9cd96b1a884 tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:40.036724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:40.041314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:40.043475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:40.044132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] PATH_INFO: `/v3/projects/44cac27a013c491199ea5392ceb9639b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:40.050380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=44cac27a013c491199ea5392ceb9639b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:40.057619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:40.079799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf203d-6f8d-473f-a129-13438b5af30b tempest-TokensV3Test-930089324 tempest-TokensV3Test-930089324-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:40.085312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:40.086571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:40.086916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:40.087298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] PATH_INFO: `/v3/users/870dcfc0b54842a8b9c6e3f657481fd8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:40.090255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=870dcfc0b54842a8b9c6e3f657481fd8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:40.093257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:40.240661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44cac27a013c491199ea5392ceb9639b', 'request_id': 'req-a6a99a24-683e-4023-80a3-0adda7d8adfe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.254188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44cac27a013c491199ea5392ceb9639b', 'request_id': 'req-a6a99a24-683e-4023-80a3-0adda7d8adfe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.254942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870dcfc0b54842a8b9c6e3f657481fd8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.274153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44cac27a013c491199ea5392ceb9639b', 'request_id': 'req-a6a99a24-683e-4023-80a3-0adda7d8adfe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.293151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870dcfc0b54842a8b9c6e3f657481fd8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.297277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870dcfc0b54842a8b9c6e3f657481fd8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.301356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] The token cache is being invalidate because project 44cac27a013c491199ea5392ceb9639b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:40.301650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.306208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a6a99a24-683e-4023-80a3-0adda7d8adfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.311315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870dcfc0b54842a8b9c6e3f657481fd8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.313158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a1048655-312d-47b7-be09-434696ff51fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870dcfc0b54842a8b9c6e3f657481fd8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:40.321186 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2553/5095] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:39 2022] DELETE /identity/v3/projects/44cac27a013c491199ea5392ceb9639b => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:40.326450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a6f8-1737-4f7b-84a1-928591bfc144 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:40.326742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a6f8-1737-4f7b-84a1-928591bfc144 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:40.327009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a6f8-1737-4f7b-84a1-928591bfc144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:40.360399 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2543/5096] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:39 2022] DELETE /identity/v3/users/870dcfc0b54842a8b9c6e3f657481fd8 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:40.364500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:40.364795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:40.365109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:40.408593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-08f0a6f8-1737-4f7b-84a1-928591bfc144 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:40.409943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-08f0a6f8-1737-4f7b-84a1-928591bfc144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:40.412539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:40.415185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:40.614003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2544/5097] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:40.627241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:40.629224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:40.673092 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2554/5098] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:40.684739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505919e-d9e7-49b0-a74d-5fdc7e47b666 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:40.685431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505919e-d9e7-49b0-a74d-5fdc7e47b666 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:40.685713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505919e-d9e7-49b0-a74d-5fdc7e47b666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:40.718259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:40.765056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0505919e-d9e7-49b0-a74d-5fdc7e47b666 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:40.766356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0505919e-d9e7-49b0-a74d-5fdc7e47b666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:40.842222 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2555/5099] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:40.843343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505919e-d9e7-49b0-a74d-5fdc7e47b666 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:40.862052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:40.875112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505919e-d9e7-49b0-a74d-5fdc7e47b666 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:40.884651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:40.886311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:40.886759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:40.887161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:40.888866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:40.889282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:40.907587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:40.982174 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2556/5100] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:40.987848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:41.009868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48c142-7962-4e8d-bc36-1d6851592b95 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:41.016130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1b308-3881-4aac-9ccf-a3840df6ada4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:41.022737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:41.024326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:41.024630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:41.026085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:41.027832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:41.032029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:41.032714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:41.033007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:41.033288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:41.034081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:41.098820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:41.105592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:41.118124 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2557/5101] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:40 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:41.119092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:41.154467 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2545/5102] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:41.333446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352474d-f235-49d2-b79b-63a51fcc7f23 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:41.363754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:41.365036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:41.366014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:41.368385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:41.369360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:41.374091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:41.433728 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2558/5103] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:41 2022] POST /identity/v3/users => generated 292 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:41.438135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:41.455164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa145e-1b08-4132-a6ad-284298117ea6 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:41.466819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:41.469382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:41.469715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:41.472746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:41.473725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:41.482008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:41.495920 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2546/5104] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:41.505352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:41.555311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b49eb5-f429-4a19-996b-7d1c75537402 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:41.562223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea054c7-4345-405c-92f4-062403d23b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:41.563140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea054c7-4345-405c-92f4-062403d23b1f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:41.563253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea054c7-4345-405c-92f4-062403d23b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:41.563568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea054c7-4345-405c-92f4-062403d23b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:41.564681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea054c7-4345-405c-92f4-062403d23b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:41.566386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea054c7-4345-405c-92f4-062403d23b1f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:41.580495 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2559/5105] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:41.588479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:41.608117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7eb2c-c162-426c-9a27-5b3e3a939615 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:41.641328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311acf7f-ba3a-406f-92f9-a8347706eb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:41.645675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311acf7f-ba3a-406f-92f9-a8347706eb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:41.645961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311acf7f-ba3a-406f-92f9-a8347706eb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:41.648344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311acf7f-ba3a-406f-92f9-a8347706eb5b admin admin] PATH_INFO: `/v3/projects/ec34a73b1f19424fa30b3f1702b80cb6/users/de070039cd2947d584c4fba38d6cb610/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:41.711534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311acf7f-ba3a-406f-92f9-a8347706eb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec34a73b1f19424fa30b3f1702b80cb6, user_id=de070039cd2947d584c4fba38d6cb610, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:41.718338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311acf7f-ba3a-406f-92f9-a8347706eb5b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:41.761198 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2547/5106] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:41 2022] PUT /identity/v3/projects/ec34a73b1f19424fa30b3f1702b80cb6/users/de070039cd2947d584c4fba38d6cb610/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:41.771414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a542b-d43f-4518-a12c-7ffb0f96bd49 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:41.771414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a542b-d43f-4518-a12c-7ffb0f96bd49 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:41.772922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a542b-d43f-4518-a12c-7ffb0f96bd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:41.891124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-083a542b-d43f-4518-a12c-7ffb0f96bd49 None None] MFA Rules not processed for user `de070039cd2947d584c4fba38d6cb610`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:41.896096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-083a542b-d43f-4518-a12c-7ffb0f96bd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:42.090146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9003d3-5538-44c6-a8a1-c6326ea2cbd6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:42.090468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9003d3-5538-44c6-a8a1-c6326ea2cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:42.090785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9003d3-5538-44c6-a8a1-c6326ea2cbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:42.125118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ef9003d3-5538-44c6-a8a1-c6326ea2cbd6 None None] MFA Rules not processed for user `4faa458c158141efaf707317411cd544`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:42.129765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ef9003d3-5538-44c6-a8a1-c6326ea2cbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:42.152207 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2560/5107] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:42.155657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a542b-d43f-4518-a12c-7ffb0f96bd49 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:42.292107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:42.314554 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2548/5108] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:42.367382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a542b-d43f-4518-a12c-7ffb0f96bd49 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:42.380263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:42.380890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:42.383187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:42.383485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:42.384145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:42.386590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:42.460065 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2561/5109] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:42 2022] POST /identity/v3/projects => generated 385 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:42.461128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9003d3-5538-44c6-a8a1-c6326ea2cbd6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:42.499280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9003d3-5538-44c6-a8a1-c6326ea2cbd6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:42.596734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:42.597626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:42.598101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:42.598452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:42.599613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:42.602282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:42.665815 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2549/5110] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:42 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:42.670967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:42.692019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813561c7-b87a-4bf9-ac8b-52013b9060cf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:42.711700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:42.712338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:42.712662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:42.712942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:42.713593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:42.727236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:42.762241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:42.780316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2562/5111] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:42.789760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89021315-e11f-407b-92d1-1073d6803d47 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:42.802555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:42.803857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:42.804144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:42.804422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] PATH_INFO: `/v3/projects/674182f91ce849dc8e9be93cdf3f5daa/users/0b6f6ccbceb04497bf50fead1235e59b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:42.841445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:42.850214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=674182f91ce849dc8e9be93cdf3f5daa, user_id=0b6f6ccbceb04497bf50fead1235e59b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:42.856057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:42.903027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:42.913798 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2550/5112] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:42 2022] PUT /identity/v3/projects/674182f91ce849dc8e9be93cdf3f5daa/users/0b6f6ccbceb04497bf50fead1235e59b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:42.917168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324703d-0b24-49ce-af9a-5b07afe9ae3d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:43.106306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23faa7c-ff53-444b-afa5-775663f1be73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:43.106899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23faa7c-ff53-444b-afa5-775663f1be73 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.111320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23faa7c-ff53-444b-afa5-775663f1be73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.111667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23faa7c-ff53-444b-afa5-775663f1be73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.112374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23faa7c-ff53-444b-afa5-775663f1be73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:43.115608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23faa7c-ff53-444b-afa5-775663f1be73 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:43.141039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2563/5113] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:43.145342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:43.163362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf86b0-d49b-4de3-9632-02ab411d6fd1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:43.177306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:43.179142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.180503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.180962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] PATH_INFO: `/v3/projects/674182f91ce849dc8e9be93cdf3f5daa/users/0b6f6ccbceb04497bf50fead1235e59b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.187308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=674182f91ce849dc8e9be93cdf3f5daa, user_id=0b6f6ccbceb04497bf50fead1235e59b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:43.191371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:43.232830 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2551/5114] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:43 2022] PUT /identity/v3/projects/674182f91ce849dc8e9be93cdf3f5daa/users/0b6f6ccbceb04497bf50fead1235e59b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:43.244718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.245251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.245716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.321351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] MFA Rules not processed for user `0b6f6ccbceb04497bf50fead1235e59b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:43.326225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:43.441888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:43.444110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:43.513434 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2564/5115] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:43.534148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:43.535890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:43.651275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:43.684730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78c2ef-096d-4d60-8388-15ec6b030048 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:43.697623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:43.698424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.698649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.700038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] PATH_INFO: `/v3/projects/c787113bd8ed404b9e45f6ed5ed55882` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.703834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c787113bd8ed404b9e45f6ed5ed55882)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:43.707295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:43.759704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca647ce-225b-4233-bc9d-44a61fa64acb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:43.773663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8e671-2614-4e28-b13b-b91ae5931943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:43.776638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8e671-2614-4e28-b13b-b91ae5931943 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.777911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8e671-2614-4e28-b13b-b91ae5931943 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.779548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8e671-2614-4e28-b13b-b91ae5931943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:43.839451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8e671-2614-4e28-b13b-b91ae5931943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:43.859776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8e671-2614-4e28-b13b-b91ae5931943 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:43.870437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c787113bd8ed404b9e45f6ed5ed55882', 'request_id': 'req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.895622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c787113bd8ed404b9e45f6ed5ed55882', 'request_id': 'req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.905769 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2565/5116] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:43.927094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c787113bd8ed404b9e45f6ed5ed55882', 'request_id': 'req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.951871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] The token cache is being invalidate because project c787113bd8ed404b9e45f6ed5ed55882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:43.952146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.952875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eeb1dbbd-d411-4f2f-98fa-5d64a073409b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:43.954951 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2552/5117] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:43 2022] DELETE /identity/v3/projects/c787113bd8ed404b9e45f6ed5ed55882 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:43.960256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b60b7-3b19-469b-a652-f97fd798e79f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:43.960561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b60b7-3b19-469b-a652-f97fd798e79f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:43.965639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b60b7-3b19-469b-a652-f97fd798e79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.023975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c02b60b7-3b19-469b-a652-f97fd798e79f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:44.026790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c02b60b7-3b19-469b-a652-f97fd798e79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:44.272934 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2566/5118] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:44.280769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f1466-aa20-47dd-a43f-ff6cdd46e097 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.281313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f1466-aa20-47dd-a43f-ff6cdd46e097 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.281795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f1466-aa20-47dd-a43f-ff6cdd46e097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.360195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a17f1466-aa20-47dd-a43f-ff6cdd46e097 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:44.361472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a17f1466-aa20-47dd-a43f-ff6cdd46e097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:44.411303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b60b7-3b19-469b-a652-f97fd798e79f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.418459 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2553/5119] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:44.425500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b60b7-3b19-469b-a652-f97fd798e79f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.449965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.450780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.450902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.451199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.451817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:44.452136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.453741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.483036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f1466-aa20-47dd-a43f-ff6cdd46e097 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.503082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f1466-aa20-47dd-a43f-ff6cdd46e097 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.503176 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2567/5120] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:44.504136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.507298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:44.508705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.509755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.510138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.510419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.511379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.514635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.536256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66b3c7-3a9e-4f5f-8808-c14369b83543 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.549738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:44.550483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.550773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.551054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] PATH_INFO: `/v3/users/de070039cd2947d584c4fba38d6cb610` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.554667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de070039cd2947d584c4fba38d6cb610)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:44.557656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:44.657751 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2554/5121] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:44.680061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:44.722784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de070039cd2947d584c4fba38d6cb610'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.763157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de070039cd2947d584c4fba38d6cb610'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.790046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de070039cd2947d584c4fba38d6cb610'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.802673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de070039cd2947d584c4fba38d6cb610'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.804305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4090bf3d-ba77-4505-a73c-d611ad73533d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de070039cd2947d584c4fba38d6cb610'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:44.815704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2568/5122] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:44 2022] DELETE /identity/v3/users/de070039cd2947d584c4fba38d6cb610 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:44.817573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:44.817958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:44.818306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:44.842542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:44.910657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:44.935837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301bae5-7072-4612-aa20-c3e912c367e0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:44.936941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:45.055311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:45.107953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:45.108889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:45.109321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:45.110511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:45.116420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:45.122425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:45.150830 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2569/5123] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:45.176076 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2555/5124] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:44 2022] POST /identity/v3/users => generated 289 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:45.180394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:45.182074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:45.191969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:45.194437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:45.213858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1d3f0-b77a-4b00-b4a7-c654ab238223 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:45.222019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:45.223524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:45.224180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:45.224466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:45.226334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:45.228173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:45.244596 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2570/5125] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:45.245557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:45.257662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fe32-651d-45dc-8d18-2367707109a3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:45.265185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:45.266025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:45.266479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:45.266910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:45.267812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:45.269709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:45.284067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2571/5126] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:45.289431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:45.292814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3b0a-36ab-431d-bbbe-db98e51a63d9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:45.296878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df4507-be8f-41aa-b723-5527e5e1dadc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:45.298012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df4507-be8f-41aa-b723-5527e5e1dadc service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:45.298750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df4507-be8f-41aa-b723-5527e5e1dadc service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:45.299258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df4507-be8f-41aa-b723-5527e5e1dadc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:45.312672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df4507-be8f-41aa-b723-5527e5e1dadc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:45.314742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b483b-a336-4b92-a884-fd978a8571e2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:45.315111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df4507-be8f-41aa-b723-5527e5e1dadc service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:45.327516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:45.328003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2556/5127] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:45.330350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:45.330796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:45.331343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] PATH_INFO: `/v3/projects/cb56c9f0633c49d5932c58e5745bcf06/users/92c2bfd90e2d47759dfd0eb47c0f03d0/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:45.354201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb56c9f0633c49d5932c58e5745bcf06, user_id=92c2bfd90e2d47759dfd0eb47c0f03d0, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:45.357149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:45.392452 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2572/5128] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:45 2022] PUT /identity/v3/projects/cb56c9f0633c49d5932c58e5745bcf06/users/92c2bfd90e2d47759dfd0eb47c0f03d0/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:45.403774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d42e2-ce30-4647-908a-8265a03127d5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:45.404482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d42e2-ce30-4647-908a-8265a03127d5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:45.404983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d42e2-ce30-4647-908a-8265a03127d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:45.549006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f08d42e2-ce30-4647-908a-8265a03127d5 None None] MFA Rules not processed for user `92c2bfd90e2d47759dfd0eb47c0f03d0`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:45.550908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f08d42e2-ce30-4647-908a-8265a03127d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:45.714117 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2557/5129] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:45.718449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:45.910182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8762da-e037-4659-9b80-5c24a4b5b5e1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:45.926527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:45.930365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:45.930715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:45.934395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:45.935354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:45.940934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:46.037870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2573/5130] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:45 2022] POST /identity/v3/projects => generated 375 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:46.040369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d42e2-ce30-4647-908a-8265a03127d5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:46.063156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d42e2-ce30-4647-908a-8265a03127d5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:46.076858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:46.297259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:46.300762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:46.301285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:46.301704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:46.302560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:46.305045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:46.352097 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2558/5131] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:46 2022] POST /identity/v3/users => generated 294 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:46.356258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:46.385140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8e7d0-d0be-419e-a3ac-fb8998fe19d8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:46.404632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:46.405725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:46.406265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:46.407635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:46.420032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:46.423629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:46.448457 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2574/5132] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:46.452385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:46.502668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce655c56-7da3-4660-9558-cbb7e8e62f83 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:46.516513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:46.517491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:46.519076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:46.519513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] PATH_INFO: `/v3/projects/f4ba3f5ffde64bee8922f85919aa7bfd/users/57ae018d888d41f9856c9e6e0c18d9fb/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:46.556479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ba3f5ffde64bee8922f85919aa7bfd, user_id=57ae018d888d41f9856c9e6e0c18d9fb, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:46.559325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:46.595750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2559/5133] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:46 2022] PUT /identity/v3/projects/f4ba3f5ffde64bee8922f85919aa7bfd/users/57ae018d888d41f9856c9e6e0c18d9fb/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:46.600297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:46.629126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263183d-4dc8-41d3-9992-b136e5f87d6a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:46.637662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:46.639966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:46.837584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:46.838701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:46.839532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:46.840018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:46.841016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:46.843776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:46.847157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:46.860879 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2575/5134] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:46.870564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:46.882376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702a907-eccf-40e0-9e59-c59815f9e076 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:46.894441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:46.897382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:46.900147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:46.900530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] PATH_INFO: `/v3/projects/ec34a73b1f19424fa30b3f1702b80cb6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:46.905135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec34a73b1f19424fa30b3f1702b80cb6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:46.909721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:46.911567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dabd7-759e-459b-8305-7f1012a1b126 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:46.925025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891a86e-00bd-46f7-a350-5852d82d45a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:46.925783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a86e-00bd-46f7-a350-5852d82d45a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:46.926804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a86e-00bd-46f7-a350-5852d82d45a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:46.927379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a86e-00bd-46f7-a350-5852d82d45a9 admin admin] PATH_INFO: `/v3/projects/f4ba3f5ffde64bee8922f85919aa7bfd/users/57ae018d888d41f9856c9e6e0c18d9fb/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:46.932210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891a86e-00bd-46f7-a350-5852d82d45a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ba3f5ffde64bee8922f85919aa7bfd, user_id=57ae018d888d41f9856c9e6e0c18d9fb, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:46.955733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891a86e-00bd-46f7-a350-5852d82d45a9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:47.060834 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2576/5135] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:46 2022] PUT /identity/v3/projects/f4ba3f5ffde64bee8922f85919aa7bfd/users/57ae018d888d41f9856c9e6e0c18d9fb/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:47.065132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:47.070014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:47.070303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:47.071396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec34a73b1f19424fa30b3f1702b80cb6', 'request_id': 'req-0b0f40cf-1e8c-4154-80a5-25261c57487c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.101327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec34a73b1f19424fa30b3f1702b80cb6', 'request_id': 'req-0b0f40cf-1e8c-4154-80a5-25261c57487c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.124736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec34a73b1f19424fa30b3f1702b80cb6', 'request_id': 'req-0b0f40cf-1e8c-4154-80a5-25261c57487c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.175676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] The token cache is being invalidate because project ec34a73b1f19424fa30b3f1702b80cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:47.175986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.176717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.179089 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2560/5136] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:46 2022] DELETE /identity/v3/projects/ec34a73b1f19424fa30b3f1702b80cb6 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:47.187201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:47.190403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:47.195693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] MFA Rules not processed for user `57ae018d888d41f9856c9e6e0c18d9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:47.196931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:47.243028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:47.516436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f40cf-1e8c-4154-80a5-25261c57487c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:47.532655 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2577/5137] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:47.640584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:47.642928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:47.651965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:47.653105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:47.654741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:47.656758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] PATH_INFO: `/v3/users/0b6f6ccbceb04497bf50fead1235e59b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:47.660688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6f6ccbceb04497bf50fead1235e59b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:47.664358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:47.752455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6f6ccbceb04497bf50fead1235e59b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.765660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6f6ccbceb04497bf50fead1235e59b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.766947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6f6ccbceb04497bf50fead1235e59b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.782008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6f6ccbceb04497bf50fead1235e59b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.783299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf06ae-bbcf-41ce-b4f2-e26d8742856d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:47.783537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6f6ccbceb04497bf50fead1235e59b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:47.793110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e46dea-ea48-41c4-823b-388286c33579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:47.794143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e46dea-ea48-41c4-823b-388286c33579 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:47.794639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e46dea-ea48-41c4-823b-388286c33579 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:47.795141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e46dea-ea48-41c4-823b-388286c33579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:47.804081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-40e46dea-ea48-41c4-823b-388286c33579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:47.808891 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2561/5138] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:47 2022] DELETE /identity/v3/users/0b6f6ccbceb04497bf50fead1235e59b => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:47.832278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e46dea-ea48-41c4-823b-388286c33579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:47.835116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e46dea-ea48-41c4-823b-388286c33579 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:47.852336 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2578/5139] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:48.848947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:48.851068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:48.957517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:49.068101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffce60-7107-4609-ba7c-e52b7b510156 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.077376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.077667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.079930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.080416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] PATH_INFO: `/v3/projects/674182f91ce849dc8e9be93cdf3f5daa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.082978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] RBAC: Authorizing `identity:delete_project(project_id=674182f91ce849dc8e9be93cdf3f5daa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.091588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.206467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '674182f91ce849dc8e9be93cdf3f5daa', 'request_id': 'req-b10adbae-fe9a-42f1-9b89-2c8b88df424f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.215968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '674182f91ce849dc8e9be93cdf3f5daa', 'request_id': 'req-b10adbae-fe9a-42f1-9b89-2c8b88df424f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.230902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '674182f91ce849dc8e9be93cdf3f5daa', 'request_id': 'req-b10adbae-fe9a-42f1-9b89-2c8b88df424f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.252121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] The token cache is being invalidate because project 674182f91ce849dc8e9be93cdf3f5daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:49.252511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.253425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b10adbae-fe9a-42f1-9b89-2c8b88df424f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:49.258248 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2562/5140] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:48 2022] DELETE /identity/v3/projects/674182f91ce849dc8e9be93cdf3f5daa => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:49.263973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9252-ce3e-4e42-bc69-32b887c88fdb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.264491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9252-ce3e-4e42-bc69-32b887c88fdb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.264491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9252-ce3e-4e42-bc69-32b887c88fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.298770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d80c9252-ce3e-4e42-bc69-32b887c88fdb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:49.300828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d80c9252-ce3e-4e42-bc69-32b887c88fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:49.411583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614ab15-2baf-41aa-8c66-f19408ef14d5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.412033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614ab15-2baf-41aa-8c66-f19408ef14d5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.412859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614ab15-2baf-41aa-8c66-f19408ef14d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.413152 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2579/5141] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:49.456135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3614ab15-2baf-41aa-8c66-f19408ef14d5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:49.462695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3614ab15-2baf-41aa-8c66-f19408ef14d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:49.495763 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2563/5142] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:49.502168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c9252-ce3e-4e42-bc69-32b887c88fdb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.519181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c9252-ce3e-4e42-bc69-32b887c88fdb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.526461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.527713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.528036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.528334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.529143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:49.529369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.531343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.560182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:49.571016 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2580/5143] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:49.571399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614ab15-2baf-41aa-8c66-f19408ef14d5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.581797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614ab15-2baf-41aa-8c66-f19408ef14d5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.587219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.587919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.588205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.588482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.589138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.591322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.625915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2564/5144] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:49.630549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.643435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285351cb-6224-4973-92ac-b861cd82dd8d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.716987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.717710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.720121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.720466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.721655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.723693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.756620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.757867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2581/5145] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:49 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:49.768310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd475f6-23be-48a8-9709-725eed0cd9a9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.774550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.775673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.775673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.775878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.776557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.778429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.819614 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2565/5146] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:49.822980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.833500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23d470-f46d-45e2-a19c-9b7406857a76 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.847137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30590bc2-aa07-4670-97ca-9080d59a85bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.847960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590bc2-aa07-4670-97ca-9080d59a85bc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.848921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590bc2-aa07-4670-97ca-9080d59a85bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.849850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30590bc2-aa07-4670-97ca-9080d59a85bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.851584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30590bc2-aa07-4670-97ca-9080d59a85bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.853943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30590bc2-aa07-4670-97ca-9080d59a85bc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:49.888286 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2582/5147] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:49.888685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:49.902261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d172b-eaa5-4b2c-a5d8-1d64302f8a1f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:49.909158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:49.911729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:49.912829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:49.913486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] PATH_INFO: `/v3/projects/4497d5a8accf4bcbbf4cd8730be8ef7a/users/6f36ffe9de344df39d25a48042af5d74/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:49.951095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4497d5a8accf4bcbbf4cd8730be8ef7a, user_id=6f36ffe9de344df39d25a48042af5d74, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:49.955960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.000336 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2566/5148] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:49 2022] PUT /identity/v3/projects/4497d5a8accf4bcbbf4cd8730be8ef7a/users/6f36ffe9de344df39d25a48042af5d74/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:50.016620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca276b0-a5a1-4b7b-aeb9-08490806c0b9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.016620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca276b0-a5a1-4b7b-aeb9-08490806c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.016620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca276b0-a5a1-4b7b-aeb9-08490806c0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.078297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dca276b0-a5a1-4b7b-aeb9-08490806c0b9 None None] MFA Rules not processed for user `6f36ffe9de344df39d25a48042af5d74`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:50.080225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dca276b0-a5a1-4b7b-aeb9-08490806c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:50.180656 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2583/5149] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:50.184915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.298899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:50.311025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af3057-b058-4832-8748-695009e4b889 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.318177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.319740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.320468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.321958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.322979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.326313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.434027 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2567/5150] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/projects => generated 381 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:50.438229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca276b0-a5a1-4b7b-aeb9-08490806c0b9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.459390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca276b0-a5a1-4b7b-aeb9-08490806c0b9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.560540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.561221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.561514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.562100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.563556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.566175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.607141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:50.620243 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2584/5151] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:50 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:50.621629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.643892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00afc2c-eb7c-4757-8da9-689814ed43e6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.665932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.667560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.668030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.668552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.669680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.671861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.713055 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2568/5152] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:50.714253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.735330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e387ab-f9b1-4daa-a443-149cd79b4669 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.752054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.752888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.753750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.753750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] PATH_INFO: `/v3/projects/3b06c820208545c49cc46b1e4865d9c7/users/dbbb4d4873134f7da0b8f00b22bdcf47/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.777458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b06c820208545c49cc46b1e4865d9c7, user_id=dbbb4d4873134f7da0b8f00b22bdcf47, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.781910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.817511 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2585/5153] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:50 2022] PUT /identity/v3/projects/3b06c820208545c49cc46b1e4865d9c7/users/dbbb4d4873134f7da0b8f00b22bdcf47/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:50.818764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:50.834518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87b0e8-54bd-4c75-a79a-dff6d7c75f6d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:50.979344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:50.980085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:50.980525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:50.980939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:50.982160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:50.985092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:50.998505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.003858 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2569/5154] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:51.005005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.009605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.022447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603449d-60cb-41d5-a311-2554a08238f5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.029539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a792a-3013-45a2-9eeb-ad2f16be8196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.030561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a792a-3013-45a2-9eeb-ad2f16be8196 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.031182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a792a-3013-45a2-9eeb-ad2f16be8196 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.031698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a792a-3013-45a2-9eeb-ad2f16be8196 admin admin] PATH_INFO: `/v3/projects/3b06c820208545c49cc46b1e4865d9c7/users/dbbb4d4873134f7da0b8f00b22bdcf47/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.034500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a792a-3013-45a2-9eeb-ad2f16be8196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b06c820208545c49cc46b1e4865d9c7, user_id=dbbb4d4873134f7da0b8f00b22bdcf47, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.037978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a792a-3013-45a2-9eeb-ad2f16be8196 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.069252 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2570/5155] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:51 2022] PUT /identity/v3/projects/3b06c820208545c49cc46b1e4865d9c7/users/dbbb4d4873134f7da0b8f00b22bdcf47/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:51.069252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.069252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.069252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.113004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] MFA Rules not processed for user `dbbb4d4873134f7da0b8f00b22bdcf47`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:51.114906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.192787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8b972-c6c2-4b5c-ac86-0daea3bba32b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.218472 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2571/5156] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:51.234815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.237868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.298188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.298985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.299401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.299781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.301810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:51.331790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:51.355687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb65f5-56e0-4747-bf8a-8f43d2c1aa1f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.361482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.363084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.364200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.365172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.388584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.390149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.391994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.393445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.406340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2586/5157] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:51.413978 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2572/5158] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:06:51.871938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.883423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ce91d-0f30-4469-95fb-3435681cee22 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.894496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbbb4d4873134f7da0b8f00b22bdcf47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b06c820208545c49cc46b1e4865d9c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.895129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.895414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.895691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.896350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:51.903441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.905060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:51.932413 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2587/5159] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:51.933521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:51.947512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce4c31-3524-414c-9c38-f01842cea411 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:51.953616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbbb4d4873134f7da0b8f00b22bdcf47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b06c820208545c49cc46b1e4865d9c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:51.954681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:51.957020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:51.957488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:51.958328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:51.961272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.038109 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2573/5160] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:51 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:52.042591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.052752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.055672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:52.059737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860e3fb-f804-4d9e-a179-1c6fa4d9b0f5 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.125602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:52.230574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6a92d-e710-40d1-ad56-50e9d41680e3 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.232929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbbb4d4873134f7da0b8f00b22bdcf47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b06c820208545c49cc46b1e4865d9c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.234115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.234417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.234703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.235375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:52.235689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.237578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.239295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.240279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.241218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.241671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] PATH_INFO: `/v3/users/c8b015ae87f44d35be4d9e3a0f4d07d9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.242976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b015ae87f44d35be4d9e3a0f4d07d9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.246411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.257100 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2588/5161] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:06:52.340429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.346236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b015ae87f44d35be4d9e3a0f4d07d9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.353717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aa9e1-a68c-4562-8aa1-fa2c38f59541 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.354941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b015ae87f44d35be4d9e3a0f4d07d9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.355721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b015ae87f44d35be4d9e3a0f4d07d9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.374962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbbb4d4873134f7da0b8f00b22bdcf47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b06c820208545c49cc46b1e4865d9c7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.375801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.377047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b015ae87f44d35be4d9e3a0f4d07d9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.378013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.378592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9dcbfed0-e670-416b-a1aa-3c8ac32e6df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b015ae87f44d35be4d9e3a0f4d07d9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.378990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] PATH_INFO: `/v3/projects/e2fe6c0be16944508a993e5144221670` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.384109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e2fe6c0be16944508a993e5144221670)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:52.387819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:52.459903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2574/5162] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:52 2022] DELETE /identity/v3/users/c8b015ae87f44d35be4d9e3a0f4d07d9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:52.466463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.466790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.467030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:52.501138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:52.504861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:52.551305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2fe6c0be16944508a993e5144221670', 'request_id': 'req-9b316240-3eae-41ed-bac6-407b216243e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.566856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2fe6c0be16944508a993e5144221670', 'request_id': 'req-9b316240-3eae-41ed-bac6-407b216243e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.578648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:52.579620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2fe6c0be16944508a993e5144221670', 'request_id': 'req-9b316240-3eae-41ed-bac6-407b216243e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.607550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] The token cache is being invalidate because project e2fe6c0be16944508a993e5144221670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:52.607746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.609896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:52.616641 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2589/5163] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:52 2022] DELETE /identity/v3/projects/e2fe6c0be16944508a993e5144221670 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:52.618473 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2575/5164] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:52.669019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:52.673666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:52.877029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b316240-3eae-41ed-bac6-407b216243e7 tempest-VolumeQuotaClassesTest-1265204571 tempest-VolumeQuotaClassesTest-1265204571-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:52.888565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fdfc6-06d0-413c-b916-c917376aee9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:52.889580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fdfc6-06d0-413c-b916-c917376aee9e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:52.891679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fdfc6-06d0-413c-b916-c917376aee9e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:52.892373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fdfc6-06d0-413c-b916-c917376aee9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.029777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:53.031969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.057008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c0998-1087-4e33-b48b-c39eadef56b8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.093202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:53.093202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fdfc6-06d0-413c-b916-c917376aee9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.096270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fdfc6-06d0-413c-b916-c917376aee9e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.127273 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2590/5165] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:53.184301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.184949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.185244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.185530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] PATH_INFO: `/v3/users/6f36ffe9de344df39d25a48042af5d74` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.187633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f36ffe9de344df39d25a48042af5d74)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.190658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.282532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f36ffe9de344df39d25a48042af5d74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.295348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f36ffe9de344df39d25a48042af5d74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.300417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f36ffe9de344df39d25a48042af5d74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.322842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f36ffe9de344df39d25a48042af5d74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.323528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f36ffe9de344df39d25a48042af5d74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:53.367828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2576/5166] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:53 2022] DELETE /identity/v3/users/6f36ffe9de344df39d25a48042af5d74 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:53.372512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.373183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.373477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.399918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:53.401252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.489443 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2591/5167] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:53.532144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:53.535615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:53.654381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf173cbb-e50d-46b7-8fee-80957595d98f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:53.659137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:53.660553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:53.661088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:53.661550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:53.683685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:53.685163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:53.698731 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2577/5168] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:54.513631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.515223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:54.527592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7d32e-9fe2-4e39-b3b2-bb15301fdff2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.534728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.535608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.536123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.536450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] PATH_INFO: `/v3/projects/c06beec165b94160b79fb689bb7e0e0d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.538962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c06beec165b94160b79fb689bb7e0e0d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.541059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.615353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c06beec165b94160b79fb689bb7e0e0d', 'request_id': 'req-a1838def-3be7-4f78-bab4-cecdc3b66768'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.622492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c06beec165b94160b79fb689bb7e0e0d', 'request_id': 'req-a1838def-3be7-4f78-bab4-cecdc3b66768'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.629334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c06beec165b94160b79fb689bb7e0e0d', 'request_id': 'req-a1838def-3be7-4f78-bab4-cecdc3b66768'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.642711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] The token cache is being invalidate because project c06beec165b94160b79fb689bb7e0e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:54.643011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.643654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.645269 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2592/5169] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:54 2022] DELETE /identity/v3/projects/c06beec165b94160b79fb689bb7e0e0d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:06:54.650487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:54.655469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:54.661654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfed9-b943-4de0-9c5f-fb47d5d38cdd service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:54.728296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:54.728858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:54.729153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:54.729419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] PATH_INFO: `/v3/users/72827b24892a420fab1be776a91f3b66` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:54.730567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72827b24892a420fab1be776a91f3b66)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:54.732495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:54.783903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72827b24892a420fab1be776a91f3b66'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.796977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72827b24892a420fab1be776a91f3b66'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.797786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72827b24892a420fab1be776a91f3b66'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.807221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72827b24892a420fab1be776a91f3b66'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.808221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72827b24892a420fab1be776a91f3b66'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:54.834317 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2578/5170] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:54 2022] DELETE /identity/v3/users/72827b24892a420fab1be776a91f3b66 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:55.340137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.342162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:55.366767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.368292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:55.391379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1838def-3be7-4f78-bab4-cecdc3b66768 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.499620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972383d-fc63-4a9e-bd96-0ec332425952 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.499985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.500636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.500921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.501199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] PATH_INFO: `/v3/projects/9086698824a3457b8bdbd9f1cc4ddaaf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.502388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9086698824a3457b8bdbd9f1cc4ddaaf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.504551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.505028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.506558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.506842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.507116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] PATH_INFO: `/v3/projects/4497d5a8accf4bcbbf4cd8730be8ef7a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.508993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4497d5a8accf4bcbbf4cd8730be8ef7a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.511190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.536507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:55.567083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:55.595215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9086698824a3457b8bdbd9f1cc4ddaaf', 'request_id': 'req-28d3e30d-dc46-4028-8b3c-9b70255d7adc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.606178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9086698824a3457b8bdbd9f1cc4ddaaf', 'request_id': 'req-28d3e30d-dc46-4028-8b3c-9b70255d7adc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.609739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4497d5a8accf4bcbbf4cd8730be8ef7a', 'request_id': 'req-ee625baf-0b1d-48f4-a842-56b9271a4296'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.622172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9086698824a3457b8bdbd9f1cc4ddaaf', 'request_id': 'req-28d3e30d-dc46-4028-8b3c-9b70255d7adc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.630013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4497d5a8accf4bcbbf4cd8730be8ef7a', 'request_id': 'req-ee625baf-0b1d-48f4-a842-56b9271a4296'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.641633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] The token cache is being invalidate because project 9086698824a3457b8bdbd9f1cc4ddaaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:55.641959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.650191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28d3e30d-dc46-4028-8b3c-9b70255d7adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.650191 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2593/5171] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:55 2022] DELETE /identity/v3/projects/9086698824a3457b8bdbd9f1cc4ddaaf => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:55.654640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6189e4-31cd-4c9e-9e2c-f7f72ef25178 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.654885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4497d5a8accf4bcbbf4cd8730be8ef7a', 'request_id': 'req-ee625baf-0b1d-48f4-a842-56b9271a4296'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.655342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6189e4-31cd-4c9e-9e2c-f7f72ef25178 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.655342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6189e4-31cd-4c9e-9e2c-f7f72ef25178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.671597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] The token cache is being invalidate because project 4497d5a8accf4bcbbf4cd8730be8ef7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:55.672330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.674221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.678157 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2579/5172] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:55 2022] DELETE /identity/v3/projects/4497d5a8accf4bcbbf4cd8730be8ef7a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:55.680035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cd6189e4-31cd-4c9e-9e2c-f7f72ef25178 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:55.681106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.683021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cd6189e4-31cd-4c9e-9e2c-f7f72ef25178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:55.685422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:55.835327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee625baf-0b1d-48f4-a842-56b9271a4296 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.845185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2594/5173] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:55.854792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f584-0a90-4f32-9b18-38f161417eb0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.855209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f584-0a90-4f32-9b18-38f161417eb0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.855341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f584-0a90-4f32-9b18-38f161417eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.871854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.874329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.874329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.874329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] PATH_INFO: `/v3/users/dbbb4d4873134f7da0b8f00b22bdcf47` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.876495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbbb4d4873134f7da0b8f00b22bdcf47)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.878599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.891507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3ec9f584-0a90-4f32-9b18-38f161417eb0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:55.893248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3ec9f584-0a90-4f32-9b18-38f161417eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:55.913938 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2595/5174] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:55.915444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9f584-0a90-4f32-9b18-38f161417eb0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:55.934794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9f584-0a90-4f32-9b18-38f161417eb0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:55.943162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:55.947239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:55.947705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbbb4d4873134f7da0b8f00b22bdcf47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.948661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:55.949376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:55.954621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:55.955675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:55.960770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:55.967835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbbb4d4873134f7da0b8f00b22bdcf47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.968776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbbb4d4873134f7da0b8f00b22bdcf47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.995123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbbb4d4873134f7da0b8f00b22bdcf47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:55.996083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbbb4d4873134f7da0b8f00b22bdcf47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:56.024055 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2580/5175] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:06:55 2022] DELETE /identity/v3/users/dbbb4d4873134f7da0b8f00b22bdcf47 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:56.037574 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2596/5176] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:56.039698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:56.041169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:56.090369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:56.159614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0834a8-497c-4e28-b385-d158d3451da9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:56.169155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:56.170914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:56.171321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:56.171692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:56.172740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:56.175235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:56.231248 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2581/5177] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:56 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:56.235568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:56.252625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588617f0-aa7d-400d-9595-d7f9ef4516dc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:56.336943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:56.337375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:56.337694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:56.338666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:56.339320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:56.341365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:56.383279 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2597/5178] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:56 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:56.387116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:56.402525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eddc85-4e27-45eb-8d45-dc54213f8c91 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:56.408574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:56.409436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:56.409819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:56.410703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:56.411578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:56.414516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:56.436976 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2582/5179] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:56.440661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:56.454252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e25ed-2834-447f-b618-05ea25169b3a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:56.460959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348ed72-29cb-4b79-b1fc-e54f6c83a4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:56.461590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348ed72-29cb-4b79-b1fc-e54f6c83a4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:56.462525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348ed72-29cb-4b79-b1fc-e54f6c83a4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:56.462817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348ed72-29cb-4b79-b1fc-e54f6c83a4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:56.463480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348ed72-29cb-4b79-b1fc-e54f6c83a4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:56.465533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348ed72-29cb-4b79-b1fc-e54f6c83a4c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:56.480440 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2598/5180] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:56.484485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:56.496503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3efb8c-04be-41c0-ab05-c7269ff5db06 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:56.502255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:56.503318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:56.503604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:56.503877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] PATH_INFO: `/v3/projects/e1fff93100b14736adda4047c374baf1/users/672f5df39780430fa7914a747984531c/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:56.522341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1fff93100b14736adda4047c374baf1, user_id=672f5df39780430fa7914a747984531c, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:56.525424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:56.554147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2583/5181] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:56 2022] PUT /identity/v3/projects/e1fff93100b14736adda4047c374baf1/users/672f5df39780430fa7914a747984531c/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:56.556310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20904c6-7104-48b3-a2f2-4b3152fe78a3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:56.556789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20904c6-7104-48b3-a2f2-4b3152fe78a3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:56.556874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20904c6-7104-48b3-a2f2-4b3152fe78a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:56.574648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:56.596290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f20904c6-7104-48b3-a2f2-4b3152fe78a3 None None] MFA Rules not processed for user `672f5df39780430fa7914a747984531c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:56.597885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f20904c6-7104-48b3-a2f2-4b3152fe78a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:56.705902 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2599/5182] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:56.709419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:56.838276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f17f-d242-47b1-a5b7-97e8c4ae0760 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:56.851278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:56.851922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:56.852203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:56.852478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:56.853132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:56.861063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:56.935725 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2584/5183] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:56 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:56.940409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20904c6-7104-48b3-a2f2-4b3152fe78a3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:56.958605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20904c6-7104-48b3-a2f2-4b3152fe78a3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:57.058762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:57.059104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.059201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.059467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:57.061092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:57.063554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:57.104419 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2600/5184] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:56 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:06:57.108234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:57.124544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a0e72-f7ac-4102-9547-4898b9d14d9c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:57.131950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:57.133659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.134234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.134669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:57.135590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:57.140350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:57.162083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2585/5185] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:57.164768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:57.189242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b784f-719c-4739-a449-51dfef3ea2a7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:57.197267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7c08c-23ae-45a9-9a29-907ad0b426f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:57.198642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7c08c-23ae-45a9-9a29-907ad0b426f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.199077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7c08c-23ae-45a9-9a29-907ad0b426f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.199455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7c08c-23ae-45a9-9a29-907ad0b426f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:57.200264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7c08c-23ae-45a9-9a29-907ad0b426f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:57.205047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7c08c-23ae-45a9-9a29-907ad0b426f6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:57.233813 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2601/5186] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:57.235100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:57.250861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9135e20-981b-443b-9181-e4466ed3fd56 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:57.268419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:57.270078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.270441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.270817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] PATH_INFO: `/v3/projects/ed8a932cb6934b6db39eddf8f4d5fb78/users/7147017275fa49d3bd6b0124d12f5c5c/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:57.302314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8a932cb6934b6db39eddf8f4d5fb78, user_id=7147017275fa49d3bd6b0124d12f5c5c, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:57.323962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:57.352806 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2586/5187] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:57 2022] PUT /identity/v3/projects/ed8a932cb6934b6db39eddf8f4d5fb78/users/7147017275fa49d3bd6b0124d12f5c5c/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:57.363709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064173d2-4994-4cb2-8efe-bc020a917d21 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.364469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064173d2-4994-4cb2-8efe-bc020a917d21 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.364976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064173d2-4994-4cb2-8efe-bc020a917d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:57.442691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-064173d2-4994-4cb2-8efe-bc020a917d21 None None] MFA Rules not processed for user `7147017275fa49d3bd6b0124d12f5c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:57.445584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-064173d2-4994-4cb2-8efe-bc020a917d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:57.549633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:57.554187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:57.661308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:57.685652 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2602/5188] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:57.690785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064173d2-4994-4cb2-8efe-bc020a917d21 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:57.700407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:57.738756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fd477-5529-4d04-9a46-1017e93564e2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:57.753298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:57.754190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.754495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.754749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] PATH_INFO: `/v3/projects/3b06c820208545c49cc46b1e4865d9c7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:57.756981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b06c820208545c49cc46b1e4865d9c7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:57.759026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:57.797925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064173d2-4994-4cb2-8efe-bc020a917d21 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:57.819149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:57.820008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.823894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.826506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:57.827183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:57.866207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:57.896969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b06c820208545c49cc46b1e4865d9c7', 'request_id': 'req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:57.913727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b06c820208545c49cc46b1e4865d9c7', 'request_id': 'req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:57.915170 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2603/5189] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:57 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:57.931091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:57.936389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b06c820208545c49cc46b1e4865d9c7', 'request_id': 'req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:57.969361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] The token cache is being invalidate because project 3b06c820208545c49cc46b1e4865d9c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:06:57.970395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:57.971446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b03fbe47-6738-4b1b-ac6f-5a955a43a9cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:06:57.973970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2587/5190] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:06:57 2022] DELETE /identity/v3/projects/3b06c820208545c49cc46b1e4865d9c7 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:57.988650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618762f9-5c27-4727-bf32-ccfab6530fdc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:57.990981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618762f9-5c27-4727-bf32-ccfab6530fdc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:57.991538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618762f9-5c27-4727-bf32-ccfab6530fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.025651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-618762f9-5c27-4727-bf32-ccfab6530fdc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:58.027856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-618762f9-5c27-4727-bf32-ccfab6530fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:58.092264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc11da-eea6-416b-b14b-292a56ddd962 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.105790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.106788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.109362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.109675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.110386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.116103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.133537 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2588/5191] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:58.140795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bedcaa-e19a-4e70-9c27-405e2a165936 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.141083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bedcaa-e19a-4e70-9c27-405e2a165936 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.141358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bedcaa-e19a-4e70-9c27-405e2a165936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.160934 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2604/5192] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:57 2022] POST /identity/v3/users => generated 299 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:58.166012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.168427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-17bedcaa-e19a-4e70-9c27-405e2a165936 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:58.169765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-17bedcaa-e19a-4e70-9c27-405e2a165936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:58.171778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:58.193851 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2589/5193] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:58.194982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bedcaa-e19a-4e70-9c27-405e2a165936 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.208064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41734-3820-460f-b6a4-d78fa37419c6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.208193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bedcaa-e19a-4e70-9c27-405e2a165936 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.215812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.216522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.216809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.217106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.218101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:06:58.218403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.219273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.220065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.220479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.220899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.221272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.222176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.228700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.238191 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2590/5194] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:06:58.240424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.261135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f026abe-5426-4c0c-925c-dc9ad469c8d4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.263794 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2605/5195] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:58.264823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.287169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e450295-d401-481a-8ab5-9481225f5654 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.289343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.292691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.294911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.294911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.299029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.299830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.302309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.302309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] PATH_INFO: `/v3/projects/af44d369d3f048aaada7923ccf3c705f/users/fcac5ca36d8a421e83503f70f84d52d5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.309746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.311776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.338472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af44d369d3f048aaada7923ccf3c705f, user_id=fcac5ca36d8a421e83503f70f84d52d5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.341275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.376853 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2606/5196] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:58 2022] PUT /identity/v3/projects/af44d369d3f048aaada7923ccf3c705f/users/fcac5ca36d8a421e83503f70f84d52d5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:58.383459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.398359 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2591/5197] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:58 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:58.402273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.455782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:58.494813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:58.519803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4d7be-f0e1-4c63-98d9-4dbf02ff3494 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.556805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c059542-b645-41e7-8e08-be29c82d7d47 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.563724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.564420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.564709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.564982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.565621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.567604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.602487 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2592/5198] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:58 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:58.606146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.621279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c706337-4fc5-40d3-b350-5bba908973aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.623871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.625704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.626070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.626442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.627097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.628232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.628734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.628852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.629108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.629383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.630064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.632684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.645979 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2593/5199] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:58.649728 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2607/5200] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:58.650197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.651037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.666314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c4a1-ea6a-4898-b8b0-f1aa36d8caa7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.667019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f6f87-6e35-45d2-b740-d8d91db6be3b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.671436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d718893-a88d-4b5f-b7a8-caaf3358940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.672106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d718893-a88d-4b5f-b7a8-caaf3358940c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.672225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.672462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d718893-a88d-4b5f-b7a8-caaf3358940c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.672659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d718893-a88d-4b5f-b7a8-caaf3358940c admin admin] PATH_INFO: `/v3/projects/af44d369d3f048aaada7923ccf3c705f/users/fcac5ca36d8a421e83503f70f84d52d5/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.672778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.673059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.673331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.674191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.675298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d718893-a88d-4b5f-b7a8-caaf3358940c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af44d369d3f048aaada7923ccf3c705f, user_id=fcac5ca36d8a421e83503f70f84d52d5, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.676012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.679584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d718893-a88d-4b5f-b7a8-caaf3358940c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.691867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2594/5201] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:06:58.705445 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2608/5202] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:58 2022] PUT /identity/v3/projects/af44d369d3f048aaada7923ccf3c705f/users/fcac5ca36d8a421e83503f70f84d52d5/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:58.706028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.707235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.707526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.707801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.718351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6182c52-5ab5-4f3b-b9c4-15fc72bc3d71 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:58.783872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] MFA Rules not processed for user `fcac5ca36d8a421e83503f70f84d52d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:58.784718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:58.815708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dac907-7be8-4812-82f7-6c8ee9fdd299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:58.816512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dac907-7be8-4812-82f7-6c8ee9fdd299 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.817443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dac907-7be8-4812-82f7-6c8ee9fdd299 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.817800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dac907-7be8-4812-82f7-6c8ee9fdd299 admin admin] PATH_INFO: `/v3/projects/fd4e29e9bf244d94a370c52fd84ad58c/users/18c270038b64446b9f9640c1c32bfb09/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.850458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dac907-7be8-4812-82f7-6c8ee9fdd299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4e29e9bf244d94a370c52fd84ad58c, user_id=18c270038b64446b9f9640c1c32bfb09, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:58.855139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dac907-7be8-4812-82f7-6c8ee9fdd299 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:58.880504 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2595/5203] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:58 2022] PUT /identity/v3/projects/fd4e29e9bf244d94a370c52fd84ad58c/users/18c270038b64446b9f9640c1c32bfb09/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:58.892622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9ffd-fe95-438d-bfd9-496d00007ab2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:58.894438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9ffd-fe95-438d-bfd9-496d00007ab2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:58.894892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9ffd-fe95-438d-bfd9-496d00007ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:58.920721 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2609/5204] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:06:58.937102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:58.939901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:58.954479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ce2f9ffd-fe95-438d-bfd9-496d00007ab2 None None] MFA Rules not processed for user `18c270038b64446b9f9640c1c32bfb09`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:58.956179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ce2f9ffd-fe95-438d-bfd9-496d00007ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:59.049666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:59.062403 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2596/5205] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:06:59.063785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9ffd-fe95-438d-bfd9-496d00007ab2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:59.088918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228999-ae31-439c-b0c7-4383e29fb952 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:59.125614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:59.224611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9ffd-fe95-438d-bfd9-496d00007ab2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:59.231203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:59.231896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:59.232296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:59.233134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:59.237172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:59.238054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:59.238345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:59.238810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:59.239373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:59.245376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:59.318389 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2597/5206] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:59 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:59.322627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:59.372024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:59.374033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:59.387837 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2610/5207] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:06:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:06:59.408295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465440b2-3c64-4f9b-88f0-2a55e281eb25 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:59.414785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:59.415403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:59.415709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:59.416005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:59.416664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:59.418788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:59.459724 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2598/5208] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:06:59 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:06:59.460664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:59.473500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253704-1792-41db-8192-3f07fec05fc0 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:59.482181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:59.482999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:59.483391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:59.483738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:59.484860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:59.487419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:59.503944 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2611/5209] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:59.508087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:59.520708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3e7be-0045-4c9f-bd86-56779634541f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:59.532164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea5d4-b4ed-4f6c-9cc3-62d20f24e701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:59.532978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea5d4-b4ed-4f6c-9cc3-62d20f24e701 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:59.533277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea5d4-b4ed-4f6c-9cc3-62d20f24e701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:59.533678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea5d4-b4ed-4f6c-9cc3-62d20f24e701 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:59.534626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea5d4-b4ed-4f6c-9cc3-62d20f24e701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:59.536595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea5d4-b4ed-4f6c-9cc3-62d20f24e701 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:59.554541 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2599/5210] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:06:59.558970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:59.593977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00bd6b-8493-47ae-85b0-54fe8b547ec8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:06:59.600959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:06:59.602468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:59.602991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:59.603683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] PATH_INFO: `/v3/projects/03b74783b0354de79b0ea4deec3fbdc2/users/994b0d1abf2c471b9b016d8c1b838309/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:59.627129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b74783b0354de79b0ea4deec3fbdc2, user_id=994b0d1abf2c471b9b016d8c1b838309, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:06:59.631205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:06:59.650707 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2612/5211] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:06:59 2022] PUT /identity/v3/projects/03b74783b0354de79b0ea4deec3fbdc2/users/994b0d1abf2c471b9b016d8c1b838309/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:06:59.659988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4380a-1f28-4f9f-a394-f6df06b83eba None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:06:59.660279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4380a-1f28-4f9f-a394-f6df06b83eba None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:06:59.660559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4380a-1f28-4f9f-a394-f6df06b83eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:06:59.700907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4de4380a-1f28-4f9f-a394-f6df06b83eba None None] MFA Rules not processed for user `994b0d1abf2c471b9b016d8c1b838309`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:06:59.702326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4de4380a-1f28-4f9f-a394-f6df06b83eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:06:59.863431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:06:59.900087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:06:59.900628 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2600/5212] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:00.067464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf1fa7-826c-4a41-be04-2526c0d815ad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.078875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.079986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.080218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.080677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.081530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.091517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.200556 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2613/5213] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:06:59 2022] POST /identity/v3/projects => generated 379 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:00.202162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4380a-1f28-4f9f-a394-f6df06b83eba None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.219425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4380a-1f28-4f9f-a394-f6df06b83eba None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.409293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.410189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.410562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.410932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.411666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.414818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.456218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2601/5214] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:00 2022] POST /identity/v3/users => generated 296 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:00.469893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.487248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a838-b9da-4321-b554-a1dde36670c6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.494965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.495712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.496095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.496468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.497578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.499572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.530010 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2614/5215] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:00.534587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.556476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50766f6-6a0a-4623-8243-68a53d2675e4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.565704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.566661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.567226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.567735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] PATH_INFO: `/v3/projects/a70fbb8e8c8444c5aeb8545753531c36/users/5f573117034646d18e9f5c4994c87082/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.608197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70fbb8e8c8444c5aeb8545753531c36, user_id=5f573117034646d18e9f5c4994c87082, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.611337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.659271 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2602/5216] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:00 2022] PUT /identity/v3/projects/a70fbb8e8c8444c5aeb8545753531c36/users/5f573117034646d18e9f5c4994c87082/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:00.661089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.669142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:00.677212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.684161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:00.685572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af81aeb-e16f-4ecd-947c-3f001c1ef3c8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.783869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.785177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.785625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.786081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.789190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.791748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.793882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26443407-73ca-4746-af6f-18b01f83cf39 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.804348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.805102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.805455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.805802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.808038 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2615/5217] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:00.811229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:00.811926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.828187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332aa18-f7f4-4bbd-b212-57a600586684 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:00.833579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34331eb6-c89a-4712-98fe-ebba7456ce78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:00.834880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34331eb6-c89a-4712-98fe-ebba7456ce78 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.835266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34331eb6-c89a-4712-98fe-ebba7456ce78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.835613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34331eb6-c89a-4712-98fe-ebba7456ce78 admin admin] PATH_INFO: `/v3/projects/a70fbb8e8c8444c5aeb8545753531c36/users/5f573117034646d18e9f5c4994c87082/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.837992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34331eb6-c89a-4712-98fe-ebba7456ce78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70fbb8e8c8444c5aeb8545753531c36, user_id=5f573117034646d18e9f5c4994c87082, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.841110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34331eb6-c89a-4712-98fe-ebba7456ce78 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.854465 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2616/5218] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:00 2022] PUT /identity/v3/projects/a70fbb8e8c8444c5aeb8545753531c36/users/5f573117034646d18e9f5c4994c87082/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:00.864059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6193905-8cab-4c73-a2a2-2aab6da6c72a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:00.864352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6193905-8cab-4c73-a2a2-2aab6da6c72a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:00.864630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6193905-8cab-4c73-a2a2-2aab6da6c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:00.886982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:00.888520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:00.900472 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2603/5219] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:00.913301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c6193905-8cab-4c73-a2a2-2aab6da6c72a None None] MFA Rules not processed for user `5f573117034646d18e9f5c4994c87082`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:00.914735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6193905-8cab-4c73-a2a2-2aab6da6c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:00.924201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:00.926325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:00.960667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:01.055664 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2617/5220] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:01.071142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6193905-8cab-4c73-a2a2-2aab6da6c72a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.127831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052c851-3710-4666-8b41-178f85109287 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.258701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6193905-8cab-4c73-a2a2-2aab6da6c72a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.267018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.267653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.267940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.268218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.289355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.290945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.309011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb79ad-57c9-49cf-b25c-59012ae5221f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.309655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb79ad-57c9-49cf-b25c-59012ae5221f service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.309972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb79ad-57c9-49cf-b25c-59012ae5221f service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.310253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb79ad-57c9-49cf-b25c-59012ae5221f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.312239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-84fb79ad-57c9-49cf-b25c-59012ae5221f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:01.322674 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2618/5221] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:01.337146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb79ad-57c9-49cf-b25c-59012ae5221f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.339009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb79ad-57c9-49cf-b25c-59012ae5221f service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.372208 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2604/5222] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:01.444053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:01.455939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911657-058c-4a75-8558-720c26243a92 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:01.469671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a241ab-55cd-4367-a1eb-ed322fc3db25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:01.471395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a241ab-55cd-4367-a1eb-ed322fc3db25 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:01.471920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a241ab-55cd-4367-a1eb-ed322fc3db25 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:01.472213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a241ab-55cd-4367-a1eb-ed322fc3db25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:01.539260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:01.640512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a241ab-55cd-4367-a1eb-ed322fc3db25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:01.642442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a241ab-55cd-4367-a1eb-ed322fc3db25 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:01.655287 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2619/5223] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:02.221267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:02.221873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:02.222366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:02.264440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:02.267815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:02.405093 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2605/5224] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:03.719736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403d36-08ce-49e0-a477-ddcb49fc8e9a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:03.720427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403d36-08ce-49e0-a477-ddcb49fc8e9a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:03.721165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc403d36-08ce-49e0-a477-ddcb49fc8e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:03.746630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cc403d36-08ce-49e0-a477-ddcb49fc8e9a None None] MFA Rules not processed for user `9546792be5a640e2a59fb2e8481c6c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:03.747862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc403d36-08ce-49e0-a477-ddcb49fc8e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:03.878847 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2620/5225] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:03.919491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:03.922457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:04.098665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:04.193891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0a6e2-42a8-4ad4-a253-b18b9a1297e8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:04.215529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57ae018d888d41f9856c9e6e0c18d9fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4ba3f5ffde64bee8922f85919aa7bfd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:04.218502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.220313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.226931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.227824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:04.228907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:04.233630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:04.281120 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2606/5226] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:04.291257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403d36-08ce-49e0-a477-ddcb49fc8e9a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:04.326835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc403d36-08ce-49e0-a477-ddcb49fc8e9a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:04.352665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af253172-200f-4af4-b7ba-69e2d4cc1191 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57ae018d888d41f9856c9e6e0c18d9fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4ba3f5ffde64bee8922f85919aa7bfd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:04.361159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af253172-200f-4af4-b7ba-69e2d4cc1191 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.361585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af253172-200f-4af4-b7ba-69e2d4cc1191 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.361902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af253172-200f-4af4-b7ba-69e2d4cc1191 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.362580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af253172-200f-4af4-b7ba-69e2d4cc1191 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:04.364555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af253172-200f-4af4-b7ba-69e2d4cc1191 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:04.477411 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2621/5227] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:04 2022] POST /identity/v3/projects => generated 348 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:04.479471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:04.503476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b32e0-42ec-4829-9254-1a954666b9f9 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:04.559086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d354e-e896-4f85-9541-31d9e7637636 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.559653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d354e-e896-4f85-9541-31d9e7637636 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.560839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d354e-e896-4f85-9541-31d9e7637636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.638516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-178d354e-e896-4f85-9541-31d9e7637636 None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:04.640207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-178d354e-e896-4f85-9541-31d9e7637636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:04.735418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57ae018d888d41f9856c9e6e0c18d9fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4ba3f5ffde64bee8922f85919aa7bfd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:04.736243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:04.736527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:04.736869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:04.737597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:04.737950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:04.743264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:04.778847 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2607/5228] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:07:04.820760 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2622/5229] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:05.458500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.525369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d354e-e896-4f85-9541-31d9e7637636 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.539852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d354e-e896-4f85-9541-31d9e7637636 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.552595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57ae018d888d41f9856c9e6e0c18d9fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4ba3f5ffde64bee8922f85919aa7bfd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.553569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.554074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.554492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] PATH_INFO: `/v3/projects/32cca9349a1d45388ac490c4d4a840d2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.560609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorizing `identity:delete_project(project_id=32cca9349a1d45388ac490c4d4a840d2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.565297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.576918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4876b1-552d-47f9-ab1b-58be62ddff23 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:05.581709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:05.582400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.582686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.583013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] PATH_INFO: `/v3/users/18c270038b64446b9f9640c1c32bfb09` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.585175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c270038b64446b9f9640c1c32bfb09)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:05.587249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:05.697862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c270038b64446b9f9640c1c32bfb09'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.707674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:05.717582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c270038b64446b9f9640c1c32bfb09'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.718477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c270038b64446b9f9640c1c32bfb09'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.727890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32cca9349a1d45388ac490c4d4a840d2', 'request_id': 'req-caf67b8d-4f27-4643-bee3-e693279a7b60'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.737639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32cca9349a1d45388ac490c4d4a840d2', 'request_id': 'req-caf67b8d-4f27-4643-bee3-e693279a7b60'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.741269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c270038b64446b9f9640c1c32bfb09'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.742559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b9dc9eb-c20d-4c1f-9fc3-ba321fd5dee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c270038b64446b9f9640c1c32bfb09'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.764859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32cca9349a1d45388ac490c4d4a840d2', 'request_id': 'req-caf67b8d-4f27-4643-bee3-e693279a7b60'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.776616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:05.785139 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2608/5230] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:05 2022] DELETE /identity/v3/users/18c270038b64446b9f9640c1c32bfb09 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:05.794799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980d137-d4b1-4cf0-b10d-951f04329478 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:05.795802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] The token cache is being invalidate because project 32cca9349a1d45388ac490c4d4a840d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:05.796265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980d137-d4b1-4cf0-b10d-951f04329478 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:05.796701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.797553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980d137-d4b1-4cf0-b10d-951f04329478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:05.799613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:05.803984 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2623/5231] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:05 2022] DELETE /identity/v3/projects/32cca9349a1d45388ac490c4d4a840d2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:05.820922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:05.823907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:05.853915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6980d137-d4b1-4cf0-b10d-951f04329478 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:05.860106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6980d137-d4b1-4cf0-b10d-951f04329478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:05.970215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf67b8d-4f27-4643-bee3-e693279a7b60 tempest-RoutersIpV6AdminTest-382391363 tempest-RoutersIpV6AdminTest-382391363-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:06.042614 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2609/5232] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:06.053861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980d137-d4b1-4cf0-b10d-951f04329478 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:06.093335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:06.093991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.094277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.094553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.098385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:06.219296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980d137-d4b1-4cf0-b10d-951f04329478 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:06.238944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:06.239755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.240083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.240361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.259717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:06.261197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.287867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2610/5233] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:06.289628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:06.300872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.320949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:06.324351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:06.328753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2624/5234] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:06.420552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae2fc9-b78c-437a-8019-7789eda00528 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:06.424823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:06.425503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:06.425932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:06.426302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:06.429591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:06.542974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:06.546260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:06.548128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:06.563378 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2611/5235] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:07.709387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:07.711010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:07.730785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c78d9-2c0c-4d12-a8f5-9035c858d1c7 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:07.738851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:07.740688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:07.741136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:07.741570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] PATH_INFO: `/v3/projects/fd4e29e9bf244d94a370c52fd84ad58c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:07.743677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4e29e9bf244d94a370c52fd84ad58c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:07.745857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:07.830307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:07.864259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4e29e9bf244d94a370c52fd84ad58c', 'request_id': 'req-49234abd-fe53-4d18-81a5-81491685176b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.876287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4e29e9bf244d94a370c52fd84ad58c', 'request_id': 'req-49234abd-fe53-4d18-81a5-81491685176b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.886143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4e29e9bf244d94a370c52fd84ad58c', 'request_id': 'req-49234abd-fe53-4d18-81a5-81491685176b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.911268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] The token cache is being invalidate because project fd4e29e9bf244d94a370c52fd84ad58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:07.912468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.914852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:07.921302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2625/5236] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:07 2022] DELETE /identity/v3/projects/fd4e29e9bf244d94a370c52fd84ad58c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:07.921976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:07.946576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f58e4d-80d2-4fd2-9168-f870eb5ac594 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:08.125392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:08.126102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:08.127039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:08.127335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] PATH_INFO: `/v3/users/994b0d1abf2c471b9b016d8c1b838309` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:08.133401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994b0d1abf2c471b9b016d8c1b838309)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:08.135435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:08.336126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994b0d1abf2c471b9b016d8c1b838309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.361261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994b0d1abf2c471b9b016d8c1b838309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.363537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994b0d1abf2c471b9b016d8c1b838309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.387648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994b0d1abf2c471b9b016d8c1b838309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.388462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994b0d1abf2c471b9b016d8c1b838309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:08.449528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2612/5237] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:07 2022] DELETE /identity/v3/users/994b0d1abf2c471b9b016d8c1b838309 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:09.900479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:09.902270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:09.919480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234abd-fe53-4d18-81a5-81491685176b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:10.087438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:10.088098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.088384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.088673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] PATH_INFO: `/v3/projects/03b74783b0354de79b0ea4deec3fbdc2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.090526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b74783b0354de79b0ea4deec3fbdc2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:10.092690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:10.179685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:10.181802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.210145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb463f0-953a-43f9-933d-e1ce95dd78e5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:10.220986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b74783b0354de79b0ea4deec3fbdc2', 'request_id': 'req-c6390ced-c474-467f-868c-278641c71f82'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.240863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b74783b0354de79b0ea4deec3fbdc2', 'request_id': 'req-c6390ced-c474-467f-868c-278641c71f82'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.261673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b74783b0354de79b0ea4deec3fbdc2', 'request_id': 'req-c6390ced-c474-467f-868c-278641c71f82'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.294268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] The token cache is being invalidate because project 03b74783b0354de79b0ea4deec3fbdc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:10.294574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.298171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.304189 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2626/5238] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:09 2022] DELETE /identity/v3/projects/03b74783b0354de79b0ea4deec3fbdc2 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:10.310082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:10.313668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.426739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:10.428942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:10.431798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.433294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.433797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] PATH_INFO: `/v3/users/672f5df39780430fa7914a747984531c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.436447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=672f5df39780430fa7914a747984531c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:10.439392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:10.451359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6390ced-c474-467f-868c-278641c71f82 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:10.459078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:10.460015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.460530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.460888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] PATH_INFO: `/v3/users/5f573117034646d18e9f5c4994c87082` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.462430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f573117034646d18e9f5c4994c87082)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:10.464873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:10.481193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:10.571201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672f5df39780430fa7914a747984531c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.583301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f573117034646d18e9f5c4994c87082'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.595363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672f5df39780430fa7914a747984531c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.597108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672f5df39780430fa7914a747984531c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.622982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672f5df39780430fa7914a747984531c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.623680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d94152e1-d0ec-4fb1-910a-3465d263a538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672f5df39780430fa7914a747984531c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.629779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f573117034646d18e9f5c4994c87082'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.656326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f573117034646d18e9f5c4994c87082'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.663638 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2613/5239] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:10 2022] DELETE /identity/v3/users/672f5df39780430fa7914a747984531c => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:10.668304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.668600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.668878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.684788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f573117034646d18e9f5c4994c87082'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.686029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f573117034646d18e9f5c4994c87082'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.700459 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2627/5240] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:10 2022] DELETE /identity/v3/users/5f573117034646d18e9f5c4994c87082 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:10.705059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:10.706949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.735713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:10.737553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.835161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ed6b-a053-4ef6-92ed-3af0cb9c00cf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:10.844544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:10.846648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:10.847193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:10.847691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] PATH_INFO: `/v3/users/d57829f24d3841eb9c1d6f5c6a5ea7ee` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:10.850292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57829f24d3841eb9c1d6f5c6a5ea7ee)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:10.854517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:10.870142 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2614/5241] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:10.906968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:10.909536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:10.953112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57829f24d3841eb9c1d6f5c6a5ea7ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.987188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57829f24d3841eb9c1d6f5c6a5ea7ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:10.997659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:11.003171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57829f24d3841eb9c1d6f5c6a5ea7ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.028960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57829f24d3841eb9c1d6f5c6a5ea7ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.029986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57829f24d3841eb9c1d6f5c6a5ea7ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.050764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2628/5242] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:10 2022] DELETE /identity/v3/users/d57829f24d3841eb9c1d6f5c6a5ea7ee => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:11.065495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8139-3991-4766-abed-28287368cd84 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.144216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976ceeb-7613-4e49-84b4-403585ee502d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.144990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976ceeb-7613-4e49-84b4-403585ee502d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.145352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976ceeb-7613-4e49-84b4-403585ee502d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.146559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976ceeb-7613-4e49-84b4-403585ee502d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.148988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4976ceeb-7613-4e49-84b4-403585ee502d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.156999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:11.263537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976ceeb-7613-4e49-84b4-403585ee502d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.266046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976ceeb-7613-4e49-84b4-403585ee502d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.279350 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2615/5243] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:11.602368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.604153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.620520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f08275-9d13-43eb-be08-6be132f9f735 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.628536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.630052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.630949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.631278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] PATH_INFO: `/v3/projects/a70fbb8e8c8444c5aeb8545753531c36` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.632824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70fbb8e8c8444c5aeb8545753531c36)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:11.635218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:11.711506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70fbb8e8c8444c5aeb8545753531c36', 'request_id': 'req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.722596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70fbb8e8c8444c5aeb8545753531c36', 'request_id': 'req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.731530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70fbb8e8c8444c5aeb8545753531c36', 'request_id': 'req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.757283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] The token cache is being invalidate because project a70fbb8e8c8444c5aeb8545753531c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:11.757665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.758378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:11.764774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2629/5244] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:11 2022] DELETE /identity/v3/projects/a70fbb8e8c8444c5aeb8545753531c36 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:11.766593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875d07c-9dbd-40f9-bf41-ba29e4cc8ed5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.766887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875d07c-9dbd-40f9-bf41-ba29e4cc8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.767345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875d07c-9dbd-40f9-bf41-ba29e4cc8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.803952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5875d07c-9dbd-40f9-bf41-ba29e4cc8ed5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:11.805206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5875d07c-9dbd-40f9-bf41-ba29e4cc8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.851698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:11.853354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:11.980038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2616/5245] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:11.981530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3cbf1-c4ee-4f78-80e5-e679cf07de66 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:11.983332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9255-19f7-4f70-87ab-06e34d733bcc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.983332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9255-19f7-4f70-87ab-06e34d733bcc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.983332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9255-19f7-4f70-87ab-06e34d733bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:11.998263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:11.999126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:11.999471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:11.999663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] PATH_INFO: `/v3/projects/d696ed7cdc3945b78d638693a83ac081` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.001630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d696ed7cdc3945b78d638693a83ac081)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.004682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.031285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8d0a9255-19f7-4f70-87ab-06e34d733bcc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:12.033105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8d0a9255-19f7-4f70-87ab-06e34d733bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.049753 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2617/5246] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:12.051347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a9255-19f7-4f70-87ab-06e34d733bcc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.067427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a9255-19f7-4f70-87ab-06e34d733bcc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.077660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.078800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.079650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.080302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.081341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:12.082426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.084946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.095606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd696ed7cdc3945b78d638693a83ac081', 'request_id': 'req-a3589d71-03a6-43a7-a1d2-747dccace354'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.107288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd696ed7cdc3945b78d638693a83ac081', 'request_id': 'req-a3589d71-03a6-43a7-a1d2-747dccace354'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.107592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2618/5247] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:12.108746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.111863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.117487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd696ed7cdc3945b78d638693a83ac081', 'request_id': 'req-a3589d71-03a6-43a7-a1d2-747dccace354'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.132221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.134351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8517a-05b7-4f01-9dfc-b6b69bbaaac8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.146964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] The token cache is being invalidate because project d696ed7cdc3945b78d638693a83ac081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:12.148009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.149541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.156389 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2630/5248] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:11 2022] DELETE /identity/v3/projects/d696ed7cdc3945b78d638693a83ac081 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:12.156389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.156389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.238404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.239090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.239400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3589d71-03a6-43a7-a1d2-747dccace354 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.240328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.240701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] PATH_INFO: `/v3/projects/e1fff93100b14736adda4047c374baf1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.241975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1fff93100b14736adda4047c374baf1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.244072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.245974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.246949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.247244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.247527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.248167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.250134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.282163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.293483 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2631/5249] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:12.295142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.298230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.313782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1fff93100b14736adda4047c374baf1', 'request_id': 'req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.324059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1fff93100b14736adda4047c374baf1', 'request_id': 'req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.337502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1fff93100b14736adda4047c374baf1', 'request_id': 'req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.351432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] The token cache is being invalidate because project e1fff93100b14736adda4047c374baf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:12.352326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.352721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.354608 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2619/5250] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:12 2022] DELETE /identity/v3/projects/e1fff93100b14736adda4047c374baf1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:12.358147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.359639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.392045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b6861-4c54-4700-a653-03464afc99f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.451205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2a9b-7ebc-4f99-9d73-bbecbb069301 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.456940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.457667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.459421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.459859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.460664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.463226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.494906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2620/5251] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:12.495096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.495096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.495317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.495412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.495412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] PATH_INFO: `/v3/users/a94bd9fd73484ac8932017748d00896b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.497385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.497529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a94bd9fd73484ac8932017748d00896b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.500674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.513350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503c205-444c-4434-8d95-02d1210d5a13 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.522473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.523280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.523584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.524730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] PATH_INFO: `/v3/users/7147017275fa49d3bd6b0124d12f5c5c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.526191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7147017275fa49d3bd6b0124d12f5c5c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.528582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.557886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:12.608792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a94bd9fd73484ac8932017748d00896b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.624358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7147017275fa49d3bd6b0124d12f5c5c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.627037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a94bd9fd73484ac8932017748d00896b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.638336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7147017275fa49d3bd6b0124d12f5c5c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.639544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7147017275fa49d3bd6b0124d12f5c5c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.645483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a94bd9fd73484ac8932017748d00896b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.655043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a94bd9fd73484ac8932017748d00896b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.655808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a94bd9fd73484ac8932017748d00896b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.664745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7147017275fa49d3bd6b0124d12f5c5c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.667846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7147017275fa49d3bd6b0124d12f5c5c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:12.667999 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2632/5252] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:12 2022] DELETE /identity/v3/users/a94bd9fd73484ac8932017748d00896b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:12.671138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.672620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.701019 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2621/5253] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:12 2022] DELETE /identity/v3/users/7147017275fa49d3bd6b0124d12f5c5c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:12.763300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c12e53-a172-42c3-9eaf-79c4a06499b7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.836342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.836961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.837246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.837524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.838218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.841272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.858458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2633/5254] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:12.860357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.863019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:12.885346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109a8e-1748-4c0a-8707-40e50278269d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.890862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c67cc9-4621-40da-a88c-1297d5f9610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.891606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c67cc9-4621-40da-a88c-1297d5f9610a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.891903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c67cc9-4621-40da-a88c-1297d5f9610a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.892180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c67cc9-4621-40da-a88c-1297d5f9610a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.892893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c67cc9-4621-40da-a88c-1297d5f9610a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.894576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c67cc9-4621-40da-a88c-1297d5f9610a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.906607 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2622/5255] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:12.911329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:12.924847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d5c6-d563-46aa-9149-e11de2eea20e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:12.933642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:12.934269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.934553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.934838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] PATH_INFO: `/v3/projects/04191026e983404ea2a556fd00dbc906/users/ff54cb7ffa6941e19539797400e5bdb7/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:12.956563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04191026e983404ea2a556fd00dbc906, user_id=ff54cb7ffa6941e19539797400e5bdb7, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:12.960516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:12.982995 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2634/5256] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:12 2022] PUT /identity/v3/projects/04191026e983404ea2a556fd00dbc906/users/ff54cb7ffa6941e19539797400e5bdb7/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:12.994651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:12.995001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:12.995286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.004376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=79,keystone:DELETE=16,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:13.031671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] MFA Rules not processed for user `ff54cb7ffa6941e19539797400e5bdb7`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:13.032903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.127038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2623/5257] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:13.128321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.168831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:13.218361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a529217-57e0-4798-a920-e8140453e351 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:13.226061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:13.227118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.227118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.227512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.228223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.230416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.265877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.267750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.284362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2635/5258] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:13 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:13.284362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.288264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.294392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b24e0e-c119-440f-8f14-afcbcfab11a6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:13.369933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6a54-24a6-4be3-850a-2e3733195846 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:13.373124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:13.373974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.374262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.374535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] PATH_INFO: `/v3/projects/983d065238544c7ea99324500c083b1a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.375279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:13.375676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983d065238544c7ea99324500c083b1a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.375920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.376204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.376486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] PATH_INFO: `/v3/projects/ed8a932cb6934b6db39eddf8f4d5fb78` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.377587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.377698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8a932cb6934b6db39eddf8f4d5fb78)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.379808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.440439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8a932cb6934b6db39eddf8f4d5fb78', 'request_id': 'req-c524019a-302c-43a3-8177-7b12f3afe6bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.443462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983d065238544c7ea99324500c083b1a', 'request_id': 'req-f81c97c9-7007-46e7-a64e-48586ebd4a99'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.450941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983d065238544c7ea99324500c083b1a', 'request_id': 'req-f81c97c9-7007-46e7-a64e-48586ebd4a99'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.454722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8a932cb6934b6db39eddf8f4d5fb78', 'request_id': 'req-c524019a-302c-43a3-8177-7b12f3afe6bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.460118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983d065238544c7ea99324500c083b1a', 'request_id': 'req-f81c97c9-7007-46e7-a64e-48586ebd4a99'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.463603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8a932cb6934b6db39eddf8f4d5fb78', 'request_id': 'req-c524019a-302c-43a3-8177-7b12f3afe6bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.475091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] The token cache is being invalidate because project 983d065238544c7ea99324500c083b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:13.475605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.477172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.482246 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2624/5259] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:13 2022] DELETE /identity/v3/projects/983d065238544c7ea99324500c083b1a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:13.482246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.482484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] The token cache is being invalidate because project ed8a932cb6934b6db39eddf8f4d5fb78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:13.482484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.482696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.485322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c524019a-302c-43a3-8177-7b12f3afe6bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.487201 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2636/5260] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:13 2022] DELETE /identity/v3/projects/ed8a932cb6934b6db39eddf8f4d5fb78 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:13.492166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.492536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.492915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.513542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:13.516167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:13.517616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.584689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c97c9-7007-46e7-a64e-48586ebd4a99 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:13.591010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:13.591396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:13.592041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.592291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.592572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.593310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.595665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.642139 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2625/5261] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:13 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:13.642255 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2637/5262] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:13.642255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.642402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.642477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.643873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.654669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b456f04-5526-42b8-bd90-2905d497671f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:13.660841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc66b5d-eb99-4987-b396-cca7d3b01b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:13.662054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc66b5d-eb99-4987-b396-cca7d3b01b91 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.662346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc66b5d-eb99-4987-b396-cca7d3b01b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.662622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc66b5d-eb99-4987-b396-cca7d3b01b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.663091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9517b77-c07e-48f7-a0b3-5820a8d1a1a7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:13.663780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc66b5d-eb99-4987-b396-cca7d3b01b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.665417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc66b5d-eb99-4987-b396-cca7d3b01b91 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.684703 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2638/5263] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:13.684703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf1c79-9752-4636-97cb-b18b7cf572f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.684703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf1c79-9752-4636-97cb-b18b7cf572f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.684703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf1c79-9752-4636-97cb-b18b7cf572f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.691413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:13.693407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.693954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.694332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] PATH_INFO: `/v3/users/fcac5ca36d8a421e83503f70f84d52d5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.696829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcac5ca36d8a421e83503f70f84d52d5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.701379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.715430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5dbf1c79-9752-4636-97cb-b18b7cf572f0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:13.717789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5dbf1c79-9752-4636-97cb-b18b7cf572f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:13.745207 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2639/5264] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:13.745207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf1c79-9752-4636-97cb-b18b7cf572f0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.807584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf1c79-9752-4636-97cb-b18b7cf572f0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:13.821967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:13.823083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:13.823726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:13.824189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] PATH_INFO: `/v3/projects/b6854c3782624640bc99e2b6a9bb8e0d/users/79a08001da9e4f19ad60b775aff1575b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:13.845518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6854c3782624640bc99e2b6a9bb8e0d, user_id=79a08001da9e4f19ad60b775aff1575b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:13.846393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcac5ca36d8a421e83503f70f84d52d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.849160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:13.866985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcac5ca36d8a421e83503f70f84d52d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.868115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcac5ca36d8a421e83503f70f84d52d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.874123 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2640/5265] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:13 2022] PUT /identity/v3/projects/b6854c3782624640bc99e2b6a9bb8e0d/users/79a08001da9e4f19ad60b775aff1575b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:13.875524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.882751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcac5ca36d8a421e83503f70f84d52d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.883530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcac5ca36d8a421e83503f70f84d52d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:13.912293 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2626/5266] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:13 2022] DELETE /identity/v3/users/fcac5ca36d8a421e83503f70f84d52d5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:13.912293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:13.914895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:14.020622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb301e3-e938-441f-ac91-617bffad88dd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.117735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6430e26-54ec-4451-b305-1945d6e87c27 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.123185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.123551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.123855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.124143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.124389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.124502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.124796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.125097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.125208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.125914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:14.127016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.127172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.128070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.142103 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2627/5267] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:14.148420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.152955 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2641/5268] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:14.154162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.157469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:14.184930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e921e4-c44d-460d-b432-0957b2f03aaf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.201918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf22fc7-114a-4ecf-86b9-6a8266191cc5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.202145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd4f2e-d58b-49c6-8ba9-134d0d89207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.203500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4f2e-d58b-49c6-8ba9-134d0d89207f admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.203855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4f2e-d58b-49c6-8ba9-134d0d89207f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.204206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4f2e-d58b-49c6-8ba9-134d0d89207f admin admin] PATH_INFO: `/v3/projects/b6854c3782624640bc99e2b6a9bb8e0d/users/79a08001da9e4f19ad60b775aff1575b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.206800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd4f2e-d58b-49c6-8ba9-134d0d89207f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6854c3782624640bc99e2b6a9bb8e0d, user_id=79a08001da9e4f19ad60b775aff1575b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.207547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.208987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.209466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.210143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.210898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd4f2e-d58b-49c6-8ba9-134d0d89207f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.211018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.213920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.228078 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2628/5269] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:14 2022] PUT /identity/v3/projects/b6854c3782624640bc99e2b6a9bb8e0d/users/79a08001da9e4f19ad60b775aff1575b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:14.239371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b001e5-25f5-4478-8c88-62e7231e6d06 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.240359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b001e5-25f5-4478-8c88-62e7231e6d06 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.240641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b001e5-25f5-4478-8c88-62e7231e6d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.262624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:14.268758 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2642/5270] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:14 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:14.270087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.295024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-39b001e5-25f5-4478-8c88-62e7231e6d06 None None] MFA Rules not processed for user `79a08001da9e4f19ad60b775aff1575b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:14.296820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-39b001e5-25f5-4478-8c88-62e7231e6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:14.324329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:14.387473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae15109-6cfe-454b-8f28-f103626949e7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.394460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.395080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.395361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.395638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.397582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.399682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.419208 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2629/5271] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:14.465704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b001e5-25f5-4478-8c88-62e7231e6d06 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.465992 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2643/5272] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:14 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:14.500602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b001e5-25f5-4478-8c88-62e7231e6d06 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.512275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.512902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.513364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.513642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.515009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.516775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.555429 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2630/5273] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:14.556192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.564213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.570318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:14.571839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9d7c4-3599-40b7-9bda-9d31131affd3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.580156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.580945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.580945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.581124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.582643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.584072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.633150 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2644/5274] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:14.634084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.639169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:14.668922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d00dcd-2315-4749-b233-29c5861a474a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.679596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.683450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.683450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.683450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] PATH_INFO: `/v3/projects/af44d369d3f048aaada7923ccf3c705f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.686702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af44d369d3f048aaada7923ccf3c705f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.688700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.723572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec58c7-64d2-4a58-863b-f5aeb02a3d40 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.729164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.729775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.730174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.730443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.747568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.749942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.765486 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2631/5275] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:14.767160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:14.782063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b44b-947a-4678-b529-5951b44fb389 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:14.799883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af44d369d3f048aaada7923ccf3c705f', 'request_id': 'req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:14.813564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af44d369d3f048aaada7923ccf3c705f', 'request_id': 'req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:14.826923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af44d369d3f048aaada7923ccf3c705f', 'request_id': 'req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:14.870145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] The token cache is being invalidate because project af44d369d3f048aaada7923ccf3c705f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:14.870651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:14.871694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7dfbd3-0033-42de-b6b3-e32fa336bd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:14.874038 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2645/5276] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:14 2022] DELETE /identity/v3/projects/af44d369d3f048aaada7923ccf3c705f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:14.879787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229a410-6066-408c-9880-a95eff767514 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.880856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229a410-6066-408c-9880-a95eff767514 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.881440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229a410-6066-408c-9880-a95eff767514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.896916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d09f8-639a-43f2-8cd1-f2a1b499dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:14.897862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d09f8-639a-43f2-8cd1-f2a1b499dce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.898411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d09f8-639a-43f2-8cd1-f2a1b499dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.898876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d09f8-639a-43f2-8cd1-f2a1b499dce2 admin admin] PATH_INFO: `/v3/projects/e2d0a21cdb7149ca9b72a502c7eb84fa/users/5d9931925a544aa894cf2b08cad10ea2/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.906515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4229a410-6066-408c-9880-a95eff767514 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:14.908061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4229a410-6066-408c-9880-a95eff767514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:14.918537 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2646/5277] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:14.930174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7dd92-39bb-4444-bc36-e23aa55077e2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.930654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7dd92-39bb-4444-bc36-e23aa55077e2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.931224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7dd92-39bb-4444-bc36-e23aa55077e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:14.934683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d09f8-639a-43f2-8cd1-f2a1b499dce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d0a21cdb7149ca9b72a502c7eb84fa, user_id=5d9931925a544aa894cf2b08cad10ea2, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:14.938929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d09f8-639a-43f2-8cd1-f2a1b499dce2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:14.967378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3af7dd92-39bb-4444-bc36-e23aa55077e2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:14.968997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3af7dd92-39bb-4444-bc36-e23aa55077e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:14.984156 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2632/5278] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:14 2022] PUT /identity/v3/projects/e2d0a21cdb7149ca9b72a502c7eb84fa/users/5d9931925a544aa894cf2b08cad10ea2/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:14.984156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:14.984156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:14.984459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.010075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:15.018271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:15.026446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:15.073243 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2647/5279] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:15.077557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7dd92-39bb-4444-bc36-e23aa55077e2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.098978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7dd92-39bb-4444-bc36-e23aa55077e2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.111790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.112502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.112868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.113210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.116261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:15.117302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.119427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.132314 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2633/5280] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:15.144847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.147876 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2648/5281] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:07:15.151685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.152296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:15.172674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86dba1-9b2a-4de6-85d9-df9a5bedc009 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.200283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.202582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.202882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.203165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.203812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.206482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.214332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:15.257131 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2649/5282] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:15 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:15.258521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.259533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb372d-cf39-4d69-8a4d-2385d610eb83 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.374768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75c8bf-5d54-49c3-8515-73d58a7384d9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.396483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.397320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.398468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.399442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.399807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.401938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.406270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.407100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.407686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.408040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.424156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.426051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.440710 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2634/5283] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:15.456540 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2650/5284] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:15 2022] POST /identity/v3/users => generated 301 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:15.457897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.478048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66445ddd-d01e-4c37-820c-6d3aa7c80fdb service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.485931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.486515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.487571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.487850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.488627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.490878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.503104 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2635/5285] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:15.508596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.535330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.538125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:15.538368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68475dde-4f85-4833-b0cc-bcf2cdda5c0f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.544124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.545050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.546059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.546442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.547364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.549564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.563732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00931b8b-871d-4efa-ad9f-afdded86751c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.568954 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2651/5286] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:15.573770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.576111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.577623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.578021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.578312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] PATH_INFO: `/v3/users/ff54cb7ffa6941e19539797400e5bdb7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.579667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff54cb7ffa6941e19539797400e5bdb7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.581861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.589541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127db0b0-390f-4da8-b60f-22a421ff778a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:15.599566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de69eba-dce3-4631-8093-dea4245cfeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:15.599566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de69eba-dce3-4631-8093-dea4245cfeab admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.599566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de69eba-dce3-4631-8093-dea4245cfeab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.599566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de69eba-dce3-4631-8093-dea4245cfeab admin admin] PATH_INFO: `/v3/projects/882e3230af48430f99f2500bc47a4243/users/a9f61a0545e74ddbb2ceafde216882a7/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.624931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de69eba-dce3-4631-8093-dea4245cfeab admin admin] RBAC: Authorizing `identity:create_grant(project_id=882e3230af48430f99f2500bc47a4243, user_id=a9f61a0545e74ddbb2ceafde216882a7, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:15.633659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de69eba-dce3-4631-8093-dea4245cfeab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:15.671399 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2652/5287] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:15 2022] PUT /identity/v3/projects/882e3230af48430f99f2500bc47a4243/users/a9f61a0545e74ddbb2ceafde216882a7/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:15.684065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.684714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.684714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.696644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff54cb7ffa6941e19539797400e5bdb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:15.704093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:15.730282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff54cb7ffa6941e19539797400e5bdb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:15.731121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff54cb7ffa6941e19539797400e5bdb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:15.742615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff54cb7ffa6941e19539797400e5bdb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:15.744958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-28c0f789-c8e1-4c7b-a594-ab88f262ec6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff54cb7ffa6941e19539797400e5bdb7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:15.771664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] MFA Rules not processed for user `a9f61a0545e74ddbb2ceafde216882a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:15.772905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:15.784998 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2636/5288] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:15 2022] DELETE /identity/v3/users/ff54cb7ffa6941e19539797400e5bdb7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:15.789469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:15.792160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:15.792457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:15.836912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:15.839818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:15.932071 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2653/5289] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:15.936714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:15.938168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:16.002263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2637/5290] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:16.020486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:16.034590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.036874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:16.091153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac541-dcad-4084-944b-2a69edf6714e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.097724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.099634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.099634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.100051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.101210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.103856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.159227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2654/5291] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:15 2022] POST /identity/v3/projects => generated 401 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:16.160206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.170056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076240b-815f-450c-882f-e4dcdb680b4e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.303757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.304666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.305126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.305549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.319626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098a95c-629c-463b-8be2-04885f8878be admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.325458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.326326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.326616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.327405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.328171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.334261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.365916 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2655/5292] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:16 2022] POST /identity/v3/users => generated 307 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:16.370659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.370877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:16.383538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4768268-346c-46f6-84f6-c84b17c917db admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.389804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.390613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.390901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.391179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.391824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.393622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.406332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2656/5293] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:16.410424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.410690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.414255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.420975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cb8dd-4cd1-469b-877a-8f32ae45a1b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.430563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.431526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.431646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.432091 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2638/5294] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:16.432272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] PATH_INFO: `/v3/projects/ba7c7a20a86e4089b804190994b26c6b/users/7de4cde79fb143f094c4e3425d0a1653/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.451061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7c7a20a86e4089b804190994b26c6b, user_id=7de4cde79fb143f094c4e3425d0a1653, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.454668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.472978 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2657/5295] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:16 2022] PUT /identity/v3/projects/ba7c7a20a86e4089b804190994b26c6b/users/7de4cde79fb143f094c4e3425d0a1653/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:16.477978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.495537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8fb05-fc3e-4380-b44e-f9e2066f6410 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.605507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68282829-497e-4578-9a03-15827112507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.606439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68282829-497e-4578-9a03-15827112507c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.606879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68282829-497e-4578-9a03-15827112507c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.607397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68282829-497e-4578-9a03-15827112507c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.608798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68282829-497e-4578-9a03-15827112507c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.610805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68282829-497e-4578-9a03-15827112507c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.624541 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2639/5296] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:16.629160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.641973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804157c7-57be-4e92-bf2f-1eb305a46b2c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.649126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.650168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.650527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.651184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] PATH_INFO: `/v3/projects/ba7c7a20a86e4089b804190994b26c6b/users/7de4cde79fb143f094c4e3425d0a1653/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.653560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7c7a20a86e4089b804190994b26c6b, user_id=7de4cde79fb143f094c4e3425d0a1653, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.656554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.678091 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2658/5297] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:16 2022] PUT /identity/v3/projects/ba7c7a20a86e4089b804190994b26c6b/users/7de4cde79fb143f094c4e3425d0a1653/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:16.690363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.690694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.690915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.734825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] MFA Rules not processed for user `7de4cde79fb143f094c4e3425d0a1653`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:16.736282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:16.815573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.819827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:16.858217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2640/5298] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:16 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:16.869907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:16.873238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:16.952694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:16.978659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266305d-1289-4f11-94fa-25a3f4ddcb95 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.979082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e0dc1-cb25-43ed-83d6-81359fb3cf67 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:16.983079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.983690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.983969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.984298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.984887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:16.985570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:16.985895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:16.986177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] PATH_INFO: `/v3/projects/04191026e983404ea2a556fd00dbc906` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:16.987426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04191026e983404ea2a556fd00dbc906)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.989755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:16.995431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:16.996973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:17.001580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:17.011730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2641/5299] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:16 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:17.050390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04191026e983404ea2a556fd00dbc906', 'request_id': 'req-e909900f-cf81-4815-bf99-4da74807d8a0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.057729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04191026e983404ea2a556fd00dbc906', 'request_id': 'req-e909900f-cf81-4815-bf99-4da74807d8a0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.065182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04191026e983404ea2a556fd00dbc906', 'request_id': 'req-e909900f-cf81-4815-bf99-4da74807d8a0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.084030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] The token cache is being invalidate because project 04191026e983404ea2a556fd00dbc906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:17.084388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.085165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.087334 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2659/5300] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:16 2022] DELETE /identity/v3/projects/04191026e983404ea2a556fd00dbc906 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:17.091167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:17.104016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:17.104984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db466b76-98c2-4bba-b4c4-8efd5b7c0234 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:17.108143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:17.175253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:17.175719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:17.176082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:17.176477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] PATH_INFO: `/v3/users/79a08001da9e4f19ad60b775aff1575b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:17.178345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a08001da9e4f19ad60b775aff1575b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:17.178843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909900f-cf81-4815-bf99-4da74807d8a0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:17.180541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:17.183029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:17.184063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:17.184063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:17.184299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:17.186283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:17.240758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a08001da9e4f19ad60b775aff1575b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.249011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a08001da9e4f19ad60b775aff1575b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.249980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a08001da9e4f19ad60b775aff1575b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.259294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a08001da9e4f19ad60b775aff1575b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.260065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a08001da9e4f19ad60b775aff1575b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:17.267369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:17.269093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:17.281711 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2660/5301] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:17 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:17.284938 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2642/5302] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:17 2022] DELETE /identity/v3/users/79a08001da9e4f19ad60b775aff1575b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:17.830365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:17.832843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:17.841580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:17.844462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:17.851869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f362b0-7fa4-4a86-b1d2-d76e2e5e8e39 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:17.933989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:17.941232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464b83b-a1e2-4c7d-886c-4c3cf633e7b3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:17.945636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308f5f8-c873-4031-a826-cf76e06109f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:17.946518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f5f8-c873-4031-a826-cf76e06109f2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:17.946806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f5f8-c873-4031-a826-cf76e06109f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:17.947080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308f5f8-c873-4031-a826-cf76e06109f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:17.950113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5308f5f8-c873-4031-a826-cf76e06109f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:17.967714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:17.977719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:17.977719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:17.977719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] PATH_INFO: `/v3/projects/b6854c3782624640bc99e2b6a9bb8e0d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:17.977719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6854c3782624640bc99e2b6a9bb8e0d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:17.977719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:18.012340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:18.078910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f5f8-c873-4031-a826-cf76e06109f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:18.080906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308f5f8-c873-4031-a826-cf76e06109f2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:18.082583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6854c3782624640bc99e2b6a9bb8e0d', 'request_id': 'req-6972828f-848c-474f-b5f7-9076de39578b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:18.095962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6854c3782624640bc99e2b6a9bb8e0d', 'request_id': 'req-6972828f-848c-474f-b5f7-9076de39578b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:18.111053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6854c3782624640bc99e2b6a9bb8e0d', 'request_id': 'req-6972828f-848c-474f-b5f7-9076de39578b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:18.112375 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2643/5303] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:18.139059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] The token cache is being invalidate because project b6854c3782624640bc99e2b6a9bb8e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:18.139556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:18.140473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6972828f-848c-474f-b5f7-9076de39578b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:18.145603 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2661/5304] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:17 2022] DELETE /identity/v3/projects/b6854c3782624640bc99e2b6a9bb8e0d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:18.147557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd03f1c-0c44-47a9-a411-2f05753fb895 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.147874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd03f1c-0c44-47a9-a411-2f05753fb895 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.148182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd03f1c-0c44-47a9-a411-2f05753fb895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.184177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cbd03f1c-0c44-47a9-a411-2f05753fb895 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:18.185418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cbd03f1c-0c44-47a9-a411-2f05753fb895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:18.287296 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2644/5305] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:18.294897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3766e1-3265-4f3e-b478-0cc1a896c697 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.295470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3766e1-3265-4f3e-b478-0cc1a896c697 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.295754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3766e1-3265-4f3e-b478-0cc1a896c697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.335266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2e3766e1-3265-4f3e-b478-0cc1a896c697 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:18.336729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2e3766e1-3265-4f3e-b478-0cc1a896c697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:18.369451 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2662/5306] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:18.374727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd03f1c-0c44-47a9-a411-2f05753fb895 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:18.393018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd03f1c-0c44-47a9-a411-2f05753fb895 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:18.404113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:18.405838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.406139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.406419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.407084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:18.407390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:18.409943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:18.431081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3766e1-3265-4f3e-b478-0cc1a896c697 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:18.445241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2645/5307] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:18.446773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3766e1-3265-4f3e-b478-0cc1a896c697 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:18.453393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:18.454072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.455026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.455311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.456534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:18.459787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:18.511653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:18.520821 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2663/5308] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:18 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:18.526357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55676e0-97d9-4fea-a20b-a1b2bbcb737c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:18.613221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:18.613853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.614165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.614987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.616122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:18.618529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:18.691150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:18.704246 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2646/5309] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:18 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:18.711664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c00ad-0c0a-4173-b54c-aaa88c153040 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:18.751981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:18.752770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.752938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.753192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.763492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:18.765237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:18.823213 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2664/5310] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:18.827764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:18.841558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b17e6c-a712-42d1-aaf3-65291ff180a7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:18.855111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197e899-e89d-450b-928e-a4ed275cbe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:18.856488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e899-e89d-450b-928e-a4ed275cbe4f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.857631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e899-e89d-450b-928e-a4ed275cbe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.861385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e899-e89d-450b-928e-a4ed275cbe4f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.862343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197e899-e89d-450b-928e-a4ed275cbe4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:18.864995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197e899-e89d-450b-928e-a4ed275cbe4f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:18.886524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:18.909815 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2647/5311] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:18.917069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452046f-ab67-4cbc-b123-c6b6540c02cb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:18.936412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:18.939043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:18.939207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:18.939894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] PATH_INFO: `/v3/projects/c124a777bb2e44ea9b190b5f8dd6fbe2/users/e2f9cacb9b8949639dbbd20cf3a01587/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:18.983560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c124a777bb2e44ea9b190b5f8dd6fbe2, user_id=e2f9cacb9b8949639dbbd20cf3a01587, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:18.989463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:19.014670 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2665/5312] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:18 2022] PUT /identity/v3/projects/c124a777bb2e44ea9b190b5f8dd6fbe2/users/e2f9cacb9b8949639dbbd20cf3a01587/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:19.025518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757374a-6c0e-4374-adc4-13a248569119 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:19.027160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757374a-6c0e-4374-adc4-13a248569119 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:19.027938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757374a-6c0e-4374-adc4-13a248569119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:19.127226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b757374a-6c0e-4374-adc4-13a248569119 None None] MFA Rules not processed for user `e2f9cacb9b8949639dbbd20cf3a01587`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:19.131485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b757374a-6c0e-4374-adc4-13a248569119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:19.204746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:19.351852 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2648/5313] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:19.353225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:19.553628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e67e5-fda8-4043-b32b-9396f13f4668 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:19.561189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:19.561935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:19.562325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:19.562914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:19.563499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:19.567078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:19.585251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:19.624824 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2666/5314] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:19 2022] POST /identity/v3/projects => generated 373 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:19.632383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757374a-6c0e-4374-adc4-13a248569119 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:19.667841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757374a-6c0e-4374-adc4-13a248569119 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:19.784300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:19.796423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:19.798175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:19.798175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:19.798175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:19.808089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:19.860124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:19.872610 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2649/5315] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:19 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:19.876290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e11e8b-e363-4953-9e67-d4c04ad27a30 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:19.898196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:19.898956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:19.899797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:19.901450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:19.902596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:19.906958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:19.939192 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2667/5316] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:19.939921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:19.956614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c139b2-54be-4a3d-8728-4fb634a2f962 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:19.992862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:19.994403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:19.995198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:19.996136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] PATH_INFO: `/v3/projects/d5dd5b65d455452e9b60e680e11f36bb/users/f803b0febdcb405bac29b377f94a5f9e/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:20.042085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5dd5b65d455452e9b60e680e11f36bb, user_id=f803b0febdcb405bac29b377f94a5f9e, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:20.044561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:20.084807 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2650/5317] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:19 2022] PUT /identity/v3/projects/d5dd5b65d455452e9b60e680e11f36bb/users/f803b0febdcb405bac29b377f94a5f9e/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:20.086573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:20.102811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b2c00-d2bf-4a5f-983e-cf6a5e84ddbb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:20.300120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e14af8-67fa-44a8-9545-c6093a678efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:20.305532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e14af8-67fa-44a8-9545-c6093a678efa admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:20.306234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e14af8-67fa-44a8-9545-c6093a678efa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:20.306891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e14af8-67fa-44a8-9545-c6093a678efa admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:20.311508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e14af8-67fa-44a8-9545-c6093a678efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:20.313305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e14af8-67fa-44a8-9545-c6093a678efa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:20.360956 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2668/5318] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:20.366959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:20.389152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e264a-d841-4c07-92a7-dfcc9531f3cd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:20.399718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:20.400805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:20.401264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:20.401710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] PATH_INFO: `/v3/projects/d5dd5b65d455452e9b60e680e11f36bb/users/f803b0febdcb405bac29b377f94a5f9e/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:20.404152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5dd5b65d455452e9b60e680e11f36bb, user_id=f803b0febdcb405bac29b377f94a5f9e, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:20.407312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:20.436872 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2651/5319] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:20 2022] PUT /identity/v3/projects/d5dd5b65d455452e9b60e680e11f36bb/users/f803b0febdcb405bac29b377f94a5f9e/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:20.451811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:20.452525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:20.452955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:20.519070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] MFA Rules not processed for user `f803b0febdcb405bac29b377f94a5f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:20.520363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:20.675056 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2669/5320] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:20.685642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:20.688464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:20.807077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d82005-bee6-4765-8f2a-8696f6db2bf9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:20.817086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aea9c2-f06a-4866-8838-9fcb02a2a5e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:20.817808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aea9c2-f06a-4866-8838-9fcb02a2a5e8 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:20.818117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aea9c2-f06a-4866-8838-9fcb02a2a5e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:20.818547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aea9c2-f06a-4866-8838-9fcb02a2a5e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:20.888349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:20.944020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aea9c2-f06a-4866-8838-9fcb02a2a5e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:20.944020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aea9c2-f06a-4866-8838-9fcb02a2a5e8 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:20.968980 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2652/5321] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:22.530985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:22.532638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:22.634737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3457ba4-853c-4cdd-babe-c8311dcdd6dc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:22.642614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:22.643270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:22.644207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:22.644519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] PATH_INFO: `/v3/users/a9f61a0545e74ddbb2ceafde216882a7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:22.646447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f61a0545e74ddbb2ceafde216882a7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:22.649199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:22.652642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:22.802556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f61a0545e74ddbb2ceafde216882a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:22.837394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f61a0545e74ddbb2ceafde216882a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:22.864932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f61a0545e74ddbb2ceafde216882a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:22.896647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f61a0545e74ddbb2ceafde216882a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:22.897841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f61a0545e74ddbb2ceafde216882a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:22.911050 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2670/5322] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:22 2022] DELETE /identity/v3/users/a9f61a0545e74ddbb2ceafde216882a7 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:22.916537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:22.917085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:22.917085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:22.962239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:22.963856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:23.107911 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2653/5323] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:23.169653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:23.170754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:23.294111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402f32e-bf04-4d89-9905-cc7ffd033f18 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:23.301001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c85ef-41c4-4f45-8fdc-d4c783138e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:23.303036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c85ef-41c4-4f45-8fdc-d4c783138e74 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:23.303337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c85ef-41c4-4f45-8fdc-d4c783138e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:23.304571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c85ef-41c4-4f45-8fdc-d4c783138e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:23.330082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c85ef-41c4-4f45-8fdc-d4c783138e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:23.331848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c85ef-41c4-4f45-8fdc-d4c783138e74 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:23.380327 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2671/5324] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:24.202026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:24.203478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:24.230332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328c6db-0b61-4294-800c-724a271b73d6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:24.252840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:24.253512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:24.253794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:24.254097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] PATH_INFO: `/v3/users/e2f9cacb9b8949639dbbd20cf3a01587` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:24.257945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f9cacb9b8949639dbbd20cf3a01587)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:24.260047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:24.393395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f9cacb9b8949639dbbd20cf3a01587'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.406097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f9cacb9b8949639dbbd20cf3a01587'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.407134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f9cacb9b8949639dbbd20cf3a01587'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.427631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f9cacb9b8949639dbbd20cf3a01587'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.429048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f9cacb9b8949639dbbd20cf3a01587'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:24.468792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:24.469310 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2654/5325] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:24 2022] DELETE /identity/v3/users/e2f9cacb9b8949639dbbd20cf3a01587 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:24.469976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:24.470429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:24.508350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:24.509758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:24.597305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2672/5326] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:24.614862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:24.619580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:24.717538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:24.750994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939a1ea-66e4-4b67-a530-f9ed3e3c5ede admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:24.759866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:24.760660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:24.761125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:24.761809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:24.787306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:24.789998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:24.807781 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2655/5327] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:24.829967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:24.831106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:24.861449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e05b-31e9-4d71-9773-2e626489484c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:24.873305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:24.874806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:24.880778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:24.881513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] PATH_INFO: `/v3/projects/882e3230af48430f99f2500bc47a4243` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:24.883585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] RBAC: Authorizing `identity:delete_project(project_id=882e3230af48430f99f2500bc47a4243)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:24.890147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:24.956550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:25.015946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882e3230af48430f99f2500bc47a4243', 'request_id': 'req-b4447750-18cb-4b1c-8570-fcb24826d86e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.023706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882e3230af48430f99f2500bc47a4243', 'request_id': 'req-b4447750-18cb-4b1c-8570-fcb24826d86e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.032174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882e3230af48430f99f2500bc47a4243', 'request_id': 'req-b4447750-18cb-4b1c-8570-fcb24826d86e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.053148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] The token cache is being invalidate because project 882e3230af48430f99f2500bc47a4243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:25.053874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.054899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.057207 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2673/5328] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:24 2022] DELETE /identity/v3/projects/882e3230af48430f99f2500bc47a4243 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:25.061138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:25.074893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15caa5-35c7-4260-816d-c0acd60687f2 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:25.206607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:25.208443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:25.210101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:25.210919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] PATH_INFO: `/v3/users/7de4cde79fb143f094c4e3425d0a1653` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:25.215207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7de4cde79fb143f094c4e3425d0a1653)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:25.219144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:25.303670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7de4cde79fb143f094c4e3425d0a1653'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.318611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7de4cde79fb143f094c4e3425d0a1653'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.319457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7de4cde79fb143f094c4e3425d0a1653'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.337115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7de4cde79fb143f094c4e3425d0a1653'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.338129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7de4cde79fb143f094c4e3425d0a1653'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:25.386637 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2656/5329] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:25 2022] DELETE /identity/v3/users/7de4cde79fb143f094c4e3425d0a1653 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:26.409774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:26.412033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:26.436877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4447750-18cb-4b1c-8570-fcb24826d86e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:26.542900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:26.544039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.544567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.545107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] PATH_INFO: `/v3/projects/c124a777bb2e44ea9b190b5f8dd6fbe2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.547290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c124a777bb2e44ea9b190b5f8dd6fbe2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:26.551331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:26.638722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c124a777bb2e44ea9b190b5f8dd6fbe2', 'request_id': 'req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.646741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c124a777bb2e44ea9b190b5f8dd6fbe2', 'request_id': 'req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.665355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c124a777bb2e44ea9b190b5f8dd6fbe2', 'request_id': 'req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.684113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] The token cache is being invalidate because project c124a777bb2e44ea9b190b5f8dd6fbe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:26.684499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.685446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.690360 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2674/5330] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:26 2022] DELETE /identity/v3/projects/c124a777bb2e44ea9b190b5f8dd6fbe2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:26.691672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:26.693452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:26.725798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4557169-0907-4098-87d3-fc981fab298d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:26.859882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:26.860533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:26.860834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:26.861081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] PATH_INFO: `/v3/users/f803b0febdcb405bac29b377f94a5f9e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:26.866501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f803b0febdcb405bac29b377f94a5f9e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:26.868597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:26.882128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:26.980648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f803b0febdcb405bac29b377f94a5f9e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.994834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f803b0febdcb405bac29b377f94a5f9e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:26.995609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f803b0febdcb405bac29b377f94a5f9e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.010758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f803b0febdcb405bac29b377f94a5f9e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.011659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f3ea4f26-4948-484b-a161-4956ff0f311a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f803b0febdcb405bac29b377f94a5f9e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.042460 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2657/5331] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:26 2022] DELETE /identity/v3/users/f803b0febdcb405bac29b377f94a5f9e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:27.407583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:27.409434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:27.435024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf81e1-0f59-4c9e-ad42-cf3bac93d988 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:27.476245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:27.515743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:27.517302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.517747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.518209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] PATH_INFO: `/v3/projects/ba7c7a20a86e4089b804190994b26c6b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.520975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7c7a20a86e4089b804190994b26c6b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:27.524648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:27.648508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7c7a20a86e4089b804190994b26c6b', 'request_id': 'req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.659536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7c7a20a86e4089b804190994b26c6b', 'request_id': 'req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.671249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7c7a20a86e4089b804190994b26c6b', 'request_id': 'req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.694713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] The token cache is being invalidate because project ba7c7a20a86e4089b804190994b26c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:27.695066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.696294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:27.699735 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2675/5332] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:27 2022] DELETE /identity/v3/projects/ba7c7a20a86e4089b804190994b26c6b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:27.706798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82558991-80ee-444f-a81c-9f86c7f296a1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.707307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82558991-80ee-444f-a81c-9f86c7f296a1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.707443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82558991-80ee-444f-a81c-9f86c7f296a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.749808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-82558991-80ee-444f-a81c-9f86c7f296a1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:27.752145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-82558991-80ee-444f-a81c-9f86c7f296a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:27.822978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:27.825228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:27.898667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2658/5333] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:27.907252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3d439-7d2f-4780-8dde-9c495093c475 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:27.907547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3d439-7d2f-4780-8dde-9c495093c475 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:27.908478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3d439-7d2f-4780-8dde-9c495093c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:27.960666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-14b3d439-7d2f-4780-8dde-9c495093c475 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:27.963678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-14b3d439-7d2f-4780-8dde-9c495093c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:27.986791 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2659/5334] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:27.991783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3d439-7d2f-4780-8dde-9c495093c475 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:28.008327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed94d7-05e6-4fe6-9ada-fd9e760fdf60 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:28.011847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3d439-7d2f-4780-8dde-9c495093c475 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:28.025524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:28.026252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.026537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.026815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] PATH_INFO: `/v3/users/4faa458c158141efaf707317411cd544` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:28.027800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:28.028644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.029132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.029755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:28.030683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:28.031205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:28.033891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:28.034718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4faa458c158141efaf707317411cd544)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:28.037299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:28.061203 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2660/5335] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:07:28.065465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:28.078783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fbaf4-0efb-4228-8b08-075f74b67b51 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:28.089630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:28.091338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.092788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.094094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:28.095266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:28.125712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:28.136419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4faa458c158141efaf707317411cd544'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:28.166909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4faa458c158141efaf707317411cd544'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:28.177409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:28.182166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4faa458c158141efaf707317411cd544'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:28.198729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4faa458c158141efaf707317411cd544'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:28.199740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4faa458c158141efaf707317411cd544'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:28.206792 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2661/5336] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:28 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:28.210502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:28.210774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2676/5337] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:27 2022] DELETE /identity/v3/users/4faa458c158141efaf707317411cd544 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:28.216702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:28.325676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:28.372107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cbb0b-3394-4848-a36d-732b27289e5b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:28.527447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:28.528206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.528503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.528840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:28.529558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:28.531931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:28.596935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2662/5338] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:28 2022] POST /identity/v3/users => generated 294 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:28.607548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:28.639611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59e179-c607-4d58-8661-5a8b250693d5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:28.656882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:28.660457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.661447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.662037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:28.664569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:28.671038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:28.696335 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2677/5339] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:28.700173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:28.712711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfcaba-40ac-4e82-b63d-86c8e5d39563 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:28.737974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3b043-6778-417d-b687-dd4393e39ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:28.738866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3b043-6778-417d-b687-dd4393e39ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.739602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3b043-6778-417d-b687-dd4393e39ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.740514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3b043-6778-417d-b687-dd4393e39ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:28.781121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3b043-6778-417d-b687-dd4393e39ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:28.793287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3b043-6778-417d-b687-dd4393e39ca8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:28.819161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:28.819383 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2663/5340] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:28.860688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cee11-0564-40e2-82f5-cced1622c53a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:28.871793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:28.872757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.873182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.873595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] PATH_INFO: `/v3/projects/8fa0fb77f348429a829126fcaa6870fd/users/f1164204a50547a18ead2921f6bc610e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:28.922622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa0fb77f348429a829126fcaa6870fd, user_id=f1164204a50547a18ead2921f6bc610e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:28.935676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:28.974952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2678/5341] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:28 2022] PUT /identity/v3/projects/8fa0fb77f348429a829126fcaa6870fd/users/f1164204a50547a18ead2921f6bc610e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:28.991358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f1c0-9c01-4dfa-a5f8-bc37d1ca5d8e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:28.991971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f1c0-9c01-4dfa-a5f8-bc37d1ca5d8e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:28.992452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f1c0-9c01-4dfa-a5f8-bc37d1ca5d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.062248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0374f1c0-9c01-4dfa-a5f8-bc37d1ca5d8e None None] MFA Rules not processed for user `f1164204a50547a18ead2921f6bc610e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:29.063574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0374f1c0-9c01-4dfa-a5f8-bc37d1ca5d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.220913 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2664/5342] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:29.226489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:29.387438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82d76b-7a53-44e7-8df1-7c9e74901439 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:29.402258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:29.403032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.403320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.403684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.404390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:29.407721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:29.467485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2679/5343] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:29 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:29.473267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f1c0-9c01-4dfa-a5f8-bc37d1ca5d8e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:29.490912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:29.493291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.506237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f1c0-9c01-4dfa-a5f8-bc37d1ca5d8e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:29.679258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c981-f478-4b52-ac25-751cd87b4e31 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:29.685016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:29.685993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.686368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.688026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] PATH_INFO: `/v3/projects/a58f02659fe74b70a8657691faaad6c9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.688198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58f02659fe74b70a8657691faaad6c9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:29.690266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:29.725005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:29.741171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:29.741807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.742124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.742601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.743547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:29.745498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:29.789951 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2665/5344] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:29 2022] POST /identity/v3/users => generated 299 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:29.789951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:29.790857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:29.802163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:29.806184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d6286-af0f-4fe9-87d9-d59b7de40328 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:29.819341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58f02659fe74b70a8657691faaad6c9', 'request_id': 'req-f04dbf4e-2b38-4e81-8cf8-9b190816e371'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.830788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58f02659fe74b70a8657691faaad6c9', 'request_id': 'req-f04dbf4e-2b38-4e81-8cf8-9b190816e371'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.850352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58f02659fe74b70a8657691faaad6c9', 'request_id': 'req-f04dbf4e-2b38-4e81-8cf8-9b190816e371'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.883962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] The token cache is being invalidate because project a58f02659fe74b70a8657691faaad6c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:29.884698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.885490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f04dbf4e-2b38-4e81-8cf8-9b190816e371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:29.891587 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2680/5345] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:29 2022] DELETE /identity/v3/projects/a58f02659fe74b70a8657691faaad6c9 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:29.893498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a390e-184f-4bea-9c93-fb628a14b7a9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.893793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a390e-184f-4bea-9c93-fb628a14b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.894215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a390e-184f-4bea-9c93-fb628a14b7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.906196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:29.907632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:29.907927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:29.908397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] PATH_INFO: `/v3/projects/d5dd5b65d455452e9b60e680e11f36bb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:29.910027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5dd5b65d455452e9b60e680e11f36bb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:29.913159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:29.981548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bd8a390e-184f-4bea-9c93-fb628a14b7a9 None None] MFA Rules not processed for user `5d9931925a544aa894cf2b08cad10ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:29.983921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bd8a390e-184f-4bea-9c93-fb628a14b7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.015503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5dd5b65d455452e9b60e680e11f36bb', 'request_id': 'req-a2340dfc-e239-464b-8428-e51d45ca20ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:30.043503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5dd5b65d455452e9b60e680e11f36bb', 'request_id': 'req-a2340dfc-e239-464b-8428-e51d45ca20ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:30.054925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5dd5b65d455452e9b60e680e11f36bb', 'request_id': 'req-a2340dfc-e239-464b-8428-e51d45ca20ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:30.068502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] The token cache is being invalidate because project d5dd5b65d455452e9b60e680e11f36bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:30.069140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:30.070501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:30.073430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2666/5346] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:29 2022] DELETE /identity/v3/projects/d5dd5b65d455452e9b60e680e11f36bb => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:30.074741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.076486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.191524 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2681/5347] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:30.194483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46488f-51ee-4f40-aceb-277bab0976ba None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.195164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46488f-51ee-4f40-aceb-277bab0976ba None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.195450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46488f-51ee-4f40-aceb-277bab0976ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.221759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7f46488f-51ee-4f40-aceb-277bab0976ba None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:30.224454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7f46488f-51ee-4f40-aceb-277bab0976ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.244312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2340dfc-e239-464b-8428-e51d45ca20ac admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.257380 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2682/5348] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:30.257847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14db14-06e0-406d-9260-0e399207c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.258498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14db14-06e0-406d-9260-0e399207c0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.260178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14db14-06e0-406d-9260-0e399207c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.260468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14db14-06e0-406d-9260-0e399207c0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.261258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14db14-06e0-406d-9260-0e399207c0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.261477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b858d-0516-4730-b387-8da4d81f91e5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.262984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b858d-0516-4730-b387-8da4d81f91e5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.263337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b858d-0516-4730-b387-8da4d81f91e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.267698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14db14-06e0-406d-9260-0e399207c0f2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.306285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2667/5349] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:30.308487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8107442-f351-453e-959f-ee9bf26379bc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.308487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8107442-f351-453e-959f-ee9bf26379bc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.309085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8107442-f351-453e-959f-ee9bf26379bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.319290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-244b858d-0516-4730-b387-8da4d81f91e5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:30.321197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-244b858d-0516-4730-b387-8da4d81f91e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.344325 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2683/5350] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:30.347835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34281d24-f607-4497-8b42-bc3c221b47e9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.350284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34281d24-f607-4497-8b42-bc3c221b47e9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.350727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34281d24-f607-4497-8b42-bc3c221b47e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.367545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c8107442-f351-453e-959f-ee9bf26379bc None None] MFA Rules not processed for user `5d9931925a544aa894cf2b08cad10ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:30.385733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8107442-f351-453e-959f-ee9bf26379bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.391572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-34281d24-f607-4497-8b42-bc3c221b47e9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:30.393130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34281d24-f607-4497-8b42-bc3c221b47e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.411129 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2684/5351] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:30.412761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34281d24-f607-4497-8b42-bc3c221b47e9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.423967 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2668/5352] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:30.425417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34281d24-f607-4497-8b42-bc3c221b47e9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.430042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7b5fe-45f5-480b-b752-879468ce876e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.430372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7b5fe-45f5-480b-b752-879468ce876e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.430650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7b5fe-45f5-480b-b752-879468ce876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.435303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.436104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.436527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.436955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] PATH_INFO: `/v3/projects/48fd7a0d09244f8f95e8e4b19ef92b27/users/21ef42334a0741fba7747793f480a942/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.449773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:30.462512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e2d7b5fe-45f5-480b-b752-879468ce876e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:30.463756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e2d7b5fe-45f5-480b-b752-879468ce876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:30.474022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48fd7a0d09244f8f95e8e4b19ef92b27, user_id=21ef42334a0741fba7747793f480a942, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.482753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.486604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2669/5353] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:30.486604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7b5fe-45f5-480b-b752-879468ce876e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.514896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7b5fe-45f5-480b-b752-879468ce876e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.519764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2685/5354] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:30 2022] PUT /identity/v3/projects/48fd7a0d09244f8f95e8e4b19ef92b27/users/21ef42334a0741fba7747793f480a942/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:30.521591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.649083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.649911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.650296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.662021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.663781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:30.664173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.670734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.704853 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2670/5355] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:30.705797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.712485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146583b-64df-4fda-bca3-69113f5f3a95 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.720958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08459a7c-5d48-4f82-b43e-f3f4a56ebe0e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.730320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.730951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.731233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.731508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.732165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:30.732470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.734701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.734836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.735326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.735612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.735850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.745179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.747181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.769941 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2686/5356] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:30.772366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.777645 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2671/5357] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:30.782643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.789180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3e3e6-a933-4e3b-b2f2-b14e1066fa32 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.794876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.795570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.795856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.796126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.796765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.798864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.804683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4092df0-5b18-4c6d-91fc-738e2841da54 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:30.805664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:30.827870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:30.831545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:30.836357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:30.836887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:30.838402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:30.843813 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2687/5358] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/projects => generated 389 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:30.848292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:30.850318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:30.924050 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2672/5359] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:30.925229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.028511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33555b-e19b-477e-bca3-5c522507f3af admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.084853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a61256-fd16-4882-b6ca-8914ed2fa1df admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.097561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.098216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.099102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.099376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.107033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.114507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.162338 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2673/5360] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:30 2022] POST /identity/v3/users => generated 295 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:31.163183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.176882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.177367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.177743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.178417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] PATH_INFO: `/v3/projects/48fd7a0d09244f8f95e8e4b19ef92b27/users/21ef42334a0741fba7747793f480a942/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.181344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=48fd7a0d09244f8f95e8e4b19ef92b27, user_id=21ef42334a0741fba7747793f480a942, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.184152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.184357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57912d00-a0a9-425a-a98b-233a7a2d1b9a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.204668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878e157-44f9-4195-a47d-32965b4bb98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.205291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878e157-44f9-4195-a47d-32965b4bb98b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.205574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878e157-44f9-4195-a47d-32965b4bb98b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.205868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878e157-44f9-4195-a47d-32965b4bb98b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.206520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878e157-44f9-4195-a47d-32965b4bb98b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.206914 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2688/5361] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:30 2022] PUT /identity/v3/projects/48fd7a0d09244f8f95e8e4b19ef92b27/users/21ef42334a0741fba7747793f480a942/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:31.209687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.209863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878e157-44f9-4195-a47d-32965b4bb98b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.270898 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2674/5362] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:31 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:31.272192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56cba-0100-4815-a7d5-a87b2f93ba93 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.272626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56cba-0100-4815-a7d5-a87b2f93ba93 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.272741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56cba-0100-4815-a7d5-a87b2f93ba93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.338686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024041e-b2e1-4b2d-a0ce-cfe7a8cf4bef admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.344136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.344891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.345429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.345962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.346718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.349700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.366441 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2689/5363] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:31.367714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.370532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cce56cba-0100-4815-a7d5-a87b2f93ba93 None None] MFA Rules not processed for user `21ef42334a0741fba7747793f480a942`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:31.375306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cce56cba-0100-4815-a7d5-a87b2f93ba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:31.379385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c985f-9312-4e48-a0f2-1d953994aecb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.388032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.388814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.389281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.389657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.390440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.392174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.410068 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2690/5364] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:31.411426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.415104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:31.431904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:31.436047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:31.546406 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2675/5365] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:31.549938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56cba-0100-4815-a7d5-a87b2f93ba93 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.564848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b87bf7-f609-4762-b228-60bcd67773b0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.568766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56cba-0100-4815-a7d5-a87b2f93ba93 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.575267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.577573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.578044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.578449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.579210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.583444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.592536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.593215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.593713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.594061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.598880 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2676/5366] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:31.603232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.618266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9502c96-5542-48a5-9c79-e4c855a1ae06 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.636782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.637916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.638476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.638855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.639608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.641416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.656814 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2677/5367] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:31.663081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.666585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:31.751325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.755820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.759678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ff81c-a2a1-49db-9ea1-7f644a3283bb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.765174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.766992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.766992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.766992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.779256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.779461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2691/5368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:31.781110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.782636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.794934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2678/5369] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:31.795896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:31.806259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317ea9b-0b54-4e25-98a7-48cc53c137fd service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.814107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be87b46-2ca0-4fa2-ae3a-4eae50469132 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:31.815096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10eb83f-5d62-4357-a35e-36923bc0fb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.816075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb83f-5d62-4357-a35e-36923bc0fb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.816399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb83f-5d62-4357-a35e-36923bc0fb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.816877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10eb83f-5d62-4357-a35e-36923bc0fb15 admin admin] PATH_INFO: `/v3/projects/861e58c5bee84a44be5045e10b4594a1/users/86fa572828244998b44eb39d9330da7e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.827131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d4645-1911-4213-8798-9e4a1561d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:31.827903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4645-1911-4213-8798-9e4a1561d692 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.828931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4645-1911-4213-8798-9e4a1561d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.830520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4645-1911-4213-8798-9e4a1561d692 admin admin] PATH_INFO: `/v3/projects/f674c81f54b347f78af72cceb4a7d2f6/users/0147bc61c3bb4e6cbc3556599dfb7fa6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.839186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10eb83f-5d62-4357-a35e-36923bc0fb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861e58c5bee84a44be5045e10b4594a1, user_id=86fa572828244998b44eb39d9330da7e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.842167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10eb83f-5d62-4357-a35e-36923bc0fb15 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.876663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d4645-1911-4213-8798-9e4a1561d692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f674c81f54b347f78af72cceb4a7d2f6, user_id=0147bc61c3bb4e6cbc3556599dfb7fa6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:31.879971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d4645-1911-4213-8798-9e4a1561d692 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:31.883867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2692/5370] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:31 2022] PUT /identity/v3/projects/861e58c5bee84a44be5045e10b4594a1/users/86fa572828244998b44eb39d9330da7e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:31.888553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.889261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.890810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.917151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2679/5371] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:31 2022] PUT /identity/v3/projects/f674c81f54b347f78af72cceb4a7d2f6/users/0147bc61c3bb4e6cbc3556599dfb7fa6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:31.918935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b7f4-3e1e-425e-b7fc-cc883354a693 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:31.919431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b7f4-3e1e-425e-b7fc-cc883354a693 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:31.919855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b7f4-3e1e-425e-b7fc-cc883354a693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:31.957470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-edb5b7f4-3e1e-425e-b7fc-cc883354a693 None None] MFA Rules not processed for user `0147bc61c3bb4e6cbc3556599dfb7fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:31.959860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-edb5b7f4-3e1e-425e-b7fc-cc883354a693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:31.978111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:31.984951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:32.125958 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2693/5372] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:32.181568 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2680/5373] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:32.182424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:32.186102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:32.199864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5b7f4-3e1e-425e-b7fc-cc883354a693 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:32.221291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:32.234118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5b7f4-3e1e-425e-b7fc-cc883354a693 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:32.248164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:32.251725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:32.252051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:32.252396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:32.254373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:32.258336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:32.262947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:32.339360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e422e6-d260-4cba-8717-3325707c242f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:32.346139 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2681/5374] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:32 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:32.355670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:32.473104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:32.474682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:32.475130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:32.475559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:32.578000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e1b79-5fbb-41b5-bb15-540e0b1f50cd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:32.585244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:32.586124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:32.586770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:32.587096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:32.588403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:32.591621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:32.618109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:32.621427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:32.634820 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2682/5375] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:32 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:32.636122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:32.647829 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2694/5376] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:32.662931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ce6e-d357-4743-bf1e-7a256d64d9a2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:32.684896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:32.685526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:32.685813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:32.686120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:32.687838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:32.689472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:32.731075 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2683/5377] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:32.734943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:32.754392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2a3f-4140-49b6-9836-a0a0a3078333 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:32.769090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643986c-fce8-4f73-b72a-4e2a97066b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:32.770207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643986c-fce8-4f73-b72a-4e2a97066b22 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:32.770687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643986c-fce8-4f73-b72a-4e2a97066b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:32.771644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643986c-fce8-4f73-b72a-4e2a97066b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:32.772521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643986c-fce8-4f73-b72a-4e2a97066b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:32.774552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643986c-fce8-4f73-b72a-4e2a97066b22 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:32.805026 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2695/5378] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:32.807830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:32.824767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10268bf-2992-4c99-8ef7-56afacb94e1d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:32.831338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:32.832367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:32.833792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:32.834553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] PATH_INFO: `/v3/projects/8152c509f2b5457ba7f84975cbafb2f5/users/179d26ff670b4955bbb9932ec6395a74/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:32.873281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8152c509f2b5457ba7f84975cbafb2f5, user_id=179d26ff670b4955bbb9932ec6395a74, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:32.881165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:32.919194 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2684/5379] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:32 2022] PUT /identity/v3/projects/8152c509f2b5457ba7f84975cbafb2f5/users/179d26ff670b4955bbb9932ec6395a74/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:32.931512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e23f58-5a31-4bef-965e-5b357b80b11a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:32.931808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e23f58-5a31-4bef-965e-5b357b80b11a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:32.932079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e23f58-5a31-4bef-965e-5b357b80b11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.004339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-76e23f58-5a31-4bef-965e-5b357b80b11a None None] MFA Rules not processed for user `179d26ff670b4955bbb9932ec6395a74`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:33.005761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76e23f58-5a31-4bef-965e-5b357b80b11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:33.044644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.178339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeae45-0691-474e-b971-d678dc8f745c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.193384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.194669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2696/5380] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:33.198955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e23f58-5a31-4bef-965e-5b357b80b11a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.204011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.204500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.206378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] PATH_INFO: `/v3/users/f1164204a50547a18ead2921f6bc610e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.209175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1164204a50547a18ead2921f6bc610e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.226972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.227892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e23f58-5a31-4bef-965e-5b357b80b11a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.237443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.238258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.241304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.241680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.243059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.249494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.269138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:33.316003 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2697/5381] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:33 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:33.318007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.320898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:33.352517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1164204a50547a18ead2921f6bc610e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:33.373435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1164204a50547a18ead2921f6bc610e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:33.376919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1164204a50547a18ead2921f6bc610e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:33.407076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1164204a50547a18ead2921f6bc610e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:33.408267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9ca3a79-9258-49b7-9bc6-4d2db56608fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1164204a50547a18ead2921f6bc610e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:33.456992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf4219-64d3-4fa4-a3dd-f1e116d41c43 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.457591 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2685/5382] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:33 2022] DELETE /identity/v3/users/f1164204a50547a18ead2921f6bc610e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:33.463330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.463819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.464253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.488396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:33.490906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:33.532916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.534263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.534502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.534877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.536305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.538573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.573447 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2698/5383] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:33 2022] POST /identity/v3/users => generated 299 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:33.577601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.579557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:33.596527 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2686/5384] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:33.597569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d67a6-e39d-47bd-95d5-1230e4272964 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.606892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.608020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.608467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.609043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.610146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.614701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.619589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.621129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:33.638045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2699/5385] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:33.644050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.658703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfebb07-d869-4737-a5b0-408ebb86f803 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.666076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.667183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.667573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.667948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] PATH_INFO: `/v3/projects/7a263eb4177c4cfda32872c394380186/users/245d0957c1e04962827600bca8a80b60/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.694818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a263eb4177c4cfda32872c394380186, user_id=245d0957c1e04962827600bca8a80b60, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.698322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.721486 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2700/5386] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:33 2022] PUT /identity/v3/projects/7a263eb4177c4cfda32872c394380186/users/245d0957c1e04962827600bca8a80b60/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:33.725595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.775047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864317-06dc-4daf-968e-11f22e0194f1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.808017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:33.828182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6226f2-444f-4be7-926e-a5cd81a01a6f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.846940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.849934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.851399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.851847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.856002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.859298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.878467 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2701/5387] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:33.886859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:33.905229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0b1ea-b0d0-4d47-81f3-e9c6a70575a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:33.912629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd6015-b505-4abf-a780-d85713fe2ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.913988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd6015-b505-4abf-a780-d85713fe2ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.914444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd6015-b505-4abf-a780-d85713fe2ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.915110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd6015-b505-4abf-a780-d85713fe2ce6 admin admin] PATH_INFO: `/v3/projects/7a263eb4177c4cfda32872c394380186/users/245d0957c1e04962827600bca8a80b60/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.918610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd6015-b505-4abf-a780-d85713fe2ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a263eb4177c4cfda32872c394380186, user_id=245d0957c1e04962827600bca8a80b60, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.922447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd6015-b505-4abf-a780-d85713fe2ce6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.930411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:33.931188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.931289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.931571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.942077 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2702/5388] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:33 2022] PUT /identity/v3/projects/7a263eb4177c4cfda32872c394380186/users/245d0957c1e04962827600bca8a80b60/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:33.952683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d210727-49cc-4318-a325-4d750e225867 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:33.954037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d210727-49cc-4318-a325-4d750e225867 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:33.954387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d210727-49cc-4318-a325-4d750e225867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:33.959365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:33.979625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:33.992961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2687/5389] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:33.997856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0d210727-49cc-4318-a325-4d750e225867 None None] MFA Rules not processed for user `245d0957c1e04962827600bca8a80b60`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:33.999166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0d210727-49cc-4318-a325-4d750e225867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:34.122012 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2703/5390] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:34.132942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:34.135102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:34.162307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:34.266464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2394c85-f281-4060-b523-c4abc018eb33 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:34.342180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:34.343165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:34.343783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:34.344333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:34.346793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:34.450821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:34.455233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:34.467888 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2688/5391] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:35.197432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d210727-49cc-4318-a325-4d750e225867 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.204705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0d210727-49cc-4318-a325-4d750e225867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:35.479729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d210727-49cc-4318-a325-4d750e225867 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.506017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.511189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.521265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.521564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] PATH_INFO: `/v3/projects/8fa0fb77f348429a829126fcaa6870fd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.523848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa0fb77f348429a829126fcaa6870fd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.525962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:35.636224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa0fb77f348429a829126fcaa6870fd', 'request_id': 'req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:35.650668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa0fb77f348429a829126fcaa6870fd', 'request_id': 'req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:35.651811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:35.663977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa0fb77f348429a829126fcaa6870fd', 'request_id': 'req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:35.690443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] The token cache is being invalidate because project 8fa0fb77f348429a829126fcaa6870fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:35.690743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:35.694045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:35.697116 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2704/5392] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:35 2022] DELETE /identity/v3/projects/8fa0fb77f348429a829126fcaa6870fd => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:35.701583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:35.730477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061e02a-5274-491a-8239-6e97c25b1f7e service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:35.908194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:35.908998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:35.909405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:35.909949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] PATH_INFO: `/v3/users/21ef42334a0741fba7747793f480a942` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:35.911249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ef42334a0741fba7747793f480a942)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:35.913901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:36.029559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:36.074805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ef42334a0741fba7747793f480a942'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:36.087146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ef42334a0741fba7747793f480a942'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:36.087945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ef42334a0741fba7747793f480a942'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:36.100810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ef42334a0741fba7747793f480a942'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:36.119212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993d0326-843b-46cf-8df6-9f6cf5f92804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ef42334a0741fba7747793f480a942'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:36.172774 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2689/5393] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:35 2022] DELETE /identity/v3/users/21ef42334a0741fba7747793f480a942 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:36.600607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:36.604726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:36.732787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0adfeb-6dd8-4dd1-81c4-cad539a5e217 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:36.818635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:36.819850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:36.820502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:36.821352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:36.825223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:37.030434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:37.033117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:37.058130 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2705/5394] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:37.554434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98a38c-f2a8-4db1-8cd6-7fad88579547 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:37.555379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98a38c-f2a8-4db1-8cd6-7fad88579547 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:37.555379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98a38c-f2a8-4db1-8cd6-7fad88579547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:37.590133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1e98a38c-f2a8-4db1-8cd6-7fad88579547 None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:37.591547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e98a38c-f2a8-4db1-8cd6-7fad88579547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:37.777483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:37.778988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:37.816685 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2690/5395] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:37.996140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:38.008047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695c028-e8fd-48db-af73-2ba1eda3b1aa service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:38.018515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:38.019179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:38.019463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:38.019740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] PATH_INFO: `/v3/projects/48fd7a0d09244f8f95e8e4b19ef92b27` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:38.022575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48fd7a0d09244f8f95e8e4b19ef92b27)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:38.024541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:38.148725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48fd7a0d09244f8f95e8e4b19ef92b27', 'request_id': 'req-25aa56dc-9225-46bb-9448-2e01648c96a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:38.159142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48fd7a0d09244f8f95e8e4b19ef92b27', 'request_id': 'req-25aa56dc-9225-46bb-9448-2e01648c96a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:38.172386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48fd7a0d09244f8f95e8e4b19ef92b27', 'request_id': 'req-25aa56dc-9225-46bb-9448-2e01648c96a7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:38.200528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] The token cache is being invalidate because project 48fd7a0d09244f8f95e8e4b19ef92b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:38.200810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:38.202193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:38.205259 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2706/5396] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:37 2022] DELETE /identity/v3/projects/48fd7a0d09244f8f95e8e4b19ef92b27 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:38.211702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4b81e-abc4-43e1-ad02-8310d6aa68b5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:38.212000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4b81e-abc4-43e1-ad02-8310d6aa68b5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:38.212280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4b81e-abc4-43e1-ad02-8310d6aa68b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:38.241686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:38.243254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:38.253172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e1f4b81e-abc4-43e1-ad02-8310d6aa68b5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:38.256143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e1f4b81e-abc4-43e1-ad02-8310d6aa68b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:38.375225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa56dc-9225-46bb-9448-2e01648c96a7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:38.387316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:38.387956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:38.390380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:38.390676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:38.397140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:38.399660 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2691/5397] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:38.403701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f70fa-a558-46a0-bc24-c6217b6dd7ff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:38.404063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f70fa-a558-46a0-bc24-c6217b6dd7ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:38.404240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f70fa-a558-46a0-bc24-c6217b6dd7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:38.469622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9f0f70fa-a558-46a0-bc24-c6217b6dd7ff None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:38.472582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9f0f70fa-a558-46a0-bc24-c6217b6dd7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:38.485387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:38.486896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:38.495379 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2692/5398] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:38.499424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f70fa-a558-46a0-bc24-c6217b6dd7ff None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:38.512546 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2707/5399] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:07:38.516163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f70fa-a558-46a0-bc24-c6217b6dd7ff None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:38.527339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:38.527967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:38.529031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:38.529031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:38.529243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:38.529470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:38.531782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:38.555393 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2693/5400] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:38.560376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:38.575868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6d242-0a59-4510-99e4-3fb7a2e1ee26 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:38.597311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:38.602245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:38.605114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:38.605534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:38.607841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:38.611257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:38.701707 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2708/5401] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:38 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:38.706102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:38.721352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cb156-9d07-471e-be71-5ee7bd895029 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:38.805192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:38.900937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:38.902496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:38.902789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:38.905568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:38.908315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:38.916363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:38.979630 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2694/5402] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:38 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:38.983796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:39.012496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b70a3d-619a-4b23-9a0a-bf0e9f810876 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:39.024562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:39.025440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:39.025771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:39.026098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:39.026758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:39.035275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:39.062185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2709/5403] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:39.062954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:39.086327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab272699-e7f5-492f-8324-855fbd62a973 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:39.096865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92159b5-739d-4bbc-9312-cc0cc553dbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:39.099677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92159b5-739d-4bbc-9312-cc0cc553dbec admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:39.101551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92159b5-739d-4bbc-9312-cc0cc553dbec admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:39.101968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92159b5-739d-4bbc-9312-cc0cc553dbec admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:39.102847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92159b5-739d-4bbc-9312-cc0cc553dbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:39.106970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92159b5-739d-4bbc-9312-cc0cc553dbec admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:39.157644 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2695/5404] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:39.163743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:39.184773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:39.190214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74f94f-e6d3-4529-80fc-26a35c3046f1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:39.205046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:39.205774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:39.206242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:39.206532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] PATH_INFO: `/v3/projects/8f28fa6f3f1a47c69fb5931ad9803148/users/500d8f7d16c74678a8eb62e76d262692/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:39.260577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f28fa6f3f1a47c69fb5931ad9803148, user_id=500d8f7d16c74678a8eb62e76d262692, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:39.271332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:39.316784 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2710/5405] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:39 2022] PUT /identity/v3/projects/8f28fa6f3f1a47c69fb5931ad9803148/users/500d8f7d16c74678a8eb62e76d262692/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:39.318858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746b98b-f1d2-4ef0-a10a-ade82e74e91f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:39.321992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746b98b-f1d2-4ef0-a10a-ade82e74e91f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:39.322552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746b98b-f1d2-4ef0-a10a-ade82e74e91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:39.371907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7746b98b-f1d2-4ef0-a10a-ade82e74e91f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:39.376014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7746b98b-f1d2-4ef0-a10a-ade82e74e91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:39.524808 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2696/5406] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:39.555336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:39.557087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:39.718923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362bdcb-1b5e-4cb9-a4a7-83e98ec0e626 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:39.728669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:39.735582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:39.747662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:39.748055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:39.770491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:39.772056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:39.791064 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2711/5407] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:43.420750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:43.422662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:43.423949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:43.424835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:43.428609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:43.471311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ee6cb-bd78-43e2-8800-cbd8c1dc3689 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:43.487178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:43.491264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:43.492219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:43.494068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] PATH_INFO: `/v3/users/0147bc61c3bb4e6cbc3556599dfb7fa6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:43.497188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0147bc61c3bb4e6cbc3556599dfb7fa6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:43.502018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:43.521229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] MFA Rules not processed for user `500d8f7d16c74678a8eb62e76d262692`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:43.524307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:43.624937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0147bc61c3bb4e6cbc3556599dfb7fa6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:43.654387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0147bc61c3bb4e6cbc3556599dfb7fa6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:43.671159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0147bc61c3bb4e6cbc3556599dfb7fa6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:43.701974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0147bc61c3bb4e6cbc3556599dfb7fa6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:43.703293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea40b7b8-fb0c-4327-8051-e1d6e4b8c243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0147bc61c3bb4e6cbc3556599dfb7fa6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:43.715903 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2712/5408] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:43 2022] DELETE /identity/v3/users/0147bc61c3bb4e6cbc3556599dfb7fa6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:43.721652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:43.722102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:43.722488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:43.750975 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2697/5409] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:43.755976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:43.757419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:43.769648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:43.771331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:43.778106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14095d6-3e61-4967-ac43-9123ca5fb85f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:43.827147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:43.839564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:43.855695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:43.856308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:43.856490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:43.856757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:43.857421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:43.860139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:43.875965 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2713/5410] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:43.893394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:43.897732 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2698/5411] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:43 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:43.898764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:43.901675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:43.995421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5662ec3-50a5-481c-887a-553f48229d57 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:44.000724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1af6f-0f0d-4dbb-a6d2-0aa7ccc5cc15 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:44.000887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:44.002000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.002827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.003981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.005320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:44.007934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.044065 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2699/5412] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:43 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:44.047860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:44.061256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede582dd-94df-4ec1-89fc-ced4ba0987ca admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:44.066960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:44.067764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.068148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.069056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.069212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:44.071056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.085482 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2700/5413] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:44.089299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:44.095977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:44.096716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.097554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.098131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.102888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800814a-9235-495e-a1e8-820d03398889 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:44.108328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:44.109086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.109439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.110877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] PATH_INFO: `/v3/projects/6cca3ea9cfe04ce58b99bf83b531cd9b/users/da69f7dc6a43499cb6d61f98af257923/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.118648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:44.121402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.145597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cca3ea9cfe04ce58b99bf83b531cd9b, user_id=da69f7dc6a43499cb6d61f98af257923, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:44.149618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.152148 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2714/5414] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:44.172022 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2701/5415] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:44 2022] PUT /identity/v3/projects/6cca3ea9cfe04ce58b99bf83b531cd9b/users/da69f7dc6a43499cb6d61f98af257923/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:07:44.178559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:44.216042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff1d38-439d-4741-be33-fa2558a2fb88 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:44.348938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:44.350255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.350605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.351254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.353361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:44.358720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.383617 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2715/5416] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:44.388130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:44.399251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3f165-779f-4af7-a4a7-c89f3f9e9b5a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:44.404967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:44.406129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:44.406129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:44.406320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] PATH_INFO: `/v3/projects/6cca3ea9cfe04ce58b99bf83b531cd9b/users/da69f7dc6a43499cb6d61f98af257923/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:44.408400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cca3ea9cfe04ce58b99bf83b531cd9b, user_id=da69f7dc6a43499cb6d61f98af257923, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:44.412246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:44.425623 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2702/5417] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:44 2022] PUT /identity/v3/projects/6cca3ea9cfe04ce58b99bf83b531cd9b/users/da69f7dc6a43499cb6d61f98af257923/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:45.456710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:45.459100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:45.512772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6982392-0412-42e0-be79-cd31b5f2f1da admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:45.620248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:45.639475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:45.640325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:45.640665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:45.641042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] PATH_INFO: `/v3/projects/f674c81f54b347f78af72cceb4a7d2f6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:45.642435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f674c81f54b347f78af72cceb4a7d2f6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:45.645376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:45.717816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f674c81f54b347f78af72cceb4a7d2f6', 'request_id': 'req-41e6ca2d-9d82-41f6-ba87-44ff499fd604'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.725401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f674c81f54b347f78af72cceb4a7d2f6', 'request_id': 'req-41e6ca2d-9d82-41f6-ba87-44ff499fd604'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.733368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f674c81f54b347f78af72cceb4a7d2f6', 'request_id': 'req-41e6ca2d-9d82-41f6-ba87-44ff499fd604'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.755380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] The token cache is being invalidate because project f674c81f54b347f78af72cceb4a7d2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:45.755719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.756512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:45.761043 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2716/5418] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:45 2022] DELETE /identity/v3/projects/f674c81f54b347f78af72cceb4a7d2f6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:45.762382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:45.764129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:45.785131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a979f-4232-4e58-bb9b-c4323c19f2b8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:45.868858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:45.870531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:45.870665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:45.871020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] PATH_INFO: `/v3/users/179d26ff670b4955bbb9932ec6395a74` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:45.878195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=179d26ff670b4955bbb9932ec6395a74)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:45.880778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:45.895750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:45.996312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '179d26ff670b4955bbb9932ec6395a74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:46.012264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '179d26ff670b4955bbb9932ec6395a74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:46.013156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '179d26ff670b4955bbb9932ec6395a74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:46.025047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '179d26ff670b4955bbb9932ec6395a74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:46.025923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '179d26ff670b4955bbb9932ec6395a74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:46.058618 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2703/5419] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:45 2022] DELETE /identity/v3/users/179d26ff670b4955bbb9932ec6395a74 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:47.391564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:47.405214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6ca2d-9d82-41f6-ba87-44ff499fd604 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:47.500801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:47.501765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:47.502245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:47.502679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] PATH_INFO: `/v3/projects/8152c509f2b5457ba7f84975cbafb2f5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:47.504093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8152c509f2b5457ba7f84975cbafb2f5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:47.507497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:47.589179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8152c509f2b5457ba7f84975cbafb2f5', 'request_id': 'req-e3255401-e026-45a4-bb0d-1394e37a714d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.596438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8152c509f2b5457ba7f84975cbafb2f5', 'request_id': 'req-e3255401-e026-45a4-bb0d-1394e37a714d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.604197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8152c509f2b5457ba7f84975cbafb2f5', 'request_id': 'req-e3255401-e026-45a4-bb0d-1394e37a714d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.622908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] The token cache is being invalidate because project 8152c509f2b5457ba7f84975cbafb2f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:47.623200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.623855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e3255401-e026-45a4-bb0d-1394e37a714d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.625601 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2717/5420] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:47 2022] DELETE /identity/v3/projects/8152c509f2b5457ba7f84975cbafb2f5 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:47.629954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:47.631603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:47.648958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecde04-ddb1-48e4-874a-79a138fef330 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:47.743033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:47.744009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:47.744703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:47.745287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] PATH_INFO: `/v3/users/245d0957c1e04962827600bca8a80b60` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:47.746670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=245d0957c1e04962827600bca8a80b60)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:47.750685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:47.845655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245d0957c1e04962827600bca8a80b60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.855405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245d0957c1e04962827600bca8a80b60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.856528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245d0957c1e04962827600bca8a80b60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.868699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245d0957c1e04962827600bca8a80b60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.869656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec718f9b-86f8-4135-9f7c-e8a979317bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245d0957c1e04962827600bca8a80b60'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:47.894616 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2704/5421] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:07:47 2022] DELETE /identity/v3/users/245d0957c1e04962827600bca8a80b60 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:48.482923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:48.483465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:48.483465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:48.528994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:48.546838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] MFA Rules not processed for user `da69f7dc6a43499cb6d61f98af257923`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:48.548115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:48.664593 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2718/5422] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:48.676530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f3fe4-8fe6-420e-ad6c-c9ec662b323d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:48.677265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f3fe4-8fe6-420e-ad6c-c9ec662b323d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:48.677265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f3fe4-8fe6-420e-ad6c-c9ec662b323d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:48.688880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:48.709085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f93f3fe4-8fe6-420e-ad6c-c9ec662b323d None None] MFA Rules not processed for user `500d8f7d16c74678a8eb62e76d262692`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:48.711286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f93f3fe4-8fe6-420e-ad6c-c9ec662b323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:48.830030 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2705/5423] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:48.884926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:48.886933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:49.015517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3362b-e7a5-4369-bde1-2392b7887545 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.027953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.029770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.030843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.031208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] PATH_INFO: `/v3/projects/7a263eb4177c4cfda32872c394380186` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.033087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a263eb4177c4cfda32872c394380186)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.036418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.108768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a263eb4177c4cfda32872c394380186', 'request_id': 'req-6db2f9a3-541a-41bf-b249-2b492895f9c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:49.115212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a263eb4177c4cfda32872c394380186', 'request_id': 'req-6db2f9a3-541a-41bf-b249-2b492895f9c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:49.122860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a263eb4177c4cfda32872c394380186', 'request_id': 'req-6db2f9a3-541a-41bf-b249-2b492895f9c2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:49.148690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] The token cache is being invalidate because project 7a263eb4177c4cfda32872c394380186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:07:49.150242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:49.151569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:07:49.154522 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2719/5424] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:07:48 2022] DELETE /identity/v3/projects/7a263eb4177c4cfda32872c394380186 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:49.159467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2c8d-b860-44bc-9125-3f6bfe256087 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.159764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2c8d-b860-44bc-9125-3f6bfe256087 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.160043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2c8d-b860-44bc-9125-3f6bfe256087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.184354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a62c2c8d-b860-44bc-9125-3f6bfe256087 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:49.189329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a62c2c8d-b860-44bc-9125-3f6bfe256087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:49.304183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:49.305872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:49.325227 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2706/5425] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:49.325755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d254-fd40-4c56-8ee2-703d5caaab94 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.328091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d254-fd40-4c56-8ee2-703d5caaab94 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.328383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d254-fd40-4c56-8ee2-703d5caaab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.360250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-87e7d254-fd40-4c56-8ee2-703d5caaab94 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:49.362580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-87e7d254-fd40-4c56-8ee2-703d5caaab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:49.370128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:49.382068 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2707/5426] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:49.383113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d254-fd40-4c56-8ee2-703d5caaab94 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:49.412617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d254-fd40-4c56-8ee2-703d5caaab94 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.424559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.425185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.425472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.425748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.429360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:07:49.429674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.432319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.463618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2f9a3-541a-41bf-b249-2b492895f9c2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.469934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2708/5427] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:07:49.471240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:49.479025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.480388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.480388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.480645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.482342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:49.484937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70b75-0cd5-4c44-8e52-32378e9fdd34 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.490696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.491627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.492729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.492965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.493729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.496766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.542744 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2709/5428] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:49 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:49.547645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:49.616055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.618319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.635184 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2720/5429] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:49.678448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9c9d-ceae-4f78-a97c-f33c18783e12 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.686676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.687580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.688109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.688575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.689398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.692817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.720400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:49.730004 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2710/5430] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:49 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:49.733554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:49.745892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5865c92-3346-429e-8011-ac96c530fed5 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.751704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.752332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.752616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.752892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.753532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.756167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.771232 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2721/5431] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:49.772777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:49.783673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92a958-579f-48b4-85f1-521a819481cd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.790192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cb72b-434f-4040-a15b-53f7fe6ab7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.790909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb72b-434f-4040-a15b-53f7fe6ab7dc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.791259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb72b-434f-4040-a15b-53f7fe6ab7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.791600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cb72b-434f-4040-a15b-53f7fe6ab7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.792317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cb72b-434f-4040-a15b-53f7fe6ab7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.794110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cb72b-434f-4040-a15b-53f7fe6ab7dc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.809038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2711/5432] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:07:49.812221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:49.824069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06e7e3-071e-4e71-a8e9-5464bf366b5c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:49.829027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:49.829778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.830158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.830501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] PATH_INFO: `/v3/projects/93ab0bcb23a241f8b22a244c4a765692/users/34e86b1dcc0a4fbe9f0d379f2acce9d3/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.851847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ab0bcb23a241f8b22a244c4a765692, user_id=34e86b1dcc0a4fbe9f0d379f2acce9d3, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:49.854968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:49.873420 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2722/5433] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:49 2022] PUT /identity/v3/projects/93ab0bcb23a241f8b22a244c4a765692/users/34e86b1dcc0a4fbe9f0d379f2acce9d3/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:49.884771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe46a-dfec-4ee9-8ed3-9b91eea312c7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:49.885982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe46a-dfec-4ee9-8ed3-9b91eea312c7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:49.886847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fe46a-dfec-4ee9-8ed3-9b91eea312c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:49.933898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-353fe46a-dfec-4ee9-8ed3-9b91eea312c7 None None] MFA Rules not processed for user `34e86b1dcc0a4fbe9f0d379f2acce9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:49.935755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-353fe46a-dfec-4ee9-8ed3-9b91eea312c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:50.103233 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2712/5434] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:50.104052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:50.272860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30814f79-8f4d-40be-9466-ec5b155dcd19 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:50.278219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:50.279230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:50.279723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:50.280151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:50.281119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:50.283256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:50.318696 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2723/5435] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:50 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:07:50.322822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe46a-dfec-4ee9-8ed3-9b91eea312c7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:50.341197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fe46a-dfec-4ee9-8ed3-9b91eea312c7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:50.484531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:50.485222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:50.485514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:50.485793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:50.486749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:50.491345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:50.555155 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2713/5436] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:50 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:50.559048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:50.575182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831097a-2d28-4e74-bfed-f2ef7df1b087 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:50.587148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:50.588230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:50.590298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:50.590298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:50.591304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:50.595464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:50.625383 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2724/5437] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:50.629451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:50.645455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259693-726f-47f7-988c-1ed12e90058e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:50.659495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad551d9-5ac9-46e2-8015-09f2532adc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:50.660800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad551d9-5ac9-46e2-8015-09f2532adc4e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:50.663233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad551d9-5ac9-46e2-8015-09f2532adc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:50.663661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad551d9-5ac9-46e2-8015-09f2532adc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:50.665013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad551d9-5ac9-46e2-8015-09f2532adc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:50.669728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad551d9-5ac9-46e2-8015-09f2532adc4e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:50.691638 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2714/5438] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:50.695812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:50.701901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:50.711239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad3b01-b777-4652-8e06-c763a9ec8402 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:50.726542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:50.727486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:50.727729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:50.728106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] PATH_INFO: `/v3/projects/e430f571a82a4cf08e1de1186a2f67c8/users/f877f2da10454f3d8aaa802516e47558/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:50.752184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e430f571a82a4cf08e1de1186a2f67c8, user_id=f877f2da10454f3d8aaa802516e47558, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:50.756003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:50.781005 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2725/5439] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:50 2022] PUT /identity/v3/projects/e430f571a82a4cf08e1de1186a2f67c8/users/f877f2da10454f3d8aaa802516e47558/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:50.791659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dcda-7b63-4612-954b-3dd2c51794ac None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:50.792895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dcda-7b63-4612-954b-3dd2c51794ac None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:50.793407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dcda-7b63-4612-954b-3dd2c51794ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:50.861095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1852dcda-7b63-4612-954b-3dd2c51794ac None None] MFA Rules not processed for user `f877f2da10454f3d8aaa802516e47558`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:50.865201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1852dcda-7b63-4612-954b-3dd2c51794ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:50.978640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:51.044991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2715/5440] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:51.046772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:51.179171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48286719-88db-44ce-8ed3-efc8af45459b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:51.193385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:51.194339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.194710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.195048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.195806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:51.205073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:51.265640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2726/5441] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:51 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:51.269512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dcda-7b63-4612-954b-3dd2c51794ac None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:51.291478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dcda-7b63-4612-954b-3dd2c51794ac None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:51.399916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:51.400558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.400839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.401166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.402056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:51.413225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:51.466373 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2716/5442] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:51 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:51.470577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:51.483697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce5c74-f81c-44d1-a0d5-ed9af3730795 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:51.489813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:51.490729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.491717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.492358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.493363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:51.506778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:51.526828 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2727/5443] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:51.530104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:51.546454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbef06-fcfc-41e1-9b3d-a879f6d53f2f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:51.562837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7a03c-2137-4270-9ccc-befcddaa01f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:51.563539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7a03c-2137-4270-9ccc-befcddaa01f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.563827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7a03c-2137-4270-9ccc-befcddaa01f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.564106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7a03c-2137-4270-9ccc-befcddaa01f7 admin admin] PATH_INFO: `/v3/projects/56f05afa3cd641719914f65269f6e53f/users/d6e024115add404aa22c96601d8d5839/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.596513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7a03c-2137-4270-9ccc-befcddaa01f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f05afa3cd641719914f65269f6e53f, user_id=d6e024115add404aa22c96601d8d5839, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:51.602637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7a03c-2137-4270-9ccc-befcddaa01f7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:51.630572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2717/5444] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:51 2022] PUT /identity/v3/projects/56f05afa3cd641719914f65269f6e53f/users/d6e024115add404aa22c96601d8d5839/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:51.634405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:51.655367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.655557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.655837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.663767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda7d8-3457-40ea-b078-89134b277ae1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:51.740938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] MFA Rules not processed for user `86fa572828244998b44eb39d9330da7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:51.742981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:51.809136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:51.809749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.810245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.810688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.811498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:51.813430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:51.871115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2728/5445] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:51.872029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:51.899144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88600ee2-9421-46db-81ae-50d70e42edbb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:51.902133 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2718/5446] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:07:51.910147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:51.912334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:51.919280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29bd6f-a88f-4bdb-81eb-7fbff213b769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:51.919966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29bd6f-a88f-4bdb-81eb-7fbff213b769 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.920249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29bd6f-a88f-4bdb-81eb-7fbff213b769 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.920527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29bd6f-a88f-4bdb-81eb-7fbff213b769 admin admin] PATH_INFO: `/v3/projects/56f05afa3cd641719914f65269f6e53f/users/d6e024115add404aa22c96601d8d5839/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.926707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29bd6f-a88f-4bdb-81eb-7fbff213b769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f05afa3cd641719914f65269f6e53f, user_id=d6e024115add404aa22c96601d8d5839, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:51.929433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29bd6f-a88f-4bdb-81eb-7fbff213b769 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:51.931167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96ef75-8611-47ca-85c6-4b7275f44c3d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:51.949613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:51.952003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.952770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.954763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.965652 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2729/5447] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:51 2022] PUT /identity/v3/projects/56f05afa3cd641719914f65269f6e53f/users/d6e024115add404aa22c96601d8d5839/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:51.967001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:51.967491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:51.967788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:51.968068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:51.972749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:52.038834 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2719/5448] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:07:51 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:52.042896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:52.045289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] MFA Rules not processed for user `d6e024115add404aa22c96601d8d5839`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:07:52.046718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:52.100040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:52.104333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:52.236066 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2730/5449] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:07:52.249230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:52.250905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:52.271550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9fa42-7492-46fa-bb48-db30402babcd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:52.281697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:52.284425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:52.284425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:52.284425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:52.284425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:52.293199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:52.340142 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2720/5450] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:07:52 2022] POST /identity/v3/users => generated 301 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:07:52.344570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:52.366688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654ae86-2524-4964-af92-d0128a74651c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:52.377667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:52.379211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:52.380058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:52.380341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:52.383821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:52.387190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c62b63-912a-4e2c-9ddb-0551965f22d6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:52.389746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:52.400833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:52.401481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:52.401767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:52.402100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:52.414708 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2721/5451] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:52.421682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:52.448363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912b65c-84e4-4987-a922-d8644f9ae321 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:52.460725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:52.464956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:52.465283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:52.465729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] PATH_INFO: `/v3/projects/ca87d496227d4cf4bccf1bdd2b7aeeae/users/d14a68a648cf45e1b929d0abbf039b8a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:52.513956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca87d496227d4cf4bccf1bdd2b7aeeae, user_id=d14a68a648cf45e1b929d0abbf039b8a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:52.517358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:52.557242 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2722/5452] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:52 2022] PUT /identity/v3/projects/ca87d496227d4cf4bccf1bdd2b7aeeae/users/d14a68a648cf45e1b929d0abbf039b8a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:52.563735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:52.568503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:52.570798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:52.596137 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2731/5453] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:52.697096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726873e-0f61-4bd2-b90a-c17eae78e8a4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:52.703621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:52.705110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:52.705605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:52.707053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:52.708487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:52.710656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:52.733755 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2723/5454] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:07:52.739716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:52.756280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889827b-ac02-44f9-86e8-cc3aed62ab00 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:52.762014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:52.762933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:52.763380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:52.763809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] PATH_INFO: `/v3/projects/ca87d496227d4cf4bccf1bdd2b7aeeae/users/d14a68a648cf45e1b929d0abbf039b8a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:52.766068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca87d496227d4cf4bccf1bdd2b7aeeae, user_id=d14a68a648cf45e1b929d0abbf039b8a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:52.769768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:52.785927 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2732/5455] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:07:52 2022] PUT /identity/v3/projects/ca87d496227d4cf4bccf1bdd2b7aeeae/users/d14a68a648cf45e1b929d0abbf039b8a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:07:54.654696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:54.656586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:07:54.855030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95397a-44d0-441a-a184-d6cfa89d5fca admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:54.920083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:55.005310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:55.005961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:55.006252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:55.006508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:55.143129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:55.143879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:55.181049 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2724/5456] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:07:56.379900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:07:56.400823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d4cf5-ba08-4ee8-a8b4-64834112620c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:07:56.407478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbbfd7-6db8-4c83-bb7d-fb75a17d49ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:07:56.408955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbbfd7-6db8-4c83-bb7d-fb75a17d49ca service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:07:56.411771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbbfd7-6db8-4c83-bb7d-fb75a17d49ca service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:07:56.415061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbbfd7-6db8-4c83-bb7d-fb75a17d49ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:07:56.546237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:07:56.556532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbbfd7-6db8-4c83-bb7d-fb75a17d49ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:07:56.562507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbbfd7-6db8-4c83-bb7d-fb75a17d49ca service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:07:56.580057 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2733/5457] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:07:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:01.542837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:01.626132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475cbbc-48f9-4748-9326-9afeaffa039d service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:01.631890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:01.632647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:01.633046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:01.634178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] PATH_INFO: `/v3/users/34e86b1dcc0a4fbe9f0d379f2acce9d3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:01.636459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e86b1dcc0a4fbe9f0d379f2acce9d3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:01.638555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:01.711136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e86b1dcc0a4fbe9f0d379f2acce9d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:01.721021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e86b1dcc0a4fbe9f0d379f2acce9d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:01.721893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e86b1dcc0a4fbe9f0d379f2acce9d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:01.732040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e86b1dcc0a4fbe9f0d379f2acce9d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:01.733236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e86b1dcc0a4fbe9f0d379f2acce9d3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:01.749726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:01.760691 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2725/5458] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:01 2022] DELETE /identity/v3/users/34e86b1dcc0a4fbe9f0d379f2acce9d3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:01.762561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:01.763026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:01.764565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:01.792071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:01.793620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:01.886357 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2734/5459] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:01.938441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:01.941050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:02.114628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dae46-e17e-4157-a7c6-48fe986ccb36 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:02.119308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:02.119989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:02.120277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:02.120552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:02.133461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:02.136720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:02.152312 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2726/5460] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:03.337662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:03.339769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:03.358723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcac31b-9825-4145-a3fb-0fa1e9ce6515 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:03.371975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:03.372898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:03.373390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:03.373872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] PATH_INFO: `/v3/projects/93ab0bcb23a241f8b22a244c4a765692` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:03.376347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ab0bcb23a241f8b22a244c4a765692)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:03.378544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:03.468640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ab0bcb23a241f8b22a244c4a765692', 'request_id': 'req-8944f3b3-3e80-4855-9faa-1500dc10a35b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.477775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ab0bcb23a241f8b22a244c4a765692', 'request_id': 'req-8944f3b3-3e80-4855-9faa-1500dc10a35b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.489099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ab0bcb23a241f8b22a244c4a765692', 'request_id': 'req-8944f3b3-3e80-4855-9faa-1500dc10a35b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.520947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] The token cache is being invalidate because project 93ab0bcb23a241f8b22a244c4a765692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:03.521247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.522395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.529027 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2735/5461] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:03 2022] DELETE /identity/v3/projects/93ab0bcb23a241f8b22a244c4a765692 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:03.533496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:03.548223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58df6-fd95-4f85-b4ee-12fc2e29f120 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:03.653600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:03.655673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:03.655673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:03.656683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] PATH_INFO: `/v3/users/f877f2da10454f3d8aaa802516e47558` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:03.658130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f877f2da10454f3d8aaa802516e47558)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:03.660542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:03.739573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f877f2da10454f3d8aaa802516e47558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.749941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f877f2da10454f3d8aaa802516e47558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.751052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f877f2da10454f3d8aaa802516e47558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.763937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f877f2da10454f3d8aaa802516e47558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.766131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f877f2da10454f3d8aaa802516e47558'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:03.796098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2727/5462] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:03 2022] DELETE /identity/v3/users/f877f2da10454f3d8aaa802516e47558 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:05.531691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:05.533412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:05.577793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f3b3-3e80-4855-9faa-1500dc10a35b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:05.711936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:05.712757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:05.713136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:05.713528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] PATH_INFO: `/v3/projects/e430f571a82a4cf08e1de1186a2f67c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:05.716132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e430f571a82a4cf08e1de1186a2f67c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:05.718589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:05.789179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:05.852968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e430f571a82a4cf08e1de1186a2f67c8', 'request_id': 'req-79b18909-681c-4d9f-938d-666e3c0637f3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:05.861392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e430f571a82a4cf08e1de1186a2f67c8', 'request_id': 'req-79b18909-681c-4d9f-938d-666e3c0637f3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:05.884454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e430f571a82a4cf08e1de1186a2f67c8', 'request_id': 'req-79b18909-681c-4d9f-938d-666e3c0637f3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:05.922495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] The token cache is being invalidate because project e430f571a82a4cf08e1de1186a2f67c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:05.922803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:05.924579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:05.933577 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2736/5463] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:05 2022] DELETE /identity/v3/projects/e430f571a82a4cf08e1de1186a2f67c8 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:05.934237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:05.936569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:05.964352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe2662-0d26-4337-8498-a7311c4202b0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:06.001937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:06.120097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:06.121283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:06.121591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:06.123870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] PATH_INFO: `/v3/users/d6e024115add404aa22c96601d8d5839` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:06.125643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e024115add404aa22c96601d8d5839)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:06.134290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:06.211597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e024115add404aa22c96601d8d5839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.221631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e024115add404aa22c96601d8d5839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.223096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e024115add404aa22c96601d8d5839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.244684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e024115add404aa22c96601d8d5839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.245506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e7095128-b020-4aca-b61c-e10d4f681dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e024115add404aa22c96601d8d5839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:06.282983 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2728/5464] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:05 2022] DELETE /identity/v3/users/d6e024115add404aa22c96601d8d5839 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:07.061246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.064946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:07.083089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b18909-681c-4d9f-938d-666e3c0637f3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.192788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.193466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.193740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.194148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] PATH_INFO: `/v3/projects/56f05afa3cd641719914f65269f6e53f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.195245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f05afa3cd641719914f65269f6e53f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.200933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:07.272015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f05afa3cd641719914f65269f6e53f', 'request_id': 'req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.279241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f05afa3cd641719914f65269f6e53f', 'request_id': 'req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.286762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f05afa3cd641719914f65269f6e53f', 'request_id': 'req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.305338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] The token cache is being invalidate because project 56f05afa3cd641719914f65269f6e53f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:07.306366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.307120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7390ca2-77cb-49e8-91a3-33cb19dbd5cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:07.312510 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2737/5465] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:07 2022] DELETE /identity/v3/projects/56f05afa3cd641719914f65269f6e53f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:07.315402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc33ac-cdee-47a7-9b94-53bd3b1e7ee8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.315950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc33ac-cdee-47a7-9b94-53bd3b1e7ee8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.316601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc33ac-cdee-47a7-9b94-53bd3b1e7ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.385490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-85fc33ac-cdee-47a7-9b94-53bd3b1e7ee8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:07.387651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-85fc33ac-cdee-47a7-9b94-53bd3b1e7ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:07.501449 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2729/5466] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:07.503450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efd284-d497-4e13-acfa-f00791ab20cf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.503836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efd284-d497-4e13-acfa-f00791ab20cf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.504155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efd284-d497-4e13-acfa-f00791ab20cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.539617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d5efd284-d497-4e13-acfa-f00791ab20cf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:07.541729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d5efd284-d497-4e13-acfa-f00791ab20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:07.555148 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2738/5467] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:08:07.559680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc33ac-cdee-47a7-9b94-53bd3b1e7ee8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.570275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc33ac-cdee-47a7-9b94-53bd3b1e7ee8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.575113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.575798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.576153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.576504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.577227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:07.577608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.579367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:07.597079 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2730/5468] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:08:07.599309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efd284-d497-4e13-acfa-f00791ab20cf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.609985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efd284-d497-4e13-acfa-f00791ab20cf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.615914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.616773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.616965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.617312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.630228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.636240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:07.699775 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2739/5469] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:07 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:08:07.700760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.730794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82aadc-7568-4ca1-9713-139b50c9ce38 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.782918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:07.842659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.843404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.843918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.844264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.845325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.847995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:07.885899 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2731/5470] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:07 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:07.887291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.902069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac3b2d-1991-4d93-bdc2-2312d7674c4b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.914285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.915460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.915954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.916155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.916793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.920672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:07.934393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:07.947804 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2740/5471] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:07.949519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:07.964693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926de5d9-9d2e-4e3d-8242-6cb0d3720f9b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:07.982617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88301d73-441e-40ab-92c3-bd85d0fcf5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:07.983557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88301d73-441e-40ab-92c3-bd85d0fcf5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:07.984060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88301d73-441e-40ab-92c3-bd85d0fcf5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:07.984572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88301d73-441e-40ab-92c3-bd85d0fcf5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:07.985617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88301d73-441e-40ab-92c3-bd85d0fcf5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:07.987676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88301d73-441e-40ab-92c3-bd85d0fcf5f2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.025968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.033396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2732/5472] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:08.039202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a10fab-e5f2-47d8-8d1e-202d25c07bf1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.044498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.045305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.045606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.045929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] PATH_INFO: `/v3/projects/e82a0bb2cc874614b0061f4c71ce082d/users/2c96909ab4eb449d81769b4c1a52ec88/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.065040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82a0bb2cc874614b0061f4c71ce082d, user_id=2c96909ab4eb449d81769b4c1a52ec88, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.068863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.091208 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2741/5473] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:08 2022] PUT /identity/v3/projects/e82a0bb2cc874614b0061f4c71ce082d/users/2c96909ab4eb449d81769b4c1a52ec88/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:08:08.096447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1916af-9c70-482c-b0c1-48534643c507 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.096758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1916af-9c70-482c-b0c1-48534643c507 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.097026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1916af-9c70-482c-b0c1-48534643c507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.118941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1f1916af-9c70-482c-b0c1-48534643c507 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:08.120116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1f1916af-9c70-482c-b0c1-48534643c507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:08.188071 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2733/5474] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:08.221640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:08.223941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:08.321516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bb144-62fb-46e3-902f-ff0b0147511e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:08.327600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:08.328383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.328939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.329324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.366926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:08.368404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:08.396899 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2742/5475] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:08.805517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11936-3466-417f-9c44-76cc17818479 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:08.806428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11936-3466-417f-9c44-76cc17818479 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:08.806882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11936-3466-417f-9c44-76cc17818479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:08.857065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-01a11936-3466-417f-9c44-76cc17818479 None None] MFA Rules not processed for user `d14a68a648cf45e1b929d0abbf039b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:08.858322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-01a11936-3466-417f-9c44-76cc17818479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:09.020572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2734/5476] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:09.033790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:09.035641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:09.148237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50981a-9bb6-4535-879b-76406dce629b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:09.153672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a8e3e-2745-4196-bdec-f74e0eaeadad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:09.154714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a8e3e-2745-4196-bdec-f74e0eaeadad service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:09.155148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a8e3e-2745-4196-bdec-f74e0eaeadad service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:09.156115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a8e3e-2745-4196-bdec-f74e0eaeadad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:09.158581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e60a8e3e-2745-4196-bdec-f74e0eaeadad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:09.341311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a8e3e-2745-4196-bdec-f74e0eaeadad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:09.342794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a8e3e-2745-4196-bdec-f74e0eaeadad service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:09.356737 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2743/5477] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:09.889114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a11936-3466-417f-9c44-76cc17818479 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:09.890784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-01a11936-3466-417f-9c44-76cc17818479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:09.924323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a11936-3466-417f-9c44-76cc17818479 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:09.943095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:09.946959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:09.947358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:09.947555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:09.950473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:10.015549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:10.132458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:10.134315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:10.157196 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2735/5478] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:10.492411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917c82-dcc2-4fa7-964c-1f042419bdce None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:10.492880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917c82-dcc2-4fa7-964c-1f042419bdce None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:10.493116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13917c82-dcc2-4fa7-964c-1f042419bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:10.582279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-13917c82-dcc2-4fa7-964c-1f042419bdce None None] MFA Rules not processed for user `2c96909ab4eb449d81769b4c1a52ec88`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:10.586607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-13917c82-dcc2-4fa7-964c-1f042419bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:10.631682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:10.760240 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2744/5479] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:10.777459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:10.801667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1762-91ac-472c-a9aa-fef72e14ceea service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:10.808999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:10.809637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:10.810352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:10.810918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:10.832633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:10.836161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:10.849874 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2736/5480] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:11.891181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13917c82-dcc2-4fa7-964c-1f042419bdce None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:11.907390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13917c82-dcc2-4fa7-964c-1f042419bdce None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:11.914098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276eaafe-585d-42c5-908b-4f6ddfebc19b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:11.914998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eaafe-585d-42c5-908b-4f6ddfebc19b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:11.915450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eaafe-585d-42c5-908b-4f6ddfebc19b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:11.915870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eaafe-585d-42c5-908b-4f6ddfebc19b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:11.920296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-276eaafe-585d-42c5-908b-4f6ddfebc19b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:11.939954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276eaafe-585d-42c5-908b-4f6ddfebc19b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:11.942729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276eaafe-585d-42c5-908b-4f6ddfebc19b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:11.983792 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2745/5481] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:15.199998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:15.201483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:15.233580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748ad38-e0d9-48f4-80c2-6a6c7b792e29 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:15.243774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:15.244439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:15.244975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:15.245263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:15.251598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:15.442384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f3ba5-2fe8-4be0-99eb-258c56f54f58 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:15.442984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f3ba5-2fe8-4be0-99eb-258c56f54f58 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:15.443068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f3ba5-2fe8-4be0-99eb-258c56f54f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:15.460196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:15.461815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:15.475705 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2737/5482] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:15.491628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b74f3ba5-2fe8-4be0-99eb-258c56f54f58 None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:15.493518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b74f3ba5-2fe8-4be0-99eb-258c56f54f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:15.599242 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2746/5483] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:08:25.469523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=56 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:25.580643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=48 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:28.964032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:28.966528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:28.994094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc828a49-4cef-4ebd-99cf-bbc0004016fd service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:29.004898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:29.006657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:29.007107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:29.007383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] PATH_INFO: `/v3/users/9546792be5a640e2a59fb2e8481c6c4b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:29.008652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9546792be5a640e2a59fb2e8481c6c4b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:29.010650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:29.094040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9546792be5a640e2a59fb2e8481c6c4b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.105063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9546792be5a640e2a59fb2e8481c6c4b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.109044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9546792be5a640e2a59fb2e8481c6c4b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.121739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9546792be5a640e2a59fb2e8481c6c4b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.123488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-dcb9a6c7-5f5c-4830-8a89-097aadedbac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9546792be5a640e2a59fb2e8481c6c4b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.158118 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2738/5484] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:28 2022] DELETE /identity/v3/users/9546792be5a640e2a59fb2e8481c6c4b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:29.411757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f3ba5-2fe8-4be0-99eb-258c56f54f58 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:29.459270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f3ba5-2fe8-4be0-99eb-258c56f54f58 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:29.632260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:29.634440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:29.635563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:29.636015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] PATH_INFO: `/v3/projects/ceebf89c39014750ae279e0741a0bb79` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:29.637767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceebf89c39014750ae279e0741a0bb79)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:29.651485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:29.786914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceebf89c39014750ae279e0741a0bb79', 'request_id': 'req-b39e1f7f-5113-4609-b799-9bac89543a87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.799170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceebf89c39014750ae279e0741a0bb79', 'request_id': 'req-b39e1f7f-5113-4609-b799-9bac89543a87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.809144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceebf89c39014750ae279e0741a0bb79', 'request_id': 'req-b39e1f7f-5113-4609-b799-9bac89543a87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.859628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] The token cache is being invalidate because project ceebf89c39014750ae279e0741a0bb79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:29.859924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.861335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b39e1f7f-5113-4609-b799-9bac89543a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:29.870874 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2747/5485] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:29 2022] DELETE /identity/v3/projects/ceebf89c39014750ae279e0741a0bb79 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:29.872560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a66c5-3797-4efd-a00b-939901c5d239 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:29.872854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a66c5-3797-4efd-a00b-939901c5d239 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:29.873137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a66c5-3797-4efd-a00b-939901c5d239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:29.930229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-582a66c5-3797-4efd-a00b-939901c5d239 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:29.935992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-582a66c5-3797-4efd-a00b-939901c5d239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:30.077992 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2739/5486] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:30.080750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855721a-3458-47b8-9b95-3a6951c83070 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.081299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855721a-3458-47b8-9b95-3a6951c83070 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.081711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855721a-3458-47b8-9b95-3a6951c83070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:30.138666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c855721a-3458-47b8-9b95-3a6951c83070 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:30.149365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c855721a-3458-47b8-9b95-3a6951c83070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:30.165598 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2748/5487] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:30.179151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a66c5-3797-4efd-a00b-939901c5d239 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:30.210549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a66c5-3797-4efd-a00b-939901c5d239 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:30.221302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:30.222838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.223233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.223786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:30.224825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:30.226799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:30.228795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:30.261854 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2740/5488] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:08:30.267136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855721a-3458-47b8-9b95-3a6951c83070 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:30.284709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855721a-3458-47b8-9b95-3a6951c83070 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:30.309212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:30.310571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.317513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.318052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:30.318761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:30.324495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:30.471553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:30.473439 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2749/5489] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:30 2022] POST /identity/v3/projects => generated 365 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:30.494035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470b5e4-f65a-40c1-b6ab-6a1cb705b24b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:30.618782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:30.619411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.619696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.620029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:30.620680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:30.622844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:30.631944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:30.668584 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2741/5490] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:30 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:08:30.672724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:30.690385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2ebdd-9f52-46db-94ba-1a4244922ec7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:30.702847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:30.704357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.704881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.705386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:30.708910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:30.711185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:30.740986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:30.742923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2750/5491] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:30.755450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4833c-d0ce-4115-9469-f3358564eb0e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:30.763663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930c568-4ce2-41ea-93cd-ad6b32d40676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:30.764391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c568-4ce2-41ea-93cd-ad6b32d40676 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.764674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c568-4ce2-41ea-93cd-ad6b32d40676 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.764946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930c568-4ce2-41ea-93cd-ad6b32d40676 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:30.765586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930c568-4ce2-41ea-93cd-ad6b32d40676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:30.767285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930c568-4ce2-41ea-93cd-ad6b32d40676 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:30.784537 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2742/5492] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:30.789581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:30.807980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32694f5f-97ac-4c6b-9e82-9603dc4b0637 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:30.852421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:30.854313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.854313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.857348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] PATH_INFO: `/v3/projects/7f09210a6e9f45cc9619b180628ba208/users/1147de803ef34d16b39e96720b2f638b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:30.903858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f09210a6e9f45cc9619b180628ba208, user_id=1147de803ef34d16b39e96720b2f638b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:30.906939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:30.947050 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2751/5493] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:30 2022] PUT /identity/v3/projects/7f09210a6e9f45cc9619b180628ba208/users/1147de803ef34d16b39e96720b2f638b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:30.953198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:30.954127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:30.954412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:31.005137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:31.012767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:31.033779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:31.036481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:31.128980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:31.184865 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2743/5494] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:31.209231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:31.211093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:31.219842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5993265-4d96-4974-a8d7-5b0e1ae296c4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:31.228614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:31.231081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:31.231591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:31.232022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] PATH_INFO: `/v3/users/5d9931925a544aa894cf2b08cad10ea2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:31.234255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9931925a544aa894cf2b08cad10ea2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:31.237735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:31.332868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ae17b-6d97-4bbd-9fc1-8cf449e5abba None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:31.335288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9931925a544aa894cf2b08cad10ea2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:31.341430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:31.343284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:31.343840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:31.344269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:31.359283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:31.361655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:31.372680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9931925a544aa894cf2b08cad10ea2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:31.377516 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2744/5495] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:08:31.423503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9931925a544aa894cf2b08cad10ea2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:31.443307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9931925a544aa894cf2b08cad10ea2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:31.444614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1f41dd3-310b-41ae-8f0f-9dbc627d2380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9931925a544aa894cf2b08cad10ea2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:31.456476 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2752/5496] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:31 2022] DELETE /identity/v3/users/5d9931925a544aa894cf2b08cad10ea2 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:32.337473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:32.349708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3f43f-77e1-41ea-b44c-0f3b953afe50 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:32.511383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:32.512180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:32.513428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:32.513428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] PATH_INFO: `/v3/projects/e2d0a21cdb7149ca9b72a502c7eb84fa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:32.520919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d0a21cdb7149ca9b72a502c7eb84fa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:32.526082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:32.582305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:32.650240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d0a21cdb7149ca9b72a502c7eb84fa', 'request_id': 'req-339a3eea-7dfd-417f-9848-688160b4b36f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:32.661212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d0a21cdb7149ca9b72a502c7eb84fa', 'request_id': 'req-339a3eea-7dfd-417f-9848-688160b4b36f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:32.674568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d0a21cdb7149ca9b72a502c7eb84fa', 'request_id': 'req-339a3eea-7dfd-417f-9848-688160b4b36f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:32.705793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] The token cache is being invalidate because project e2d0a21cdb7149ca9b72a502c7eb84fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:32.706348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:32.711414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-339a3eea-7dfd-417f-9848-688160b4b36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:32.716219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2745/5497] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:32 2022] DELETE /identity/v3/projects/e2d0a21cdb7149ca9b72a502c7eb84fa => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:32.722407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3afd3e-c030-4d05-9411-f4502968d677 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:32.723889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3afd3e-c030-4d05-9411-f4502968d677 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:32.724177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3afd3e-c030-4d05-9411-f4502968d677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:32.768579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8f3afd3e-c030-4d05-9411-f4502968d677 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:32.770781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8f3afd3e-c030-4d05-9411-f4502968d677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:32.919850 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2753/5498] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:32.928980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ec42f-1543-43a0-a8c5-35bec5e2746c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:32.929269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ec42f-1543-43a0-a8c5-35bec5e2746c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:32.929555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ec42f-1543-43a0-a8c5-35bec5e2746c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:32.956687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5c2ec42f-1543-43a0-a8c5-35bec5e2746c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:32.958670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5c2ec42f-1543-43a0-a8c5-35bec5e2746c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:32.985865 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2746/5499] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:32.990944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3afd3e-c030-4d05-9411-f4502968d677 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:33.002429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3afd3e-c030-4d05-9411-f4502968d677 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:33.008766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1cc2c-8c78-44d7-b299-8ccd96aff8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:33.009951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1cc2c-8c78-44d7-b299-8ccd96aff8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.010446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1cc2c-8c78-44d7-b299-8ccd96aff8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.010879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1cc2c-8c78-44d7-b299-8ccd96aff8a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.011803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1cc2c-8c78-44d7-b299-8ccd96aff8a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:33.012271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1cc2c-8c78-44d7-b299-8ccd96aff8a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:33.014146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1cc2c-8c78-44d7-b299-8ccd96aff8a7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:33.029349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2754/5500] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:08:33.033409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ec42f-1543-43a0-a8c5-35bec5e2746c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:33.054167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ec42f-1543-43a0-a8c5-35bec5e2746c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:33.061429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:33.062495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.062913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.063416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.064318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:33.067960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:33.092681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f091f-3b1f-45e9-b02f-d287626a0f91 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.093596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f091f-3b1f-45e9-b02f-d287626a0f91 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.094128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f091f-3b1f-45e9-b02f-d287626a0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.143134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-989f091f-3b1f-45e9-b02f-d287626a0f91 None None] MFA Rules not processed for user `86fa572828244998b44eb39d9330da7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:33.145598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-989f091f-3b1f-45e9-b02f-d287626a0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:33.173449 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2747/5501] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:33 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:33.174788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:33.194114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:33.252668 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2755/5502] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:33.256940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad75d8-1a2e-48fc-8b84-10b9defd3f69 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.257428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad75d8-1a2e-48fc-8b84-10b9defd3f69 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.257948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad75d8-1a2e-48fc-8b84-10b9defd3f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.286795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d9ad75d8-1a2e-48fc-8b84-10b9defd3f69 None None] MFA Rules not processed for user `86fa572828244998b44eb39d9330da7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:33.294393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d9ad75d8-1a2e-48fc-8b84-10b9defd3f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:33.307622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f966b6-128c-4b8a-b257-0468adb45789 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:33.323099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:33.324010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.324586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.324945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.327225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:33.332720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:33.380583 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2748/5503] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:33 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:33.382115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:33.415755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc844c-af19-490d-bee0-af072c33e732 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:33.431250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:33.433298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.433695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.434314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.435083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:33.439576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:33.450010 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2756/5504] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:33.487036 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2749/5505] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:33.487977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad75d8-1a2e-48fc-8b84-10b9defd3f69 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:33.506962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad75d8-1a2e-48fc-8b84-10b9defd3f69 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:33.517723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:33.518738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.519024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.519306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.520012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:33.521537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:33.560973 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2757/5506] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:08:33.563639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:33.583468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364be45f-d5eb-47c4-9ed8-f60186bdcff5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:33.593397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0043d1-ddc7-4d2e-9eb2-ff578fd67f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:33.595394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0043d1-ddc7-4d2e-9eb2-ff578fd67f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.595685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0043d1-ddc7-4d2e-9eb2-ff578fd67f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.595961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0043d1-ddc7-4d2e-9eb2-ff578fd67f94 admin admin] PATH_INFO: `/v3/projects/862a718cd74e4668a4fc9559c9d9d470/users/c882de4b1ee547e79f86dc21a4a01b6d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.629039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:33.631458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:33.660354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0043d1-ddc7-4d2e-9eb2-ff578fd67f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862a718cd74e4668a4fc9559c9d9d470, user_id=c882de4b1ee547e79f86dc21a4a01b6d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:33.662003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b1afd-401c-4097-b15d-ee6f6531cc7e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:33.667418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0043d1-ddc7-4d2e-9eb2-ff578fd67f94 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:33.684105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:33.684853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.685154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.704560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] PATH_INFO: `/v3/users/92c2bfd90e2d47759dfd0eb47c0f03d0` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.707881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c2bfd90e2d47759dfd0eb47c0f03d0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:33.710780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:33.712590 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2750/5507] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:33 2022] PUT /identity/v3/projects/862a718cd74e4668a4fc9559c9d9d470/users/c882de4b1ee547e79f86dc21a4a01b6d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:08:33.718243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:33.718699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:33.719086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:33.795199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:33.805108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:33.886809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c2bfd90e2d47759dfd0eb47c0f03d0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:33.922134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:33.935248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c2bfd90e2d47759dfd0eb47c0f03d0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:33.961190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c2bfd90e2d47759dfd0eb47c0f03d0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:33.985384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c2bfd90e2d47759dfd0eb47c0f03d0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:33.987109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1948062e-8e2a-4de0-bd5c-0c397e054f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c2bfd90e2d47759dfd0eb47c0f03d0'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:34.007108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2758/5508] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:33 2022] DELETE /identity/v3/users/92c2bfd90e2d47759dfd0eb47c0f03d0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:34.011620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04995d99-8abf-4c75-8417-f74560dad260 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:34.012179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04995d99-8abf-4c75-8417-f74560dad260 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:34.012558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04995d99-8abf-4c75-8417-f74560dad260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:34.040957 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2751/5509] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:34.063347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:34.065472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:34.068968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-04995d99-8abf-4c75-8417-f74560dad260 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:34.070531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-04995d99-8abf-4c75-8417-f74560dad260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:34.147982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:34.229005 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2759/5510] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:34.263320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04995d99-8abf-4c75-8417-f74560dad260 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:34.291207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c0fb-86c0-453b-9121-57659f8b0817 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:34.383747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04995d99-8abf-4c75-8417-f74560dad260 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:34.392883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:34.394121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:34.394690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:34.395263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:34.421559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:34.426670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:34.478406 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2760/5511] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:34.698392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:34.702217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:34.703325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:34.703863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:34.736131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:34.745798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:34.777448 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2752/5512] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:35.765356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:35.766853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:35.787627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67646e8a-86cf-4c18-a435-984b0e980675 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:35.800525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:35.801187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:35.801472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:35.801756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] PATH_INFO: `/v3/projects/cb56c9f0633c49d5932c58e5745bcf06` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:35.804361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb56c9f0633c49d5932c58e5745bcf06)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:35.810705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:35.989490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb56c9f0633c49d5932c58e5745bcf06', 'request_id': 'req-f4243b9f-eb46-4da4-a76d-97a10e705b48'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.017637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb56c9f0633c49d5932c58e5745bcf06', 'request_id': 'req-f4243b9f-eb46-4da4-a76d-97a10e705b48'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.039548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb56c9f0633c49d5932c58e5745bcf06', 'request_id': 'req-f4243b9f-eb46-4da4-a76d-97a10e705b48'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.095250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] The token cache is being invalidate because project cb56c9f0633c49d5932c58e5745bcf06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:36.095557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.096570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4243b9f-eb46-4da4-a76d-97a10e705b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.103717 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2761/5513] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:35 2022] DELETE /identity/v3/projects/cb56c9f0633c49d5932c58e5745bcf06 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:36.106938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:36.130831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0c0eb-b995-4520-9bd6-144917ee40cc service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:36.415878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:36.418168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:36.418452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:36.418726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] PATH_INFO: `/v3/users/57ae018d888d41f9856c9e6e0c18d9fb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:36.424932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ae018d888d41f9856c9e6e0c18d9fb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:36.436885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:36.629303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:36.639166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:36.639166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:36.639469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:36.656133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ae018d888d41f9856c9e6e0c18d9fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.675039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ae018d888d41f9856c9e6e0c18d9fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.676066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ae018d888d41f9856c9e6e0c18d9fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.690266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ae018d888d41f9856c9e6e0c18d9fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.691214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ae018d888d41f9856c9e6e0c18d9fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:36.717558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:36.730006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:36.766181 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2753/5514] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:36 2022] DELETE /identity/v3/users/57ae018d888d41f9856c9e6e0c18d9fb => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:36.887978 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2762/5515] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:08:38.919934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:38.921449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:39.123945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf780b-2ffb-4e18-bb32-825e735249f8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:39.138090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:39.138748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:39.143689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:39.143689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] PATH_INFO: `/v3/projects/f4ba3f5ffde64bee8922f85919aa7bfd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:39.144274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:39.144274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:39.148934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ba3f5ffde64bee8922f85919aa7bfd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:39.159969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:39.160789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:39.329547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ba3f5ffde64bee8922f85919aa7bfd', 'request_id': 'req-257bcaf3-8829-44c4-84c0-15fcc2d90d89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:39.337552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ba3f5ffde64bee8922f85919aa7bfd', 'request_id': 'req-257bcaf3-8829-44c4-84c0-15fcc2d90d89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:39.351368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ba3f5ffde64bee8922f85919aa7bfd', 'request_id': 'req-257bcaf3-8829-44c4-84c0-15fcc2d90d89'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:39.377435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] The token cache is being invalidate because project f4ba3f5ffde64bee8922f85919aa7bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:39.379904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:39.380836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-257bcaf3-8829-44c4-84c0-15fcc2d90d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:39.384742 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2754/5516] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:38 2022] DELETE /identity/v3/projects/f4ba3f5ffde64bee8922f85919aa7bfd => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:39.389354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215bd1-ba89-4963-9c64-c0a89e085aec None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:39.389874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215bd1-ba89-4963-9c64-c0a89e085aec None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:39.390310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215bd1-ba89-4963-9c64-c0a89e085aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:39.395783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3ad05-bcc4-426d-94f9-04f1ab0838f6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:39.423224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-05215bd1-ba89-4963-9c64-c0a89e085aec None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:39.424829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-05215bd1-ba89-4963-9c64-c0a89e085aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:39.484712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:39.511215 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2755/5517] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:08:39.519186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae8fad-f128-4104-b6af-bee22a20da87 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:39.519573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae8fad-f128-4104-b6af-bee22a20da87 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:39.520488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae8fad-f128-4104-b6af-bee22a20da87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:39.552046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-16ae8fad-f128-4104-b6af-bee22a20da87 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:39.553536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-16ae8fad-f128-4104-b6af-bee22a20da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:39.580577 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2756/5518] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:39.584028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae8fad-f128-4104-b6af-bee22a20da87 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:39.599276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:39.599919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:39.600035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae8fad-f128-4104-b6af-bee22a20da87 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:39.600197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:39.600476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:39.603162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:39.608242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:39.608979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:39.609360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:39.609763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:39.611418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:39.612184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:39.614964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:39.631941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2757/5519] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:08:39.634222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:39.636157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:39.637517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:39.650140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b95627-eaf5-412d-b3d6-e8e10c4478b9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:39.661746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:39.662772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:39.663186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:39.663595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:39.663863 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2763/5520] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:08:39.664346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:39.675226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:39.736341 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2758/5521] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:39 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:39.738658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:39.760574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b8f7c-9ffc-45b0-a9c5-01158561fe4f service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:39.912537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:39.913329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:39.913617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:39.913958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:39.914755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:39.926267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:40.020446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:40.026666 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2764/5522] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:39 2022] POST /identity/v3/users => generated 295 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:08:40.040884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add2e26-d31d-45e6-b0a8-a59549ecf2d2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:40.058355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:40.059004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:40.060652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:40.060944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:40.061639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:40.065241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:40.100212 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2759/5523] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:40.101927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:40.124727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926573-a685-45a3-b3e4-72e0421416b2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:40.154961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74a529-3297-4339-b217-5f39afaf7d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:40.163384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a529-3297-4339-b217-5f39afaf7d5b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:40.165415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a529-3297-4339-b217-5f39afaf7d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:40.166266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a529-3297-4339-b217-5f39afaf7d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:40.167341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74a529-3297-4339-b217-5f39afaf7d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:40.169302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74a529-3297-4339-b217-5f39afaf7d5b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:40.191718 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2765/5524] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:40.196938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:40.216128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f47d08-e708-4c5a-b0d7-4f073ffcb6a0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:40.232669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:40.234065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:40.239963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:40.240524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] PATH_INFO: `/v3/projects/1829a58d2c73430e9a86f65534de9df4/users/ec4533181f17478280294f70befba7da/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:40.286816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1829a58d2c73430e9a86f65534de9df4, user_id=ec4533181f17478280294f70befba7da, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:40.298876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:40.363486 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2760/5525] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:40 2022] PUT /identity/v3/projects/1829a58d2c73430e9a86f65534de9df4/users/ec4533181f17478280294f70befba7da/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:40.368780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e45c4-6b13-4ed1-81fc-b746a4af380d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:40.369374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e45c4-6b13-4ed1-81fc-b746a4af380d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:40.369891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e45c4-6b13-4ed1-81fc-b746a4af380d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:40.500623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-293e45c4-6b13-4ed1-81fc-b746a4af380d None None] MFA Rules not processed for user `ec4533181f17478280294f70befba7da`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:40.516162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-293e45c4-6b13-4ed1-81fc-b746a4af380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:40.555516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:40.756711 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2766/5526] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:40.762427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:40.966715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feac78-d023-492f-b914-b1b97b341a43 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:40.982123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:40.983256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:40.983551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:40.984085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:40.984743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:40.989862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:41.095241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2761/5527] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:40 2022] POST /identity/v3/projects => generated 391 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:41.098562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e45c4-6b13-4ed1-81fc-b746a4af380d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:41.125934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e45c4-6b13-4ed1-81fc-b746a4af380d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:41.335305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:41.336011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:41.336212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:41.336486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:41.342618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:41.362733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:41.446726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:41.453303 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2767/5528] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:41 2022] POST /identity/v3/users => generated 302 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:08:41.461689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ab2c0-2b9b-47e2-b5f1-d8c95e661abe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:41.490942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:41.491578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:41.492416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:41.492416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:41.492792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:41.494446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:41.553940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:41.555457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2762/5529] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:41.568346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cc49e-b87c-4658-8c54-479f12f7b387 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:41.597183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:41.597908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:41.598188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:41.598470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] PATH_INFO: `/v3/projects/b4a8fed172a24aaf83c8721e0f1d1552/users/54fc44f7b9834f008e8a36586e876f45/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:41.646980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a8fed172a24aaf83c8721e0f1d1552, user_id=54fc44f7b9834f008e8a36586e876f45, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:41.649956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:41.694461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2768/5530] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:41 2022] PUT /identity/v3/projects/b4a8fed172a24aaf83c8721e0f1d1552/users/54fc44f7b9834f008e8a36586e876f45/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:08:41.698325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:41.709560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ef28d-c291-42a4-8bd6-c4434a20d2e4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:41.895825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:41.970048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df477f-0497-4fdf-8533-6f3d2bcb614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:41.977461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df477f-0497-4fdf-8533-6f3d2bcb614b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:41.977461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df477f-0497-4fdf-8533-6f3d2bcb614b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:41.977461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df477f-0497-4fdf-8533-6f3d2bcb614b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:41.977461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df477f-0497-4fdf-8533-6f3d2bcb614b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:41.977461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df477f-0497-4fdf-8533-6f3d2bcb614b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:42.032458 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2763/5531] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:08:42.036962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:42.058429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a410832-32be-4bbb-a669-421c3c4f4fdf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:42.075828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:42.078442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:42.084365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:42.085362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] PATH_INFO: `/v3/projects/b4a8fed172a24aaf83c8721e0f1d1552/users/54fc44f7b9834f008e8a36586e876f45/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:42.091697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a8fed172a24aaf83c8721e0f1d1552, user_id=54fc44f7b9834f008e8a36586e876f45, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:42.099734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:42.146107 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2769/5532] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:42 2022] PUT /identity/v3/projects/b4a8fed172a24aaf83c8721e0f1d1552/users/54fc44f7b9834f008e8a36586e876f45/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:42.156827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a518c-380a-49b0-8691-064262b7561c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:42.157749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a518c-380a-49b0-8691-064262b7561c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:42.158202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a518c-380a-49b0-8691-064262b7561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:42.264583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-db9a518c-380a-49b0-8691-064262b7561c None None] MFA Rules not processed for user `54fc44f7b9834f008e8a36586e876f45`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:42.267660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-db9a518c-380a-49b0-8691-064262b7561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:42.642313 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2764/5533] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:42.691954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:42.694790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:42.933594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b51fd9-ff02-4d26-a55c-5b18eda4b7bd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:42.934645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a518c-380a-49b0-8691-064262b7561c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:42.942344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eb771-5a03-4cb5-af99-f97160e6ce40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:42.943276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eb771-5a03-4cb5-af99-f97160e6ce40 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:42.943276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eb771-5a03-4cb5-af99-f97160e6ce40 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:42.943883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eb771-5a03-4cb5-af99-f97160e6ce40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:42.957847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a518c-380a-49b0-8691-064262b7561c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:43.000256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370eb771-5a03-4cb5-af99-f97160e6ce40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:43.003246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370eb771-5a03-4cb5-af99-f97160e6ce40 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:43.005642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:43.006762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:43.007200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:43.007679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:43.020966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:43.059909 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2770/5534] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:43.262501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:43.264073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:43.302673 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2765/5535] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:48.518212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:48.518713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:48.518851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:48.531551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:48.588744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] MFA Rules not processed for user `1147de803ef34d16b39e96720b2f638b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:48.590974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:48.828429 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2771/5536] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:48.842754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:48.844318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:48.885199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2de162-dac7-4a9e-b63f-83b563bd1169 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:48.906288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:48.907604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:48.908110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:48.909207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:48.938696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:48.941581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:48.964334 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2766/5537] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:49.147985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:49.150202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:49.155041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:49.178588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0b879-c061-4136-9750-4dbb17521f7d service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:49.184672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747276f-342e-47f0-be68-9045da6fa0f0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:49.194096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a646b-fd60-4e46-9933-dfb5051ef845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:49.195350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a646b-fd60-4e46-9933-dfb5051ef845 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:49.196029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373e637-7294-4439-9de8-7d42cd3fe0a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:49.196985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a646b-fd60-4e46-9933-dfb5051ef845 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:49.197450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a646b-fd60-4e46-9933-dfb5051ef845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:49.197768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e637-7294-4439-9de8-7d42cd3fe0a5 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:49.198282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e637-7294-4439-9de8-7d42cd3fe0a5 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:49.199512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e637-7294-4439-9de8-7d42cd3fe0a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:49.202136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bd3a646b-fd60-4e46-9933-dfb5051ef845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:49.204792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9373e637-7294-4439-9de8-7d42cd3fe0a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:49.240973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a646b-fd60-4e46-9933-dfb5051ef845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:49.242706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a646b-fd60-4e46-9933-dfb5051ef845 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:49.248415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373e637-7294-4439-9de8-7d42cd3fe0a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:49.254659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373e637-7294-4439-9de8-7d42cd3fe0a5 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:49.270834 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2772/5538] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:49.277316 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2767/5539] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:50.049781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:50.050288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:50.050401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:50.119759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] MFA Rules not processed for user `2c96909ab4eb449d81769b4c1a52ec88`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:50.121919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:50.360888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2773/5540] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:50.395209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85daa7-8c51-4c5a-866c-4a27131d0c3a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:50.396264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85daa7-8c51-4c5a-866c-4a27131d0c3a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:50.396783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85daa7-8c51-4c5a-866c-4a27131d0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:50.446614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5b85daa7-8c51-4c5a-866c-4a27131d0c3a None None] MFA Rules not processed for user `c882de4b1ee547e79f86dc21a4a01b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:50.448233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b85daa7-8c51-4c5a-866c-4a27131d0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:50.477736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:50.629848 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2768/5541] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:51.308124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:51.312041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:51.504126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8919ead-b0fa-4481-9031-fa668014b768 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:51.514537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:51.516154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:51.516442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:51.516734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] PATH_INFO: `/v3/users/500d8f7d16c74678a8eb62e76d262692` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:51.522152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500d8f7d16c74678a8eb62e76d262692)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:51.524570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:51.623623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:51.646307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85daa7-8c51-4c5a-866c-4a27131d0c3a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:51.665137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85daa7-8c51-4c5a-866c-4a27131d0c3a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:51.674622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:51.676707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:51.677373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:51.677929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:51.688930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:51.691740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:51.698274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500d8f7d16c74678a8eb62e76d262692'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:51.706426 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2769/5542] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:51.764325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500d8f7d16c74678a8eb62e76d262692'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:51.791785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500d8f7d16c74678a8eb62e76d262692'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:51.805467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500d8f7d16c74678a8eb62e76d262692'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:51.806265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500d8f7d16c74678a8eb62e76d262692'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:51.833031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2774/5543] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:51 2022] DELETE /identity/v3/users/500d8f7d16c74678a8eb62e76d262692 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:52.479216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:52.510277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8766-2c42-4d02-930e-7967924730c1 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:52.661168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:52.663553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:52.666263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:52.666569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] PATH_INFO: `/v3/projects/8f28fa6f3f1a47c69fb5931ad9803148` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:52.668375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f28fa6f3f1a47c69fb5931ad9803148)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:52.676358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:52.823224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f28fa6f3f1a47c69fb5931ad9803148', 'request_id': 'req-09b49992-be3b-4890-bf1e-ebb527f74d58'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:52.835622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f28fa6f3f1a47c69fb5931ad9803148', 'request_id': 'req-09b49992-be3b-4890-bf1e-ebb527f74d58'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:52.866328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f28fa6f3f1a47c69fb5931ad9803148', 'request_id': 'req-09b49992-be3b-4890-bf1e-ebb527f74d58'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:52.897658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] The token cache is being invalidate because project 8f28fa6f3f1a47c69fb5931ad9803148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:52.898571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:52.899505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:52.903135 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2770/5544] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:52 2022] DELETE /identity/v3/projects/8f28fa6f3f1a47c69fb5931ad9803148 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:52.906907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:52.908350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:52.935135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5df4b-93a1-4448-b626-f7fc071c7ee1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:53.193741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:53.197691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:53.198025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:53.198437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] PATH_INFO: `/v3/users/da69f7dc6a43499cb6d61f98af257923` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:53.209725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=da69f7dc6a43499cb6d61f98af257923)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:53.218520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:53.315430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da69f7dc6a43499cb6d61f98af257923'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.348190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da69f7dc6a43499cb6d61f98af257923'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.363032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da69f7dc6a43499cb6d61f98af257923'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.379398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da69f7dc6a43499cb6d61f98af257923'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.380343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e08b0366-3301-4b69-b6f2-b7086684d2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da69f7dc6a43499cb6d61f98af257923'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.391922 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2775/5545] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:52 2022] DELETE /identity/v3/users/da69f7dc6a43499cb6d61f98af257923 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:53.621942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:53.623400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:53.652913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b49992-be3b-4890-bf1e-ebb527f74d58 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:53.732609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:53.810107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:53.812089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:53.812599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:53.813214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] PATH_INFO: `/v3/users/ec4533181f17478280294f70befba7da` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:53.814789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4533181f17478280294f70befba7da)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:53.817095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:53.915286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4533181f17478280294f70befba7da'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.934491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4533181f17478280294f70befba7da'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.935798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4533181f17478280294f70befba7da'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.983922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4533181f17478280294f70befba7da'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:53.986136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4533181f17478280294f70befba7da'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:54.026219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2771/5546] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:53 2022] DELETE /identity/v3/users/ec4533181f17478280294f70befba7da => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:54.034287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:54.034868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:54.034868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:54.072211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:54.073447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:54.120804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:54.245511 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2776/5547] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:54.371633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:54.373652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:54.374020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:54.375653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:54.397540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0808e-b89d-484e-a87f-8327080bab25 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:54.404071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:54.405066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:54.405508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:54.406059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] PATH_INFO: `/v3/projects/6cca3ea9cfe04ce58b99bf83b531cd9b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:54.407795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cca3ea9cfe04ce58b99bf83b531cd9b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:54.411129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:54.567338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b2b4f-e059-4df4-8bc6-eaade4fb65a7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:54.583093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1748a-6579-4d06-a66e-5cfe768a3d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:54.583830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1748a-6579-4d06-a66e-5cfe768a3d27 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:54.584404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1748a-6579-4d06-a66e-5cfe768a3d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:54.584843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1748a-6579-4d06-a66e-5cfe768a3d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:54.610933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cca3ea9cfe04ce58b99bf83b531cd9b', 'request_id': 'req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:54.614634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1748a-6579-4d06-a66e-5cfe768a3d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:54.625023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1748a-6579-4d06-a66e-5cfe768a3d27 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:54.633100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cca3ea9cfe04ce58b99bf83b531cd9b', 'request_id': 'req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:54.660884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cca3ea9cfe04ce58b99bf83b531cd9b', 'request_id': 'req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:54.665353 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2777/5548] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:08:54.684035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] The token cache is being invalidate because project 6cca3ea9cfe04ce58b99bf83b531cd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:54.684624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:54.689080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-43c3ac1f-91c4-4d4d-973e-08dfca8575a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:54.698839 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2772/5549] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:54 2022] DELETE /identity/v3/projects/6cca3ea9cfe04ce58b99bf83b531cd9b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:54.705451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6555be-9913-4def-b531-fc995b7674f9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:54.705451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6555be-9913-4def-b531-fc995b7674f9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:54.705451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6555be-9913-4def-b531-fc995b7674f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:54.743380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6e6555be-9913-4def-b531-fc995b7674f9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:54.744500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6e6555be-9913-4def-b531-fc995b7674f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:54.854463 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2778/5550] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:54.865083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58478c85-dbb8-486a-b1f8-ab506735968f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:54.865936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58478c85-dbb8-486a-b1f8-ab506735968f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:54.866395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58478c85-dbb8-486a-b1f8-ab506735968f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:54.941520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-58478c85-dbb8-486a-b1f8-ab506735968f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:54.945137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-58478c85-dbb8-486a-b1f8-ab506735968f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:54.984178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2773/5551] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:54.986359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6555be-9913-4def-b531-fc995b7674f9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:55.004693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6555be-9913-4def-b531-fc995b7674f9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:55.013691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:55.013691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:55.013691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:55.014059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:55.014701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:55.015013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:55.017393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:55.038708 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2779/5552] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:08:55.042693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58478c85-dbb8-486a-b1f8-ab506735968f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:55.054527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58478c85-dbb8-486a-b1f8-ab506735968f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:55.061399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:55.062485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:55.063036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:55.063482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:55.064450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:55.068763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:55.145480 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2774/5553] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:55 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:55.150163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:55.162714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caeb605-f888-452b-aa12-1fbff93bdab8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:55.247774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:55.264055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:55.265373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:55.265925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:55.266356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:55.274148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:55.276845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:55.333055 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2780/5554] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:55 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:55.337111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:55.347712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:55.395057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887b9f6-b19f-43d0-8820-20c90ecf8c64 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:55.419920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:55.421652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:55.423904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:55.424833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:55.425699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:55.430254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:55.448508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2775/5555] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:55.448854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:55.463600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c492a-95f3-41c4-9856-3b77cd58e139 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:55.475567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9646438-d118-48a4-ba9f-0382803d49c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:55.477267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646438-d118-48a4-ba9f-0382803d49c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:55.479430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646438-d118-48a4-ba9f-0382803d49c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:55.481462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646438-d118-48a4-ba9f-0382803d49c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:55.486471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646438-d118-48a4-ba9f-0382803d49c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:55.489900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646438-d118-48a4-ba9f-0382803d49c7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:55.528842 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2781/5556] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:55.537496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:55.557378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187237e2-c1a4-40e6-937c-c78d2e0a8d5e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:55.570726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:55.571431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:55.571858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:55.574540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] PATH_INFO: `/v3/projects/c01fb3ab003b4967bdca70e3e8a3422d/users/f7365af66abb4559b07fd096f84c4d61/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:55.599809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01fb3ab003b4967bdca70e3e8a3422d, user_id=f7365af66abb4559b07fd096f84c4d61, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:55.604282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:55.630725 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2776/5557] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:55 2022] PUT /identity/v3/projects/c01fb3ab003b4967bdca70e3e8a3422d/users/f7365af66abb4559b07fd096f84c4d61/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:55.644502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3540f7-d31e-4f02-9ef2-abbda1fb365b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:55.644845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3540f7-d31e-4f02-9ef2-abbda1fb365b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:55.645153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3540f7-d31e-4f02-9ef2-abbda1fb365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:55.678392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2c3540f7-d31e-4f02-9ef2-abbda1fb365b None None] MFA Rules not processed for user `f7365af66abb4559b07fd096f84c4d61`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:55.679660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c3540f7-d31e-4f02-9ef2-abbda1fb365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:55.789127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2782/5558] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:55.808584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:55.810632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:55.986178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d19cf-52d8-4d89-a012-e11dc3e34839 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:55.998009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705741f1-7432-4775-98f3-0f651ee44470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:55.999932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705741f1-7432-4775-98f3-0f651ee44470 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:56.000221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705741f1-7432-4775-98f3-0f651ee44470 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:56.001647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705741f1-7432-4775-98f3-0f651ee44470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:56.027972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705741f1-7432-4775-98f3-0f651ee44470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:56.031408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705741f1-7432-4775-98f3-0f651ee44470 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:56.067057 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2777/5559] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:56.098642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3540f7-d31e-4f02-9ef2-abbda1fb365b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:56.258792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3540f7-d31e-4f02-9ef2-abbda1fb365b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:56.271121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:56.271789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:56.272122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:56.272477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] PATH_INFO: `/v3/users/f7365af66abb4559b07fd096f84c4d61` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:56.275665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7365af66abb4559b07fd096f84c4d61)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:56.277695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:56.356536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e267c8-a8a5-4f9b-8ced-430fa80b8929 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:56.356863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e267c8-a8a5-4f9b-8ced-430fa80b8929 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:56.357138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e267c8-a8a5-4f9b-8ced-430fa80b8929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:56.414024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7365af66abb4559b07fd096f84c4d61'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.415909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a2e267c8-a8a5-4f9b-8ced-430fa80b8929 None None] MFA Rules not processed for user `1147de803ef34d16b39e96720b2f638b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:56.417137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2e267c8-a8a5-4f9b-8ced-430fa80b8929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:56.457575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:56.459580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7365af66abb4559b07fd096f84c4d61'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.478227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7365af66abb4559b07fd096f84c4d61'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.503455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7365af66abb4559b07fd096f84c4d61'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.504281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7365af66abb4559b07fd096f84c4d61'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.512627 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2783/5560] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:56 2022] DELETE /identity/v3/users/f7365af66abb4559b07fd096f84c4d61 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:56.513639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:56.516381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:56.549700 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2778/5561] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:56.551192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:56.551631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:56.552750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:56.627146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:56.632371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:56.683332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce6866-1881-4c2e-a2ea-177992894302 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:56.691787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:56.692446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:56.692740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:56.693020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] PATH_INFO: `/v3/projects/1829a58d2c73430e9a86f65534de9df4` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:56.695961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1829a58d2c73430e9a86f65534de9df4)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:56.697912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:56.775417 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2779/5562] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:56.791381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:56.792824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:56.811536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1829a58d2c73430e9a86f65534de9df4', 'request_id': 'req-452981f9-7123-48fc-9a29-b498c9532d25'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.826236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1829a58d2c73430e9a86f65534de9df4', 'request_id': 'req-452981f9-7123-48fc-9a29-b498c9532d25'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.827619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:56.838587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1829a58d2c73430e9a86f65534de9df4', 'request_id': 'req-452981f9-7123-48fc-9a29-b498c9532d25'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.855329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] The token cache is being invalidate because project 1829a58d2c73430e9a86f65534de9df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:56.855650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.857225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:56.859634 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2784/5563] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:56 2022] DELETE /identity/v3/projects/1829a58d2c73430e9a86f65534de9df4 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:56.864828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:56.875817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:56.941020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256da6ab-8e04-40e0-8939-aab0b10b8df4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:57.016219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452981f9-7123-48fc-9a29-b498c9532d25 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:57.024845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:57.025669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:57.025985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:57.026255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] PATH_INFO: `/v3/users/54fc44f7b9834f008e8a36586e876f45` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:57.027612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fc44f7b9834f008e8a36586e876f45)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:57.029948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:57.060738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:57.063279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:57.064518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:57.065363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:57.066013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:57.069447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:57.088499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:57.090520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:57.109991 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2780/5564] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:08:57.145996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fc44f7b9834f008e8a36586e876f45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:57.159671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fc44f7b9834f008e8a36586e876f45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:57.170804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fc44f7b9834f008e8a36586e876f45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:57.181710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fc44f7b9834f008e8a36586e876f45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:57.182437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f099826a-e896-4c55-88c8-d29c55e9311e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fc44f7b9834f008e8a36586e876f45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:57.191889 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2785/5565] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:08:56 2022] DELETE /identity/v3/users/54fc44f7b9834f008e8a36586e876f45 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:57.975729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:57.996891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694dd37-1200-4b92-b6e2-2a93c45c6e70 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:58.157677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:58.160887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.162013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.162460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] PATH_INFO: `/v3/projects/b4a8fed172a24aaf83c8721e0f1d1552` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.163940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a8fed172a24aaf83c8721e0f1d1552)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:58.166107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:58.234416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a8fed172a24aaf83c8721e0f1d1552', 'request_id': 'req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.242322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a8fed172a24aaf83c8721e0f1d1552', 'request_id': 'req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.251731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a8fed172a24aaf83c8721e0f1d1552', 'request_id': 'req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.267214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] The token cache is being invalidate because project b4a8fed172a24aaf83c8721e0f1d1552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:58.268560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.269475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3ea3f6-ebfc-47b8-aba4-72223fe6b631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.271300 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2781/5566] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:57 2022] DELETE /identity/v3/projects/b4a8fed172a24aaf83c8721e0f1d1552 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:58.277104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89b7ee-416f-46bd-b08f-fe14d48f7081 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.277968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89b7ee-416f-46bd-b08f-fe14d48f7081 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.278401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89b7ee-416f-46bd-b08f-fe14d48f7081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.316895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0e89b7ee-416f-46bd-b08f-fe14d48f7081 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:58.318266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0e89b7ee-416f-46bd-b08f-fe14d48f7081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:58.396358 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2786/5567] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:58.407803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb00fbb-267b-4cad-91c3-bc728aa75086 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.409212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb00fbb-267b-4cad-91c3-bc728aa75086 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.409602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb00fbb-267b-4cad-91c3-bc728aa75086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.429075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:58.446179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3cb00fbb-267b-4cad-91c3-bc728aa75086 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:58.448034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3cb00fbb-267b-4cad-91c3-bc728aa75086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:58.463496 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2782/5568] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:58.464399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89b7ee-416f-46bd-b08f-fe14d48f7081 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:58.477449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89b7ee-416f-46bd-b08f-fe14d48f7081 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:58.483693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:58.484499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.485444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.486001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.488533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:58.489092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:58.493430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:58.554088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb00fbb-267b-4cad-91c3-bc728aa75086 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:58.560997 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2787/5569] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:08:58.562206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:58.568932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb00fbb-267b-4cad-91c3-bc728aa75086 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:58.570981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:58.587150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:58.589536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.589800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.590118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.590767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:58.599258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:58.616981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d7f19-3327-4075-8eb9-65c97a543fa0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:58.648324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:58.650174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.650458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.656690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] PATH_INFO: `/v3/projects/c01fb3ab003b4967bdca70e3e8a3422d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.659211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c01fb3ab003b4967bdca70e3e8a3422d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:58.663845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:58.703293 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2783/5570] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:58 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:58.704251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:58.804860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c01fb3ab003b4967bdca70e3e8a3422d', 'request_id': 'req-89045d20-64ce-4455-b3b0-b7c1250a55e9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.812872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c01fb3ab003b4967bdca70e3e8a3422d', 'request_id': 'req-89045d20-64ce-4455-b3b0-b7c1250a55e9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.825408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c01fb3ab003b4967bdca70e3e8a3422d', 'request_id': 'req-89045d20-64ce-4455-b3b0-b7c1250a55e9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.833187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e781f99-ffab-4902-9122-dd4fef477957 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:58.844380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:58.861016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] The token cache is being invalidate because project c01fb3ab003b4967bdca70e3e8a3422d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:08:58.861874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.863126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89045d20-64ce-4455-b3b0-b7c1250a55e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:08:58.865429 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2788/5571] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:08:58 2022] DELETE /identity/v3/projects/c01fb3ab003b4967bdca70e3e8a3422d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:58.873790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b20aee-337a-4eec-a84f-5d2736b1d9a4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.874064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b20aee-337a-4eec-a84f-5d2736b1d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.874305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b20aee-337a-4eec-a84f-5d2736b1d9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.929626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-43b20aee-337a-4eec-a84f-5d2736b1d9a4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:58.931111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-43b20aee-337a-4eec-a84f-5d2736b1d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:58.961863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:58.962137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:58.966100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:58.966100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:58.966100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:58.966100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:58.995313 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2784/5572] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:58 2022] POST /identity/v3/users => generated 296 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:58.999535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.002537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:59.017573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e898489-0249-4e59-9348-e4f28749d122 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.024578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.025365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.025744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.026176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.027190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.029101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.042441 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2785/5573] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:59.046121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.079648 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2789/5574] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:08:59.092602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576632d4-96e8-4920-9e90-6a069fb336e2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.092910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576632d4-96e8-4920-9e90-6a069fb336e2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.093138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576632d4-96e8-4920-9e90-6a069fb336e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.096229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a102f9-2886-4047-8f58-d759261f8487 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.103955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.108110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.108467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.108749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.109423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.112903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.116985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-576632d4-96e8-4920-9e90-6a069fb336e2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:59.118276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-576632d4-96e8-4920-9e90-6a069fb336e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:59.127071 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2786/5575] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:59.129426 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2790/5576] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:08:59.134451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576632d4-96e8-4920-9e90-6a069fb336e2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.136087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.156174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576632d4-96e8-4920-9e90-6a069fb336e2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.157229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064e5d-81cf-4bd7-a25b-56ab09e7afc9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.165438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.165964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913be47-4c11-4983-a941-c3c3ce120099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.166461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.166879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913be47-4c11-4983-a941-c3c3ce120099 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.167287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.167407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913be47-4c11-4983-a941-c3c3ce120099 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.167726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.168303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913be47-4c11-4983-a941-c3c3ce120099 admin admin] PATH_INFO: `/v3/projects/39c617de61874844bdb9b1b8cd54eba3/users/1efcbd2114624772986d3b20c9ae4f21/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.169550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:08:59.170062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.172522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.196312 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2791/5577] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:08:59.197274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.201249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913be47-4c11-4983-a941-c3c3ce120099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c617de61874844bdb9b1b8cd54eba3, user_id=1efcbd2114624772986d3b20c9ae4f21, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.206240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913be47-4c11-4983-a941-c3c3ce120099 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.210612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4996e2-18cd-481d-bfb1-c1da03b71456 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.219301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.220110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.220456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.220883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.221629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.224458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.241850 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2787/5578] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:59 2022] PUT /identity/v3/projects/39c617de61874844bdb9b1b8cd54eba3/users/1efcbd2114624772986d3b20c9ae4f21/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:59.247180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd357-8b30-4e67-a1cf-5e2f9c1ebdc5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.247702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd357-8b30-4e67-a1cf-5e2f9c1ebdc5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.248207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd357-8b30-4e67-a1cf-5e2f9c1ebdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.281870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2792/5579] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:08:59 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:59.284295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.312064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a2ffd357-8b30-4e67-a1cf-5e2f9c1ebdc5 None None] MFA Rules not processed for user `1efcbd2114624772986d3b20c9ae4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:08:59.317120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2ffd357-8b30-4e67-a1cf-5e2f9c1ebdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:59.364485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:59.407429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a552f5-5a21-4d33-a539-acf3dbeee6d8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.415108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.416080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.416613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.418434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.419534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.422388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.466512 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2793/5580] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:08:59 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:08:59.467453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.498046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2788/5581] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:08:59.500898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509b6e6-8fd0-43f0-b1ab-14909b7bd187 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.511149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.511785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.512064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.512603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.513596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.518404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.544960 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2794/5582] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:08:59.549598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffd357-8b30-4e67-a1cf-5e2f9c1ebdc5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.573650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.576069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:59.592812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffd357-8b30-4e67-a1cf-5e2f9c1ebdc5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.613683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.617415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.631515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.642157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.643514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.650174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.701587 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2789/5583] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:08:59.710668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.739701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df537e7-d058-4ac9-ad79-0c3ebb19a1f7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.756229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:08:59.757339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.758188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.758520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.759201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] PATH_INFO: `/v3/projects/da91b10f0f79456e963d2c84dbda30c6/users/ef124f7e89824ce7a485025227ed5d81/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.784818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da91b10f0f79456e963d2c84dbda30c6, user_id=ef124f7e89824ce7a485025227ed5d81, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.787947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.801304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed5e9c-2b63-4e2f-80a9-7f2d5bdd8671 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.809595 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2790/5584] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:08:59 2022] PUT /identity/v3/projects/da91b10f0f79456e963d2c84dbda30c6/users/ef124f7e89824ce7a485025227ed5d81/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:08:59.810923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:08:59.829521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:59.896812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.897427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.897861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.897996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.909228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b199c-ba97-436a-be3c-65d0476d285c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:08:59.917570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ddb17-1b8f-409e-bc9b-3181fe900bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:08:59.918296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ddb17-1b8f-409e-bc9b-3181fe900bbb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.918580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ddb17-1b8f-409e-bc9b-3181fe900bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.918856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ddb17-1b8f-409e-bc9b-3181fe900bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:08:59.920819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-576ddb17-1b8f-409e-bc9b-3181fe900bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:08:59.934725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ddb17-1b8f-409e-bc9b-3181fe900bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:08:59.937367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ddb17-1b8f-409e-bc9b-3181fe900bbb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:08:59.950128 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2791/5585] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:08:59.965355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:08:59.965708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:08:59.969862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:00.013036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:00.015048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:00.015445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] MFA Rules not processed for user `ef124f7e89824ce7a485025227ed5d81`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:00.016741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:00.034166 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2795/5586] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:08:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:00.124267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:00.204008 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2792/5587] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:00.219321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:00.223667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:00.346969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:00.491206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3d987-7de8-4389-a0ea-8152adb3159e service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:00.523479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:00.524149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:00.524405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:00.524681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] PATH_INFO: `/v3/users/1efcbd2114624772986d3b20c9ae4f21` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:00.525958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efcbd2114624772986d3b20c9ae4f21)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:00.527901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:00.550863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe3f98-c30b-4bcb-8cd8-60aa3173cb2a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:00.569205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef8e48-db67-4068-b831-3f4e9bb00354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:00.573321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef8e48-db67-4068-b831-3f4e9bb00354 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:00.573589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef8e48-db67-4068-b831-3f4e9bb00354 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:00.575665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef8e48-db67-4068-b831-3f4e9bb00354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:00.610933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef8e48-db67-4068-b831-3f4e9bb00354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:00.617056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef8e48-db67-4068-b831-3f4e9bb00354 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:00.621873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efcbd2114624772986d3b20c9ae4f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:00.623547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:00.645098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2793/5588] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:09:00.650925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efcbd2114624772986d3b20c9ae4f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:00.667965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efcbd2114624772986d3b20c9ae4f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:00.689408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efcbd2114624772986d3b20c9ae4f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:00.695094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efcbd2114624772986d3b20c9ae4f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:00.706124 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2796/5589] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:09:00 2022] DELETE /identity/v3/users/1efcbd2114624772986d3b20c9ae4f21 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:09:00.707724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:00.708221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:00.708698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:00.746106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:00.749447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:00.842159 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2794/5590] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:00.886833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:00.888309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:01.011567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf066c4-6eac-494f-896f-cba2456f79f3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:01.016403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd674c-2a0d-4b9c-9e25-b27d6869f5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:01.017483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd674c-2a0d-4b9c-9e25-b27d6869f5de service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:01.017679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd674c-2a0d-4b9c-9e25-b27d6869f5de service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:01.018019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd674c-2a0d-4b9c-9e25-b27d6869f5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:01.029368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd674c-2a0d-4b9c-9e25-b27d6869f5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:01.032124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd674c-2a0d-4b9c-9e25-b27d6869f5de service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:01.048933 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2797/5591] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:02.041491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:02.043093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:02.073685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792384c-5f2f-4652-af3c-9f802761e445 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:02.086499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:02.087333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.087906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.088321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] PATH_INFO: `/v3/projects/39c617de61874844bdb9b1b8cd54eba3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.090789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39c617de61874844bdb9b1b8cd54eba3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:02.093996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:02.180554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39c617de61874844bdb9b1b8cd54eba3', 'request_id': 'req-182a61d1-d12c-4255-be6b-107fe64f806b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:02.202377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39c617de61874844bdb9b1b8cd54eba3', 'request_id': 'req-182a61d1-d12c-4255-be6b-107fe64f806b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:02.216458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39c617de61874844bdb9b1b8cd54eba3', 'request_id': 'req-182a61d1-d12c-4255-be6b-107fe64f806b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:02.240060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] The token cache is being invalidate because project 39c617de61874844bdb9b1b8cd54eba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:09:02.240946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:02.241876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-182a61d1-d12c-4255-be6b-107fe64f806b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:02.244012 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2795/5592] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:09:02 2022] DELETE /identity/v3/projects/39c617de61874844bdb9b1b8cd54eba3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:02.248988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc254e-791c-4846-a153-62113f24848c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.250039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc254e-791c-4846-a153-62113f24848c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.250329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc254e-791c-4846-a153-62113f24848c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.295326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-34dc254e-791c-4846-a153-62113f24848c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:02.297884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34dc254e-791c-4846-a153-62113f24848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:02.438858 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2798/5593] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:02.447017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ff6b-5d73-4b3d-8ba8-1133940cce6e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.447017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ff6b-5d73-4b3d-8ba8-1133940cce6e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.447461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ff6b-5d73-4b3d-8ba8-1133940cce6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.474226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ab37ff6b-5d73-4b3d-8ba8-1133940cce6e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:02.475490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab37ff6b-5d73-4b3d-8ba8-1133940cce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:02.485427 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2796/5594] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:02.490211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc254e-791c-4846-a153-62113f24848c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:02.504747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc254e-791c-4846-a153-62113f24848c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:02.519792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:02.520648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.521027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.521429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.522228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:09:02.522632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:02.525413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:02.544786 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2799/5595] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:09:02.546680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37ff6b-5d73-4b3d-8ba8-1133940cce6e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:02.566233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37ff6b-5d73-4b3d-8ba8-1133940cce6e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:02.574200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:02.574947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.575348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.575717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.576605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:02.579254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:02.626922 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2797/5596] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:09:02 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:02.631586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:02.645185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92d993-d872-454f-ba16-6c2980ebdcb7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:02.754002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:02.763502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:02.764300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.766291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.766626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.768267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:02.784986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:02.815410 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2800/5597] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:09:02 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:02.816803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:02.829265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945593e-1b95-46bb-8df2-406b39ead000 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:02.835717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:02.836586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.837037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.837469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.838502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:02.840715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:02.858105 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2798/5598] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:02.858802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:02.870001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6c65b-6922-4c79-82e5-c33c23899be3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:02.876722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8211a0b-06df-42d7-88e8-2d9befe12a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:02.878119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8211a0b-06df-42d7-88e8-2d9befe12a5d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.878529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8211a0b-06df-42d7-88e8-2d9befe12a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.879176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8211a0b-06df-42d7-88e8-2d9befe12a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.879838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8211a0b-06df-42d7-88e8-2d9befe12a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:02.881460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8211a0b-06df-42d7-88e8-2d9befe12a5d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:02.898098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2801/5599] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:02.903045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:02.916562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:02.925319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825380f-9d13-42d4-b824-5265083a974f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:02.931984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:02.932954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:02.933448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:02.933817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] PATH_INFO: `/v3/projects/91208ffe0c6d412aab6751dc60e5f223/users/b02899f9e0934dcabbce985564dcb612/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:02.986219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] RBAC: Authorizing `identity:create_grant(project_id=91208ffe0c6d412aab6751dc60e5f223, user_id=b02899f9e0934dcabbce985564dcb612, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:02.989745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:03.027698 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2799/5600] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:02 2022] PUT /identity/v3/projects/91208ffe0c6d412aab6751dc60e5f223/users/b02899f9e0934dcabbce985564dcb612/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:03.039195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:03.039818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:03.040651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:03.125056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] MFA Rules not processed for user `b02899f9e0934dcabbce985564dcb612`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:03.126461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:03.279342 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2802/5601] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:03.397146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:03.399096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:03.513444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bfcd3-4817-4466-bbbc-70a8e5ea337a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:03.517956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb0c13-ea7e-4478-a9cf-49f05be2cd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:03.518665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb0c13-ea7e-4478-a9cf-49f05be2cd2b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:03.519046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb0c13-ea7e-4478-a9cf-49f05be2cd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:03.519386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb0c13-ea7e-4478-a9cf-49f05be2cd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:03.552549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb0c13-ea7e-4478-a9cf-49f05be2cd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:03.555851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb0c13-ea7e-4478-a9cf-49f05be2cd2b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:03.581436 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2800/5602] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:04.669658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:04.671470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:04.816578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055951-75ad-4ccc-b6b4-61492ed71e2d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:04.824322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:04.825564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:04.826509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:04.827196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] PATH_INFO: `/v3/users/ef124f7e89824ce7a485025227ed5d81` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:04.828683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef124f7e89824ce7a485025227ed5d81)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:04.831199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:04.901355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef124f7e89824ce7a485025227ed5d81'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:04.927390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef124f7e89824ce7a485025227ed5d81'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:04.929730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:04.938384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef124f7e89824ce7a485025227ed5d81'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:04.952893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef124f7e89824ce7a485025227ed5d81'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:04.954731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef124f7e89824ce7a485025227ed5d81'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:04.971401 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2803/5603] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:09:04 2022] DELETE /identity/v3/users/ef124f7e89824ce7a485025227ed5d81 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:04.979431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:04.979889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:04.979974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:05.025692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:05.027171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:05.137532 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2801/5604] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:05.172682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:05.175910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:05.252776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:05.254671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:05.278227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ff551-dfab-4b6e-ae69-9a26e071c120 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:05.290487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:05.291283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:05.291640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:05.291989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] PATH_INFO: `/v3/users/b02899f9e0934dcabbce985564dcb612` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:05.293896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b02899f9e0934dcabbce985564dcb612)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:05.295911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:05.315882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de462193-53d6-4a76-9793-225dd10f67ca admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:05.328992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17827e87-2491-4b8d-9d7b-bfbaf6fc9ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:05.329646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17827e87-2491-4b8d-9d7b-bfbaf6fc9ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:05.330475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17827e87-2491-4b8d-9d7b-bfbaf6fc9ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:05.330761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17827e87-2491-4b8d-9d7b-bfbaf6fc9ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:05.374228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17827e87-2491-4b8d-9d7b-bfbaf6fc9ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:05.375696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17827e87-2491-4b8d-9d7b-bfbaf6fc9ec5 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:05.392399 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2804/5605] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:05.435760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b02899f9e0934dcabbce985564dcb612'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.447776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b02899f9e0934dcabbce985564dcb612'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.449592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b02899f9e0934dcabbce985564dcb612'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.463853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b02899f9e0934dcabbce985564dcb612'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.465648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b02899f9e0934dcabbce985564dcb612'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:05.486004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:05.504170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:05.504811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:05.505305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:05.508961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2802/5606] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:09:05 2022] DELETE /identity/v3/users/b02899f9e0934dcabbce985564dcb612 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:05.554201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:05.555971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:05.654992 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2805/5607] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:05.695403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:05.697072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:05.790342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676a4d4-41a6-40d3-8552-2d49590366b3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:05.794286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:05.794897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:05.795177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:05.795448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:05.806162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:05.807602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:05.820794 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2803/5608] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:06.903908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:06.905644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:06.918606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80ca9-b42e-4f8a-9e8d-8542d7d03beb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:06.926665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:06.927356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:06.927613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:06.928053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] PATH_INFO: `/v3/projects/da91b10f0f79456e963d2c84dbda30c6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:06.929297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da91b10f0f79456e963d2c84dbda30c6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:06.931416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:06.989178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:06.991332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.013054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17e7c5-c6ce-42c0-a232-e4ca47708061 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.048569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da91b10f0f79456e963d2c84dbda30c6', 'request_id': 'req-f62dcf24-70fc-40a2-adbe-4447671cc861'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.064188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da91b10f0f79456e963d2c84dbda30c6', 'request_id': 'req-f62dcf24-70fc-40a2-adbe-4447671cc861'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.072529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da91b10f0f79456e963d2c84dbda30c6', 'request_id': 'req-f62dcf24-70fc-40a2-adbe-4447671cc861'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.127663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:07.128618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.129140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.129978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] PATH_INFO: `/v3/users/c882de4b1ee547e79f86dc21a4a01b6d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.131560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c882de4b1ee547e79f86dc21a4a01b6d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:07.134815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:07.147961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] The token cache is being invalidate because project da91b10f0f79456e963d2c84dbda30c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:09:07.149045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.150030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f62dcf24-70fc-40a2-adbe-4447671cc861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.156169 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2806/5609] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:09:06 2022] DELETE /identity/v3/projects/da91b10f0f79456e963d2c84dbda30c6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:07.159440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.159915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.160311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.221225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:07.228750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.246853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c882de4b1ee547e79f86dc21a4a01b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.250943 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2807/5610] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:07.261638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:07.264903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.265114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c882de4b1ee547e79f86dc21a4a01b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.267186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c882de4b1ee547e79f86dc21a4a01b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.287378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:07.294910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c882de4b1ee547e79f86dc21a4a01b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.298040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-405233df-bef2-48c2-8e78-1f7afe03825f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c882de4b1ee547e79f86dc21a4a01b6d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.321077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d67c6-5a33-44ff-b17c-0e182183e131 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.332842 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2804/5611] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:09:06 2022] DELETE /identity/v3/users/c882de4b1ee547e79f86dc21a4a01b6d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:07.334112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:07.335351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.336770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f331b86-6d9c-4f7c-8734-28f3e63887e1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.337389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.338291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f331b86-6d9c-4f7c-8734-28f3e63887e1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.338575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f331b86-6d9c-4f7c-8734-28f3e63887e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.339072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] PATH_INFO: `/v3/projects/91208ffe0c6d412aab6751dc60e5f223` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.341743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91208ffe0c6d412aab6751dc60e5f223)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:07.347673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:07.387011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8f331b86-6d9c-4f7c-8734-28f3e63887e1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:07.399906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8f331b86-6d9c-4f7c-8734-28f3e63887e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.453623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:07.502400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91208ffe0c6d412aab6751dc60e5f223', 'request_id': 'req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.523198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91208ffe0c6d412aab6751dc60e5f223', 'request_id': 'req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.542219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2805/5612] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:07.546241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f331b86-6d9c-4f7c-8734-28f3e63887e1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:07.546966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91208ffe0c6d412aab6751dc60e5f223', 'request_id': 'req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.560838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f331b86-6d9c-4f7c-8734-28f3e63887e1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.577256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] The token cache is being invalidate because project 91208ffe0c6d412aab6751dc60e5f223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:09:07.578663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.582282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-579fe9dd-e491-4cce-9b9b-196ee2bb4a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:07.588600 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2808/5613] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:09:07 2022] DELETE /identity/v3/projects/91208ffe0c6d412aab6751dc60e5f223 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:07.593138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.593434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.593760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.648529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:07.650766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.655787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cdd83-fba5-48be-bee5-dc1444c6bc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:07.657109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cdd83-fba5-48be-bee5-dc1444c6bc42 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.657552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cdd83-fba5-48be-bee5-dc1444c6bc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.658738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cdd83-fba5-48be-bee5-dc1444c6bc42 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.659573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cdd83-fba5-48be-bee5-dc1444c6bc42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:09:07.660245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cdd83-fba5-48be-bee5-dc1444c6bc42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:07.664018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cdd83-fba5-48be-bee5-dc1444c6bc42 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:07.678171 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2809/5614] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:07.678717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2806/5615] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:09:07.684343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:07.686473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.687695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081cfb6-c2af-4b3c-8409-62d461afcbf9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.687990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081cfb6-c2af-4b3c-8409-62d461afcbf9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.688265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081cfb6-c2af-4b3c-8409-62d461afcbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.702570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9bd13-2871-4168-bc66-cabc6609d2de None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.712213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:07.713244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.713649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.714106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.715095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:07.717600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:07.734937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9081cfb6-c2af-4b3c-8409-62d461afcbf9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:07.738483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9081cfb6-c2af-4b3c-8409-62d461afcbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:07.762507 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2807/5616] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:07.766428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081cfb6-c2af-4b3c-8409-62d461afcbf9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:07.785694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081cfb6-c2af-4b3c-8409-62d461afcbf9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.794941 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2810/5617] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:09:07 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:07.798712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:07.804925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:07.805560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.805852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.806140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.806798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:09:07.807106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:07.808973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:07.833391 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2808/5618] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:09:07.838303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:07.863488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844751bc-5e1e-471e-bfde-2591cf373a99 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.925548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aa006-a5f3-444f-a90f-b6a7a46c3ca5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:07.932925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:07.934036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:07.934603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:07.935582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:07.936551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:07.940915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:07.985384 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2811/5619] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:09:07 2022] POST /identity/v3/users => generated 284 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:07.987409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.003272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3dba9-a3c4-42ec-ad16-5b005209b9fa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.004046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.004687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.004977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.005258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.006079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.008396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.012757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.013965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.014648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.015143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.017075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.020783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.042057 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2812/5620] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:08.050237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.068143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854fa19-3f12-42a1-8036-6caa4ff1bc3d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.072759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2809/5621] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:09:07 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:08.077800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.119446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:08.123633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:08.157203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.157699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.158211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.158756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.159821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.161752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.183115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2813/5622] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:08.186246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.219105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c62ff-c421-43b5-b7d6-5270668548de admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.228864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354cddb-7ac5-4ab3-a4bb-9d0221b753aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.236020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.237859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.237859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.238141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] PATH_INFO: `/v3/projects/03237011b8904871b3cb999f46c1588c/users/23d9ece24b174a40b86b76325b9e6f93/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.260735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.261005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.261322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.261602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.262271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.264210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.268506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03237011b8904871b3cb999f46c1588c, user_id=23d9ece24b174a40b86b76325b9e6f93, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.271391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.306657 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2814/5623] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:08 2022] PUT /identity/v3/projects/03237011b8904871b3cb999f46c1588c/users/23d9ece24b174a40b86b76325b9e6f93/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:08.306657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.306657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.313808 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2810/5624] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:09:08 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:08.315343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.319449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.429863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c614626-698e-44f4-887c-aae1246db235 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.436762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.437488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.437907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.438288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] PATH_INFO: `/v3/projects/862a718cd74e4668a4fc9559c9d9d470` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.440162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] RBAC: Authorizing `identity:delete_project(project_id=862a718cd74e4668a4fc9559c9d9d470)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.442262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.468906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602426c2-2079-4d14-b5d7-70081a6ea5e7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.508620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862a718cd74e4668a4fc9559c9d9d470', 'request_id': 'req-bfda1127-7ac8-495b-b6f9-f071ebd726da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:08.518345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862a718cd74e4668a4fc9559c9d9d470', 'request_id': 'req-bfda1127-7ac8-495b-b6f9-f071ebd726da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:08.527152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862a718cd74e4668a4fc9559c9d9d470', 'request_id': 'req-bfda1127-7ac8-495b-b6f9-f071ebd726da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:08.543675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] The token cache is being invalidate because project 862a718cd74e4668a4fc9559c9d9d470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:09:08.544064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:08.544799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bfda1127-7ac8-495b-b6f9-f071ebd726da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:08.546948 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2815/5625] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:09:08 2022] DELETE /identity/v3/projects/862a718cd74e4668a4fc9559c9d9d470 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:08.548322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.548660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.548951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.561112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.562323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.562323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.562579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.565142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.579804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.582157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.588463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] MFA Rules not processed for user `23d9ece24b174a40b86b76325b9e6f93`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:08.591089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.597871 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2811/5626] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:08.598538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.600138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.616013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23d0f2-b288-49ee-92e6-bf826caaec59 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.664062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:08.670597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:08.729232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c40de-fd69-4fe9-8eba-881bdf6b76f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.729931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c40de-fd69-4fe9-8eba-881bdf6b76f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.730217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c40de-fd69-4fe9-8eba-881bdf6b76f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.730493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c40de-fd69-4fe9-8eba-881bdf6b76f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.731134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c40de-fd69-4fe9-8eba-881bdf6b76f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.733104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c40de-fd69-4fe9-8eba-881bdf6b76f9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.745405 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2812/5627] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:08.746785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674349e-33f1-4ee2-9bc0-c0f3b598e95c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.747068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674349e-33f1-4ee2-9bc0-c0f3b598e95c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.747341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674349e-33f1-4ee2-9bc0-c0f3b598e95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.758966 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2816/5628] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:08.760039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.762821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.774150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b674349e-33f1-4ee2-9bc0-c0f3b598e95c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:08.775717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b674349e-33f1-4ee2-9bc0-c0f3b598e95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.788325 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2813/5629] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:08.789575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674349e-33f1-4ee2-9bc0-c0f3b598e95c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.800841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674349e-33f1-4ee2-9bc0-c0f3b598e95c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.809018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.810283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.810785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.811292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.812709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.816345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.833582 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2814/5630] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:08.834586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.836889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.874770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8a97b-c92a-411f-891f-27103a059828 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.883440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deecaf7-0352-4608-bd31-d2f5baf42b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.885330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deecaf7-0352-4608-bd31-d2f5baf42b9d service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.885623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deecaf7-0352-4608-bd31-d2f5baf42b9d service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.885934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deecaf7-0352-4608-bd31-d2f5baf42b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.888652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6deecaf7-0352-4608-bd31-d2f5baf42b9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.906488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deecaf7-0352-4608-bd31-d2f5baf42b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.908198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deecaf7-0352-4608-bd31-d2f5baf42b9d service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.920820 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2817/5631] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:08.922144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fe193-55cd-4c45-9f23-2b20081458df None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.922427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fe193-55cd-4c45-9f23-2b20081458df None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.922701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fe193-55cd-4c45-9f23-2b20081458df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.945321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfca09-7a08-4acc-a0af-a119c2dc29eb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.949342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.949992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.950275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.950605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1a2fe193-55cd-4c45-9f23-2b20081458df None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:08.951083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.952022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1a2fe193-55cd-4c45-9f23-2b20081458df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:08.963137 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2818/5632] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:08.965245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fe193-55cd-4c45-9f23-2b20081458df None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:08.971187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:08.973650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:08.980858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fe193-55cd-4c45-9f23-2b20081458df None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:08.987822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb42f30-59c3-4766-aee6-081070d43ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:08.988128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb42f30-59c3-4766-aee6-081070d43ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:08.988487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb42f30-59c3-4766-aee6-081070d43ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:08.988830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb42f30-59c3-4766-aee6-081070d43ad0 admin admin] PATH_INFO: `/v3/projects/811bc255a8f84b96b561d120fb4398d6/users/dca372f2f382480a9d2ec44ad76e0c13/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:08.993698 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2815/5633] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:08.994601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:09.008287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb42f30-59c3-4766-aee6-081070d43ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811bc255a8f84b96b561d120fb4398d6, user_id=dca372f2f382480a9d2ec44ad76e0c13, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.011154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb42f30-59c3-4766-aee6-081070d43ad0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.016779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e073-3aaa-4380-81b3-5670f92e4361 service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:09.029482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:09.030180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.030458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.030793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.031453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:09:09.031808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.034063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.035226 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2819/5634] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:08 2022] PUT /identity/v3/projects/811bc255a8f84b96b561d120fb4398d6/users/dca372f2f382480a9d2ec44ad76e0c13/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:09.045743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.046163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.046439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.059675 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2816/5635] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:09:09.063464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:09.077496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c044224-3e16-42bb-88d4-307f6375e14a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:09.101978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] MFA Rules not processed for user `dca372f2f382480a9d2ec44ad76e0c13`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:09.103482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:09.222870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:09.223082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.223361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.223530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.224168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.227232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.271011 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2817/5636] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:09:09 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:09.271961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:09.278713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2820/5637] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:09:09.316898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:09.322028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:09.324813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:09.352704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:09.430829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c8ab5-a6f5-450d-a2a5-16f38aeadfaf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:09.447068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:09.448793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.449345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.451188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.454472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.457708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.483561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1334713-f124-4f58-aef2-f9cfd2425de7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:09.527666 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2818/5638] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:09:09 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:09.531577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:09.543568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c22478-7111-4c7d-9817-b549b4d5636a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:09.553672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:09.555043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.555333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.555616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.556262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.559123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.564044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13604-9593-4766-b631-d663eddd52d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:09.564893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13604-9593-4766-b631-d663eddd52d1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.565404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13604-9593-4766-b631-d663eddd52d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.565788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13604-9593-4766-b631-d663eddd52d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.574062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2819/5639] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:09:09.576777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:09.587770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8e63e-9318-4605-9e83-33875c7439d0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:09.593617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:09.594333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.594681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.595024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.596375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.598508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.623690 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2820/5640] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:09.624720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:09.641453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ab7d3-7b96-45d5-8c24-6d89611651c7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:09.656501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13604-9593-4766-b631-d663eddd52d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.658942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13604-9593-4766-b631-d663eddd52d1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.668265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:09.669058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.669455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.670281 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2821/5641] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:09:09.679873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] PATH_INFO: `/v3/projects/c8d7ded29d1345b4b3111a69f4efec6f/users/54dcc78ee39c4119b08023d3cc692d6b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.717134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d7ded29d1345b4b3111a69f4efec6f, user_id=54dcc78ee39c4119b08023d3cc692d6b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:09.728812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:09.752949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2821/5642] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:09 2022] PUT /identity/v3/projects/c8d7ded29d1345b4b3111a69f4efec6f/users/54dcc78ee39c4119b08023d3cc692d6b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:09.759896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7f591-82ad-4d3b-8689-13921e51ee07 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:09.760178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7f591-82ad-4d3b-8689-13921e51ee07 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:09.760508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7f591-82ad-4d3b-8689-13921e51ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:09.787404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0ef7f591-82ad-4d3b-8689-13921e51ee07 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:09.802127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ef7f591-82ad-4d3b-8689-13921e51ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:09.964232 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2822/5643] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:10.023939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:10.156758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90569b7-68a6-447e-b65c-ebb6512e84d0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:10.161876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:10.163661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:10.163661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:10.164980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:10.190922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:10.192572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:10.209236 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2822/5644] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:17.843343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ede54-9507-4db0-a6c8-e5fa0211f4de None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:17.844260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ede54-9507-4db0-a6c8-e5fa0211f4de None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:17.844750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ede54-9507-4db0-a6c8-e5fa0211f4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:17.869886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-628ede54-9507-4db0-a6c8-e5fa0211f4de None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:17.872024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-628ede54-9507-4db0-a6c8-e5fa0211f4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:17.907951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:17.989745 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2823/5645] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:18.536141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:18.537620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:18.554493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c102fc7-fed5-4592-8694-9e6eed3ce055 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:18.562632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:18.565642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:18.566488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:18.567052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:18.569490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:18.586209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:18.589731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:18.626366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2823/5646] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:27.993573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=14 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:28.616646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=76,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:28.865211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:28.865453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:28.865709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:28.913121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] MFA Rules not processed for user `54dcc78ee39c4119b08023d3cc692d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:28.914550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:29.043180 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2824/5647] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:09:29.353056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:29.355128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:29.411330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:29.413112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:29.439752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c5ee-1ab8-49f5-a069-c71bafd6520c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:29.455883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:29.457028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:29.457500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:29.458716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] PATH_INFO: `/v3/users/23d9ece24b174a40b86b76325b9e6f93` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:29.460549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d9ece24b174a40b86b76325b9e6f93)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:29.464109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:29.516993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297f7e3-81fc-4b12-8a21-36c02390638e service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:29.527822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19693308-3d6c-4b89-a3b4-9944b9b3aac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:29.528668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693308-3d6c-4b89-a3b4-9944b9b3aac4 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:29.529109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693308-3d6c-4b89-a3b4-9944b9b3aac4 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:29.529772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19693308-3d6c-4b89-a3b4-9944b9b3aac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:29.541816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19693308-3d6c-4b89-a3b4-9944b9b3aac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:29.543909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19693308-3d6c-4b89-a3b4-9944b9b3aac4 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:29.557993 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2824/5648] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:29.582038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d9ece24b174a40b86b76325b9e6f93'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:29.604746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d9ece24b174a40b86b76325b9e6f93'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:29.638782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d9ece24b174a40b86b76325b9e6f93'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:29.660175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d9ece24b174a40b86b76325b9e6f93'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:29.660915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d9ece24b174a40b86b76325b9e6f93'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:29.677956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:29.678991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:29.679698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:29.688429 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2825/5649] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:09:29 2022] DELETE /identity/v3/users/23d9ece24b174a40b86b76325b9e6f93 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:09:29.723244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:29.724801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:29.858963 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2825/5650] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:29.885698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:29.887260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:30.038651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c336d2-b7c2-4dc5-b17f-8e776a53507a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:30.046987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d289e-0060-4c48-8827-2cef336f4750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:30.048669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d289e-0060-4c48-8827-2cef336f4750 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:30.049371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d289e-0060-4c48-8827-2cef336f4750 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:30.049850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d289e-0060-4c48-8827-2cef336f4750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:30.065516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d289e-0060-4c48-8827-2cef336f4750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:30.067266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d289e-0060-4c48-8827-2cef336f4750 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:30.079933 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2826/5651] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:31.658804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:31.663242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:31.742349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b715d-dae6-4988-8f95-7a8c3bcd8f78 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:31.749580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:31.750542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:31.751149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:31.751741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] PATH_INFO: `/v3/projects/03237011b8904871b3cb999f46c1588c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:31.759904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03237011b8904871b3cb999f46c1588c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:31.762496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:31.826158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03237011b8904871b3cb999f46c1588c', 'request_id': 'req-830b5ddf-920f-42b3-8b2a-0f98d22f0279'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:31.834356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03237011b8904871b3cb999f46c1588c', 'request_id': 'req-830b5ddf-920f-42b3-8b2a-0f98d22f0279'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:31.843824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03237011b8904871b3cb999f46c1588c', 'request_id': 'req-830b5ddf-920f-42b3-8b2a-0f98d22f0279'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:31.860264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] The token cache is being invalidate because project 03237011b8904871b3cb999f46c1588c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:09:31.861813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:31.863018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-830b5ddf-920f-42b3-8b2a-0f98d22f0279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:31.869423 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2826/5652] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:09:31 2022] DELETE /identity/v3/projects/03237011b8904871b3cb999f46c1588c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:31.871211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:31.871679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:31.871764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:31.891974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:31.914849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:31.918144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:32.065429 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2827/5653] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:32.075085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da401b7-b144-467e-be0b-46acd24111b3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.075397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da401b7-b144-467e-be0b-46acd24111b3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.075779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da401b7-b144-467e-be0b-46acd24111b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.103151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0da401b7-b144-467e-be0b-46acd24111b3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:32.106332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0da401b7-b144-467e-be0b-46acd24111b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:32.124245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.125712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:32.131535 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2827/5654] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:32.135694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da401b7-b144-467e-be0b-46acd24111b3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.140981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:32.167004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da401b7-b144-467e-be0b-46acd24111b3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.167483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c68c07-3643-4cdb-8f35-b568537e3089 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.176704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.177755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.179059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.179584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.180078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.180425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.180772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] PATH_INFO: `/v3/users/dca372f2f382480a9d2ec44ad76e0c13` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.181102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.182183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:09:32.182870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.183753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca372f2f382480a9d2ec44ad76e0c13)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.187107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.188028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.218062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2828/5655] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:09:32.221117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.242109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6901ed9-6cdc-4399-87bb-f5ce4c2e9ecc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.257190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.258069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.258407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.264125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.265169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.270134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.275135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca372f2f382480a9d2ec44ad76e0c13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:32.299419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca372f2f382480a9d2ec44ad76e0c13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:32.312634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca372f2f382480a9d2ec44ad76e0c13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:32.321047 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2829/5656] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:09:32 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:09:32.322139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.325495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca372f2f382480a9d2ec44ad76e0c13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:32.327300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1192807c-6b18-4692-893f-e1855f4265a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca372f2f382480a9d2ec44ad76e0c13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:32.336724 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2828/5657] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:09:32 2022] DELETE /identity/v3/users/dca372f2f382480a9d2ec44ad76e0c13 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:32.340936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96013924-228a-4422-8201-7ed0367efa7b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.341225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96013924-228a-4422-8201-7ed0367efa7b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.343322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96013924-228a-4422-8201-7ed0367efa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.381715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-96013924-228a-4422-8201-7ed0367efa7b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:32.383880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-96013924-228a-4422-8201-7ed0367efa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:32.440157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e92e4-7f27-45de-816b-eea0a29dd745 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.527371 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2829/5658] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:32.538322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96013924-228a-4422-8201-7ed0367efa7b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.542796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-96013924-228a-4422-8201-7ed0367efa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:32.579669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:32.589791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.593576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.594185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.594610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.597586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.604821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.661129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2830/5659] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:09:32 2022] POST /identity/v3/users => generated 291 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:32.665219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.669530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96013924-228a-4422-8201-7ed0367efa7b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.670006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:32.681617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.682296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.682583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.682855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.699626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e95c3-8123-4637-9d9b-042c927f81e4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.715671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.716324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.716620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.716877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.717538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.719817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.720976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.722492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.735875 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2830/5660] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:09:32.751641 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2831/5661] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:32.752907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.778861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352069bb-6572-4569-bcd7-d94b09271712 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.788804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820cc50-8b95-4427-93f8-e5da982fdcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.790997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820cc50-8b95-4427-93f8-e5da982fdcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.791463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820cc50-8b95-4427-93f8-e5da982fdcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.791904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820cc50-8b95-4427-93f8-e5da982fdcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.793683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820cc50-8b95-4427-93f8-e5da982fdcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.796551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820cc50-8b95-4427-93f8-e5da982fdcf2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.849873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:32.851001 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2831/5662] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:32.884426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f72d36-d2c4-4b47-9cf9-a8342593f534 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:32.902921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:32.904938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.905419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.907501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] PATH_INFO: `/v3/projects/1889c0bd7fe14cc6ac7d8bd30e0cbff9/users/245e04d38c3842b5acdf096628e77025/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:32.935794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1889c0bd7fe14cc6ac7d8bd30e0cbff9, user_id=245e04d38c3842b5acdf096628e77025, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:32.939983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:32.960958 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2832/5663] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:32 2022] PUT /identity/v3/projects/1889c0bd7fe14cc6ac7d8bd30e0cbff9/users/245e04d38c3842b5acdf096628e77025/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:32.978911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:32.979465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:32.979465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.030881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:33.033729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:33.137679 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2832/5664] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:33.146100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.148446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:33.158352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.169188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aec4c8-a1fe-4c4d-97bf-de06a2014a0e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.173974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:33.176603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.177766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.178339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.178843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] PATH_INFO: `/v3/projects/811bc255a8f84b96b561d120fb4398d6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.180524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811bc255a8f84b96b561d120fb4398d6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.187801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.237072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:33.287144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811bc255a8f84b96b561d120fb4398d6', 'request_id': 'req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:33.295573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811bc255a8f84b96b561d120fb4398d6', 'request_id': 'req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:33.305498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811bc255a8f84b96b561d120fb4398d6', 'request_id': 'req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:33.315634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b8af6-e2a5-4fb8-afe3-a14d4c0629bf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.317571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] The token cache is being invalidate because project 811bc255a8f84b96b561d120fb4398d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:09:33.317904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:33.318584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8df41939-1cb8-4ea4-bd75-3861ae69cbf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:09:33.320298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2833/5665] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:09:33 2022] DELETE /identity/v3/projects/811bc255a8f84b96b561d120fb4398d6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:33.325230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d3258-6f96-47be-92f0-5f3d2e395205 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.325594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d3258-6f96-47be-92f0-5f3d2e395205 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.325997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d3258-6f96-47be-92f0-5f3d2e395205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.350697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f29d3258-6f96-47be-92f0-5f3d2e395205 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:33.352445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f29d3258-6f96-47be-92f0-5f3d2e395205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:33.375734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.377093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.377722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.378571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.380963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:33.408754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:33.432416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2834/5666] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:09:33.440714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927f3b1-e1ce-48aa-9068-f1fc3d1438eb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.441238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927f3b1-e1ce-48aa-9068-f1fc3d1438eb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.441520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927f3b1-e1ce-48aa-9068-f1fc3d1438eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.462996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6927f3b1-e1ce-48aa-9068-f1fc3d1438eb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:33.464309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6927f3b1-e1ce-48aa-9068-f1fc3d1438eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:33.466639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.468082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.473547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2835/5667] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:33.477743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927f3b1-e1ce-48aa-9068-f1fc3d1438eb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.480612 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2833/5668] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:09:33.488392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927f3b1-e1ce-48aa-9068-f1fc3d1438eb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.494056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.494664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.494962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.495220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.495895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:09:33.496204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.497979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.512963 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2836/5669] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:09:33.517282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.527824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572dd0b-d4f2-4b89-8599-17b86a6d1280 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.533736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.534933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.534933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.535190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.535651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.537630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.571646 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2834/5670] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:09:33 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:09:33.572812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.584644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d8168-f862-414d-ba20-c84e645f063b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.657119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.659210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.659571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.660127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.661090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.663577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.687936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2837/5671] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:09:33 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:09:33.692820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.703782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9ca15-a951-4fc1-9969-2d97c0b844fd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.709600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.710508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.710800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.711075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.711743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.713793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.728817 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2835/5672] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:09:33.729985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.740785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478af6-080d-401d-b006-c8b2d5db2345 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.745625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7ec52-a886-445e-b3f8-d61d8376dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.746290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7ec52-a886-445e-b3f8-d61d8376dbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.746565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7ec52-a886-445e-b3f8-d61d8376dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.746836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7ec52-a886-445e-b3f8-d61d8376dbe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.747467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7ec52-a886-445e-b3f8-d61d8376dbe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.749051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7ec52-a886-445e-b3f8-d61d8376dbe2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.761704 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2838/5673] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:09:33.766023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.779070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b9310-8790-4860-b188-a747f93ad81b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:33.784539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:33.785392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.785651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.786055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] PATH_INFO: `/v3/projects/00dbf3b5688344e2aca9a0925608f1f6/users/01d8e8b6bbf84f66b8f6ec42d20bd81e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.802292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00dbf3b5688344e2aca9a0925608f1f6, user_id=01d8e8b6bbf84f66b8f6ec42d20bd81e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:33.806305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:33.821755 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2836/5674] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:33 2022] PUT /identity/v3/projects/00dbf3b5688344e2aca9a0925608f1f6/users/01d8e8b6bbf84f66b8f6ec42d20bd81e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:33.833106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02e35a-a48b-408a-a2d8-f1fc5847ed8b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:33.833391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02e35a-a48b-408a-a2d8-f1fc5847ed8b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:33.833648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02e35a-a48b-408a-a2d8-f1fc5847ed8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:33.856887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:33.866993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0a02e35a-a48b-408a-a2d8-f1fc5847ed8b None None] MFA Rules not processed for user `01d8e8b6bbf84f66b8f6ec42d20bd81e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:33.870180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0a02e35a-a48b-408a-a2d8-f1fc5847ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:33.964811 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2839/5675] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:09:33.981860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:33.982588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:34.045715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d486d-288f-4022-8c04-2e99a11200f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:34.051097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128a21-24a7-42f1-a0e3-e3d09f31cd96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:34.051903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128a21-24a7-42f1-a0e3-e3d09f31cd96 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:34.052352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128a21-24a7-42f1-a0e3-e3d09f31cd96 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:34.052791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128a21-24a7-42f1-a0e3-e3d09f31cd96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:34.063037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128a21-24a7-42f1-a0e3-e3d09f31cd96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:34.064501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128a21-24a7-42f1-a0e3-e3d09f31cd96 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:34.076915 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2837/5676] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:09:34.353970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:34.354255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:34.354752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:34.395155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] MFA Rules not processed for user `54dcc78ee39c4119b08023d3cc692d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:34.397557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:34.546332 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2840/5677] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:42.811538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda05dc-be3b-43b2-a729-ccfb1b9e7533 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:42.812877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda05dc-be3b-43b2-a729-ccfb1b9e7533 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:42.814284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda05dc-be3b-43b2-a729-ccfb1b9e7533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:42.875619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7eda05dc-be3b-43b2-a729-ccfb1b9e7533 None None] MFA Rules not processed for user `245e04d38c3842b5acdf096628e77025`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:42.879317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7eda05dc-be3b-43b2-a729-ccfb1b9e7533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:42.983385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:43.041786 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2838/5678] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:43.042407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:43.044714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:43.161682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e0ec8-dc9d-40c2-a896-ed56fda6cf18 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:43.170694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:43.173209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:43.174329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:43.174745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:43.175697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:43.178061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:43.234593 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2841/5679] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:09:43 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:43.236479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda05dc-be3b-43b2-a729-ccfb1b9e7533 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:43.259877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda05dc-be3b-43b2-a729-ccfb1b9e7533 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:43.387814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:43.388755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:43.389246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:43.389712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:43.390652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:43.394131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:43.445945 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2839/5680] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:09:43 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:09:43.450642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:43.473613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ab449-5f8c-4114-ba17-e31233e6ce12 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:43.497028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:43.497681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:43.498004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:43.498284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:43.499863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:43.501937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:43.537287 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2842/5681] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:43.541053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:43.561257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d204c-3349-42b4-8aca-bc5394dbf334 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:43.576985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:43.578026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:43.578507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:43.579832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] PATH_INFO: `/v3/projects/b4d852fc888d4b8194be800cd6d30d9b/users/32637934818447b8af590a7edcadb9f4/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:43.617003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4d852fc888d4b8194be800cd6d30d9b, user_id=32637934818447b8af590a7edcadb9f4, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:43.621295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:43.658454 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2840/5682] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:43 2022] PUT /identity/v3/projects/b4d852fc888d4b8194be800cd6d30d9b/users/32637934818447b8af590a7edcadb9f4/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:09:43.663370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:43.695600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d1a04-28c4-41e7-af30-9e7e8e2e95b5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:43.730345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:09:43.811213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcbf7d-9195-4845-a6ea-8f84c76e8db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:43.812192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcbf7d-9195-4845-a6ea-8f84c76e8db3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:43.813654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcbf7d-9195-4845-a6ea-8f84c76e8db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:43.815469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcbf7d-9195-4845-a6ea-8f84c76e8db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:43.817520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcbf7d-9195-4845-a6ea-8f84c76e8db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:43.823571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcbf7d-9195-4845-a6ea-8f84c76e8db3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:43.854075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:43.857103 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2843/5683] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:09:43.874307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcbfc2-7931-4462-91f1-6939d3f1aa51 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:43.900128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528eff8e-01a7-43d5-b3dc-1a24bc4b72cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:43.903518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eff8e-01a7-43d5-b3dc-1a24bc4b72cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:43.903953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eff8e-01a7-43d5-b3dc-1a24bc4b72cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:43.905783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eff8e-01a7-43d5-b3dc-1a24bc4b72cf admin admin] PATH_INFO: `/v3/projects/b4d852fc888d4b8194be800cd6d30d9b/users/32637934818447b8af590a7edcadb9f4/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:43.921637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528eff8e-01a7-43d5-b3dc-1a24bc4b72cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4d852fc888d4b8194be800cd6d30d9b, user_id=32637934818447b8af590a7edcadb9f4, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:43.929800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528eff8e-01a7-43d5-b3dc-1a24bc4b72cf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:43.973285 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2841/5684] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:09:43 2022] PUT /identity/v3/projects/b4d852fc888d4b8194be800cd6d30d9b/users/32637934818447b8af590a7edcadb9f4/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:09:51.591808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:51.594063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:51.595145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:51.637320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] MFA Rules not processed for user `6bfa3af0a7284bc6854f2c9c58392935`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:51.638807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:51.747189 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2844/5685] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:53.623182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:53.623740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:53.623740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:53.667084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] MFA Rules not processed for user `32637934818447b8af590a7edcadb9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:09:53.668558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:53.777265 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2842/5686] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:09:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:09:53.790378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:09:53.792547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:09:53.879034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e5653-3f4f-4c5c-88c2-84f4cfd08629 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:09:53.889068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:09:53.889774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:09:53.890281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:09:53.890844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:09:54.009449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:09:54.011074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:09:54.024124 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2845/5687] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:09:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:03.773547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:04.018910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:11.375836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:11.377876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:11.469400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce528b2-e6c1-4431-bc0a-68c38c4f4b69 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:11.475332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:11.476226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:11.476697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:11.476993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] PATH_INFO: `/v3/users/1147de803ef34d16b39e96720b2f638b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:11.480214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1147de803ef34d16b39e96720b2f638b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:11.483404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:11.542384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1147de803ef34d16b39e96720b2f638b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.555079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1147de803ef34d16b39e96720b2f638b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.563627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1147de803ef34d16b39e96720b2f638b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.572620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1147de803ef34d16b39e96720b2f638b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.573180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5856c4d2-6cc6-42a1-a0df-e6d994fcba4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1147de803ef34d16b39e96720b2f638b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.579353 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2843/5688] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:10:11 2022] DELETE /identity/v3/users/1147de803ef34d16b39e96720b2f638b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:11.758980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:11.771348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd95d50-9161-4a5e-9aff-758bac32c645 service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:11.837793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:11.838669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:11.839137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:11.839573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] PATH_INFO: `/v3/projects/7f09210a6e9f45cc9619b180628ba208` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:11.840960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f09210a6e9f45cc9619b180628ba208)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:11.844278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:11.909697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f09210a6e9f45cc9619b180628ba208', 'request_id': 'req-7af011ee-2994-44ca-aaf5-21cf16541463'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.918295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f09210a6e9f45cc9619b180628ba208', 'request_id': 'req-7af011ee-2994-44ca-aaf5-21cf16541463'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.932282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f09210a6e9f45cc9619b180628ba208', 'request_id': 'req-7af011ee-2994-44ca-aaf5-21cf16541463'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.968427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] The token cache is being invalidate because project 7f09210a6e9f45cc9619b180628ba208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:10:11.969105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.973180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7af011ee-2994-44ca-aaf5-21cf16541463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:11.975421 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2846/5689] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:10:11 2022] DELETE /identity/v3/projects/7f09210a6e9f45cc9619b180628ba208 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:11.981570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025075de-9c71-4a79-9308-0735316a18dc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:11.982240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025075de-9c71-4a79-9308-0735316a18dc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:11.982522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025075de-9c71-4a79-9308-0735316a18dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.012063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-025075de-9c71-4a79-9308-0735316a18dc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:12.014053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-025075de-9c71-4a79-9308-0735316a18dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:12.117090 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2844/5690] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:12.125132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe7138-be84-48cb-b97c-2745fa8fe556 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.125459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe7138-be84-48cb-b97c-2745fa8fe556 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.125783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe7138-be84-48cb-b97c-2745fa8fe556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.147861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7bfe7138-be84-48cb-b97c-2745fa8fe556 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:12.149097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7bfe7138-be84-48cb-b97c-2745fa8fe556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:12.159229 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2847/5691] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:12.163761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025075de-9c71-4a79-9308-0735316a18dc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:12.175711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025075de-9c71-4a79-9308-0735316a18dc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:12.184778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:12.185414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.185702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.186019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.186735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:10:12.186990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:12.189508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:12.203029 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2845/5692] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:10:12.208197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe7138-be84-48cb-b97c-2745fa8fe556 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:12.218040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe7138-be84-48cb-b97c-2745fa8fe556 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:12.223385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:12.224030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.224333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.224608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.225232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:12.227576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:12.258009 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2848/5693] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:10:12 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:10:12.262517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:12.274987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507698cc-cec6-44af-94b5-4417f250d23f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:12.278879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:12.338674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:12.339297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.339655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.340000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.340735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:12.342875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:12.373636 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2846/5694] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:10:12 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:10:12.374713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:12.389020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256592-2516-48c4-933d-0c428c5058ea admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:12.395047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:12.395673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.396061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.396983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.397335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:12.399141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:12.417235 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2849/5695] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:10:12.417906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:12.438879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa26b1-6d49-4342-949e-d46c38c340b0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:12.444140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea333b2f-697f-4dd8-a306-c173170cab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:12.444862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea333b2f-697f-4dd8-a306-c173170cab6d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.445074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea333b2f-697f-4dd8-a306-c173170cab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.445511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea333b2f-697f-4dd8-a306-c173170cab6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.446395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea333b2f-697f-4dd8-a306-c173170cab6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:12.448084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea333b2f-697f-4dd8-a306-c173170cab6d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:12.463373 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2847/5696] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:10:12.467790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:12.482139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6c9db-5ad9-4db1-a935-ea857259d0d3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:12.488931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:12.489851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.490253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.490617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] PATH_INFO: `/v3/projects/15a76a25754c42099c542fa13046f01b/users/d83b6ceb599746b394e379d52683b70c/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.508591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a76a25754c42099c542fa13046f01b, user_id=d83b6ceb599746b394e379d52683b70c, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:12.512247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:12.533744 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2850/5697] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:10:12 2022] PUT /identity/v3/projects/15a76a25754c42099c542fa13046f01b/users/d83b6ceb599746b394e379d52683b70c/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:12.536337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd71432-cade-44fd-bf2d-ac3e170a4861 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.536632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd71432-cade-44fd-bf2d-ac3e170a4861 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.537093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd71432-cade-44fd-bf2d-ac3e170a4861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.575156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-edd71432-cade-44fd-bf2d-ac3e170a4861 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:12.576650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-edd71432-cade-44fd-bf2d-ac3e170a4861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:12.700537 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2848/5698] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:12.711893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:12.713746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:12.759110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:12.799977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04ccd9-0086-469f-a551-5512d7279574 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:12.804301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:12.804935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:12.805213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:12.805484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:12.822617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:12.824196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:12.838986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2851/5699] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:19.474099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa78318-1e39-4c33-98b5-9d81eed69e49 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:19.474670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa78318-1e39-4c33-98b5-9d81eed69e49 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:19.474916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa78318-1e39-4c33-98b5-9d81eed69e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:19.522089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-afa78318-1e39-4c33-98b5-9d81eed69e49 None None] MFA Rules not processed for user `d83b6ceb599746b394e379d52683b70c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:19.525004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-afa78318-1e39-4c33-98b5-9d81eed69e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:19.662722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2849/5700] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:19.677295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:19.680269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:19.804775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29b391-9741-4c3f-a434-027f0124f28a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:19.814419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:19.815087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:19.815373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:19.815652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:19.826011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:19.832521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:19.842663 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2852/5701] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:21.860796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:21.861727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:21.862233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:21.899048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] MFA Rules not processed for user `d83b6ceb599746b394e379d52683b70c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:21.900338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:21.913362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2850/5702] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:22.305255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:22.307624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:22.329233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add92826-7dfa-4a16-8071-24561cf30eb3 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:22.336539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:22.337252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:22.337798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:22.338256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:22.341124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:22.357244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:22.358735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:22.370110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2853/5703] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:10:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:31.891761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:32.364249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=54 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:32.952437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:32.955933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:33.106723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ab90-16d3-4a83-9a3d-65803bccf4f8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:33.115074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:33.116051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:33.116411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:33.116874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:33.120143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:33.143457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:33.145159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:33.157971 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2851/5704] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:10:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:38.746528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:38.748825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:38.790969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4432-7646-4e94-b29d-1abbe2b3fb58 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:38.798154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:38.799446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:38.799751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:38.800483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] PATH_INFO: `/v3/users/245e04d38c3842b5acdf096628e77025` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:38.802791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=245e04d38c3842b5acdf096628e77025)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:38.805032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:38.929972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245e04d38c3842b5acdf096628e77025'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:38.954201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245e04d38c3842b5acdf096628e77025'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:38.956604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245e04d38c3842b5acdf096628e77025'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:38.983136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245e04d38c3842b5acdf096628e77025'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:38.984379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245e04d38c3842b5acdf096628e77025'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:39.033759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2854/5705] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:10:38 2022] DELETE /identity/v3/users/245e04d38c3842b5acdf096628e77025 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:39.792087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:39.808368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2a4b8-14ca-40bf-9aed-fafdce9f0708 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:39.934704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:39.936331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:39.938127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:39.939177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] PATH_INFO: `/v3/projects/1889c0bd7fe14cc6ac7d8bd30e0cbff9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:39.941364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1889c0bd7fe14cc6ac7d8bd30e0cbff9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:39.944499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:40.050202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1889c0bd7fe14cc6ac7d8bd30e0cbff9', 'request_id': 'req-3d1b9826-98bd-4b99-8ce5-22845906b91b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.083691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1889c0bd7fe14cc6ac7d8bd30e0cbff9', 'request_id': 'req-3d1b9826-98bd-4b99-8ce5-22845906b91b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.101074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1889c0bd7fe14cc6ac7d8bd30e0cbff9', 'request_id': 'req-3d1b9826-98bd-4b99-8ce5-22845906b91b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.123320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] The token cache is being invalidate because project 1889c0bd7fe14cc6ac7d8bd30e0cbff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:10:40.124754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.126833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.129593 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2852/5706] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:10:39 2022] DELETE /identity/v3/projects/1889c0bd7fe14cc6ac7d8bd30e0cbff9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:40.134470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:40.136573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:40.157498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8313e-a265-41d7-b19b-0d7b3036a30e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:40.245850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:40.247197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:40.247405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:40.248004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] PATH_INFO: `/v3/users/32637934818447b8af590a7edcadb9f4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:40.251002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32637934818447b8af590a7edcadb9f4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:40.253024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:40.342740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32637934818447b8af590a7edcadb9f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.354954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32637934818447b8af590a7edcadb9f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.356328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32637934818447b8af590a7edcadb9f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.376818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32637934818447b8af590a7edcadb9f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.378864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32637934818447b8af590a7edcadb9f4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:40.410117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:40.435366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2855/5707] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:10:40 2022] DELETE /identity/v3/users/32637934818447b8af590a7edcadb9f4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:40.869025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:40.871309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:40.918745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b9826-98bd-4b99-8ce5-22845906b91b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:40.993150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:41.080495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:41.081347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:41.081505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:41.081758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] PATH_INFO: `/v3/projects/b4d852fc888d4b8194be800cd6d30d9b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:41.083024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4d852fc888d4b8194be800cd6d30d9b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:41.085629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:41.179544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4d852fc888d4b8194be800cd6d30d9b', 'request_id': 'req-77e69393-2b20-4332-b6ce-20d96029f437'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:41.188589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4d852fc888d4b8194be800cd6d30d9b', 'request_id': 'req-77e69393-2b20-4332-b6ce-20d96029f437'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:41.220381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4d852fc888d4b8194be800cd6d30d9b', 'request_id': 'req-77e69393-2b20-4332-b6ce-20d96029f437'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:41.258913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] The token cache is being invalidate because project b4d852fc888d4b8194be800cd6d30d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:10:41.259599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:41.261101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-77e69393-2b20-4332-b6ce-20d96029f437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:41.263279 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2853/5708] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:10:40 2022] DELETE /identity/v3/projects/b4d852fc888d4b8194be800cd6d30d9b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:50.430055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:51.229373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:51.233119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:51.250659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b9d70-0975-4d58-947f-c97cf141f567 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:51.256610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:51.341265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:51.342034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:51.343816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:51.344321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] PATH_INFO: `/v3/users/01d8e8b6bbf84f66b8f6ec42d20bd81e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:51.347285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d8e8b6bbf84f66b8f6ec42d20bd81e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:51.349794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:51.425881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d8e8b6bbf84f66b8f6ec42d20bd81e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:51.438640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d8e8b6bbf84f66b8f6ec42d20bd81e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:51.440195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d8e8b6bbf84f66b8f6ec42d20bd81e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:51.449100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d8e8b6bbf84f66b8f6ec42d20bd81e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:51.449998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d8e8b6bbf84f66b8f6ec42d20bd81e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:51.474453 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2856/5709] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:10:51 2022] DELETE /identity/v3/users/01d8e8b6bbf84f66b8f6ec42d20bd81e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:10:51.479010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:51.479406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:51.480102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:51.507714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:51.507714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:51.586534 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2854/5710] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:10:51.606667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:51.609250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:51.724165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd1e37-3b52-410d-97dc-368599f5de56 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:51.748057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1519a2-874e-42d5-baec-771fc8c138df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:51.750198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1519a2-874e-42d5-baec-771fc8c138df service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:51.750594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1519a2-874e-42d5-baec-771fc8c138df service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:51.751540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1519a2-874e-42d5-baec-771fc8c138df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:51.767908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1519a2-874e-42d5-baec-771fc8c138df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:51.769647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1519a2-874e-42d5-baec-771fc8c138df service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:51.788984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2857/5711] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:52.195272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:52.197113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:52.213099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9de73-a162-4f52-8759-e252b639b060 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:52.220459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:52.221323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.221743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.222116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] PATH_INFO: `/v3/projects/00dbf3b5688344e2aca9a0925608f1f6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.223470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=00dbf3b5688344e2aca9a0925608f1f6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:52.225935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:52.310543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00dbf3b5688344e2aca9a0925608f1f6', 'request_id': 'req-66499ed1-ae84-4514-bdfe-5cd3a17080aa'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:52.320024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00dbf3b5688344e2aca9a0925608f1f6', 'request_id': 'req-66499ed1-ae84-4514-bdfe-5cd3a17080aa'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:52.330464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00dbf3b5688344e2aca9a0925608f1f6', 'request_id': 'req-66499ed1-ae84-4514-bdfe-5cd3a17080aa'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:52.348265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] The token cache is being invalidate because project 00dbf3b5688344e2aca9a0925608f1f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:10:52.348563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:52.349353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-66499ed1-ae84-4514-bdfe-5cd3a17080aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:52.354582 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2855/5712] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:10:52 2022] DELETE /identity/v3/projects/00dbf3b5688344e2aca9a0925608f1f6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:52.358853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40e298-e7ea-4662-85b5-d6f294a7d11b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.359356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40e298-e7ea-4662-85b5-d6f294a7d11b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.359901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40e298-e7ea-4662-85b5-d6f294a7d11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.387430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8c40e298-e7ea-4662-85b5-d6f294a7d11b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:52.388732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8c40e298-e7ea-4662-85b5-d6f294a7d11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:52.415717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:52.487292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2858/5713] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:52.495133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e82b3c-c4a6-45df-83fd-6dd05ac79fd8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.495444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e82b3c-c4a6-45df-83fd-6dd05ac79fd8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.495723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e82b3c-c4a6-45df-83fd-6dd05ac79fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.521958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-78e82b3c-c4a6-45df-83fd-6dd05ac79fd8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:52.523199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-78e82b3c-c4a6-45df-83fd-6dd05ac79fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:52.533474 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2856/5714] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:52.537637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40e298-e7ea-4662-85b5-d6f294a7d11b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:52.550014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40e298-e7ea-4662-85b5-d6f294a7d11b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:52.555623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:52.556396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.556678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.556979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.557721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:10:52.558099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:52.559668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:52.573315 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2859/5715] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:10:52.578746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e82b3c-c4a6-45df-83fd-6dd05ac79fd8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:52.588570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e82b3c-c4a6-45df-83fd-6dd05ac79fd8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:52.594794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:52.595623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.595801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.596133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.596809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:52.599166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:52.634787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2857/5716] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:10:52 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:10:52.635929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:52.647531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a74d2-d2ac-4d0f-bb11-c46f9fa74afb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:52.725585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:52.726299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.726634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.726968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.727837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:52.730290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:52.758868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2860/5717] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:10:52 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:10:52.763283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:52.776047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b48ff-c339-4653-aa17-5d1f357472f4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:52.782073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:52.782976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.783499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.783972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.784590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:52.786719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:52.801924 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2858/5718] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:10:52.803125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:52.815171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533046b4-0adb-4a3e-b80b-c26612112f8a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:52.819691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ce608-e291-4ad2-84e6-5a6381b5ba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:52.820737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce608-e291-4ad2-84e6-5a6381b5ba04 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.820737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce608-e291-4ad2-84e6-5a6381b5ba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.820992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce608-e291-4ad2-84e6-5a6381b5ba04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.821461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ce608-e291-4ad2-84e6-5a6381b5ba04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:52.823110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ce608-e291-4ad2-84e6-5a6381b5ba04 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:52.834954 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2861/5719] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:10:52.839439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:52.851281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42397abd-7a13-4223-bee2-4ffe28088e71 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:52.857730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:52.858569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.858977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.859298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] PATH_INFO: `/v3/projects/890c9456a15440f38f74ec0acdc4894e/users/054335f339424a499a1c2a1e91ae3f55/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.876592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=890c9456a15440f38f74ec0acdc4894e, user_id=054335f339424a499a1c2a1e91ae3f55, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:52.879843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:52.895600 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2859/5720] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:10:52 2022] PUT /identity/v3/projects/890c9456a15440f38f74ec0acdc4894e/users/054335f339424a499a1c2a1e91ae3f55/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:10:52.908078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7b35-9569-4c92-8a31-52c6bef1fd57 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:52.908365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7b35-9569-4c92-8a31-52c6bef1fd57 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:52.908903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af7b35-9569-4c92-8a31-52c6bef1fd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:52.940485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a7af7b35-9569-4c92-8a31-52c6bef1fd57 None None] MFA Rules not processed for user `054335f339424a499a1c2a1e91ae3f55`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:52.941201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7af7b35-9569-4c92-8a31-52c6bef1fd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:53.024271 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2862/5721] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:53.036642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:53.038753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:53.044977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:53.045393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:53.046447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:53.065976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:53.070280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] MFA Rules not processed for user `13e5077421c44eb7abcda40582c7ea9a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:10:53.072125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:53.089739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:10:53.118010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23a7c8-86db-4be8-b31c-c3f327baeaf3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:53.123301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:53.124110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:53.124712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:53.125094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:53.136662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:53.138399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:53.146021 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2863/5722] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:10:53.149702 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2860/5723] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:10:59.432423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:10:59.434064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:10:59.561389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569b7c9-9ed1-4934-93f3-f657b92dc53c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:10:59.570680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:10:59.571530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:10:59.571916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:10:59.572279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] PATH_INFO: `/v3/users/6bfa3af0a7284bc6854f2c9c58392935` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:10:59.573899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bfa3af0a7284bc6854f2c9c58392935)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:10:59.582655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:10:59.689552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bfa3af0a7284bc6854f2c9c58392935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:59.714700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bfa3af0a7284bc6854f2c9c58392935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:59.716167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bfa3af0a7284bc6854f2c9c58392935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:59.730405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bfa3af0a7284bc6854f2c9c58392935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:59.731812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c115e385-f2ea-4c94-b304-47843da2711f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bfa3af0a7284bc6854f2c9c58392935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:10:59.773171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2864/5724] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:10:59 2022] DELETE /identity/v3/users/6bfa3af0a7284bc6854f2c9c58392935 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:00.154408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:00.183069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c55b-beb4-4020-9d6d-11869f6f6a1a service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:00.298306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:00.310636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:00.312737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:00.314700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] PATH_INFO: `/v3/projects/27acb763ffd24512afe7ca5e1d06f031` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:00.321901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=27acb763ffd24512afe7ca5e1d06f031)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:00.324434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:00.470113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27acb763ffd24512afe7ca5e1d06f031', 'request_id': 'req-054464a5-a655-4816-91d5-7dd82e8cdb4f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:00.488145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27acb763ffd24512afe7ca5e1d06f031', 'request_id': 'req-054464a5-a655-4816-91d5-7dd82e8cdb4f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:00.506325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27acb763ffd24512afe7ca5e1d06f031', 'request_id': 'req-054464a5-a655-4816-91d5-7dd82e8cdb4f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:00.540650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] The token cache is being invalidate because project 27acb763ffd24512afe7ca5e1d06f031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:00.541120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:00.541995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-054464a5-a655-4816-91d5-7dd82e8cdb4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:00.545951 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2861/5725] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:00 2022] DELETE /identity/v3/projects/27acb763ffd24512afe7ca5e1d06f031 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:00.550866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944fcb7-62c9-4113-ab5b-ef7c19268aba None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:00.551379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944fcb7-62c9-4113-ab5b-ef7c19268aba None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:00.551794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944fcb7-62c9-4113-ab5b-ef7c19268aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:00.605680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b944fcb7-62c9-4113-ab5b-ef7c19268aba None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:00.609977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b944fcb7-62c9-4113-ab5b-ef7c19268aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:00.725967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:00.765784 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2865/5726] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:00.766832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655482da-90ad-4df2-9bfd-54808812c261 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:00.767387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655482da-90ad-4df2-9bfd-54808812c261 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:00.767763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655482da-90ad-4df2-9bfd-54808812c261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:00.797065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-655482da-90ad-4df2-9bfd-54808812c261 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:00.799229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-655482da-90ad-4df2-9bfd-54808812c261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:00.822655 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2862/5727] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:00.826265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944fcb7-62c9-4113-ab5b-ef7c19268aba None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:00.840226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944fcb7-62c9-4113-ab5b-ef7c19268aba None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:00.851497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:00.852834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:00.856499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:00.856968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:00.862999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:00.869222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:00.871399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:00.924913 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2866/5728] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:00.929505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655482da-90ad-4df2-9bfd-54808812c261 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:00.950528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655482da-90ad-4df2-9bfd-54808812c261 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:00.956111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:00.956874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:00.957154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:00.957429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:00.958176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:00.960392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:01.009183 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2863/5729] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:00 2022] POST /identity/v3/projects => generated 401 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:01.013524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:01.025879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb1afc-e0e0-4b62-a3ef-ec4ca82ddeb5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:01.129550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:01.130374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:01.131794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:01.132210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:01.132985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:01.135389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:01.183791 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2867/5730] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:01 2022] POST /identity/v3/users => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:01.190829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:01.205899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fb4bf-bba8-41c3-9de9-29a71fc146b0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:01.215604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:01.216254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:01.217001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:01.218022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:01.218839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:01.221517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:01.236390 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2864/5731] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:01.240578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:01.255318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f7b17-8cb8-401e-a8ee-b1124f0097b0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:01.261066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a247-aea9-459c-af17-97e01892b243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:01.261740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a247-aea9-459c-af17-97e01892b243 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:01.262204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a247-aea9-459c-af17-97e01892b243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:01.262500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a247-aea9-459c-af17-97e01892b243 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:01.263576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a247-aea9-459c-af17-97e01892b243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:01.265753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a247-aea9-459c-af17-97e01892b243 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:01.278984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2868/5732] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:01.284484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:01.303361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1709c-ca1f-4ec7-af44-8f5380ba5866 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:01.313267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:01.314535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:01.315075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:01.315614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] PATH_INFO: `/v3/projects/f7ee66f8c7434a0082e825634b47383d/users/77ccd529f9234e40a217b05694fc39e6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:01.339333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:01.350463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ee66f8c7434a0082e825634b47383d, user_id=77ccd529f9234e40a217b05694fc39e6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:01.354842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:01.382637 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2865/5733] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:01 2022] PUT /identity/v3/projects/f7ee66f8c7434a0082e825634b47383d/users/77ccd529f9234e40a217b05694fc39e6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:01.396243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cb093-c7af-4901-970f-b546df71cf84 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:01.396518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cb093-c7af-4901-970f-b546df71cf84 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:01.396763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cb093-c7af-4901-970f-b546df71cf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:01.444728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-333cb093-c7af-4901-970f-b546df71cf84 None None] MFA Rules not processed for user `77ccd529f9234e40a217b05694fc39e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:01.446729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-333cb093-c7af-4901-970f-b546df71cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:01.567631 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2869/5734] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:01.581925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:01.694004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ec27b-bfef-4ab9-8a41-df0fdf508d45 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:01.700344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:01.701218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:01.702563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:01.703167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] PATH_INFO: `/v3/users/77ccd529f9234e40a217b05694fc39e6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:01.704784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ccd529f9234e40a217b05694fc39e6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:01.709814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:01.878726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ccd529f9234e40a217b05694fc39e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:01.914233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ccd529f9234e40a217b05694fc39e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:01.930688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ccd529f9234e40a217b05694fc39e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:01.949022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ccd529f9234e40a217b05694fc39e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:01.949816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ccd529f9234e40a217b05694fc39e6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:01.966897 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2866/5735] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:01 2022] DELETE /identity/v3/users/77ccd529f9234e40a217b05694fc39e6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:01.969646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:01.970348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:01.970888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:02.022425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:02.024048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:02.083554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:02.124885 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2870/5736] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:02.135116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:02.136858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:02.264943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b4b8-0583-410b-9d4a-21b76f38f066 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:02.268928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794a1f1-9b12-42a1-99d3-9b2fd73c24cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:02.269597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794a1f1-9b12-42a1-99d3-9b2fd73c24cc service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:02.269931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794a1f1-9b12-42a1-99d3-9b2fd73c24cc service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:02.270207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794a1f1-9b12-42a1-99d3-9b2fd73c24cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:02.289787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794a1f1-9b12-42a1-99d3-9b2fd73c24cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:02.291283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794a1f1-9b12-42a1-99d3-9b2fd73c24cc service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:02.308950 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2867/5737] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:02.736543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:02.738985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:02.761090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab053d32-db64-4720-9e6f-3df27fe3064b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:02.785080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:02.797423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:02.797772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:02.798407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] PATH_INFO: `/v3/projects/f7ee66f8c7434a0082e825634b47383d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:02.800181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ee66f8c7434a0082e825634b47383d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:02.803033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:02.898874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ee66f8c7434a0082e825634b47383d', 'request_id': 'req-60207662-0729-43a7-99e4-ee1d8457600e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:02.909151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ee66f8c7434a0082e825634b47383d', 'request_id': 'req-60207662-0729-43a7-99e4-ee1d8457600e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:02.919576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ee66f8c7434a0082e825634b47383d', 'request_id': 'req-60207662-0729-43a7-99e4-ee1d8457600e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:02.939661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] The token cache is being invalidate because project f7ee66f8c7434a0082e825634b47383d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:02.939908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:02.940431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60207662-0729-43a7-99e4-ee1d8457600e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:02.942040 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2871/5738] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:02 2022] DELETE /identity/v3/projects/f7ee66f8c7434a0082e825634b47383d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:02.947703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb1b61-6a81-4a26-bbf9-c4bc30116088 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:02.947997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb1b61-6a81-4a26-bbf9-c4bc30116088 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:02.948274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb1b61-6a81-4a26-bbf9-c4bc30116088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:02.969114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-80bb1b61-6a81-4a26-bbf9-c4bc30116088 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:02.970381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-80bb1b61-6a81-4a26-bbf9-c4bc30116088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:02.977238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:03.059794 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2868/5739] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:03.080620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74582b4a-be73-442f-9e04-1f0067310b43 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.080916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74582b4a-be73-442f-9e04-1f0067310b43 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.081202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74582b4a-be73-442f-9e04-1f0067310b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.119345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-74582b4a-be73-442f-9e04-1f0067310b43 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:03.120831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74582b4a-be73-442f-9e04-1f0067310b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:03.132787 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2872/5740] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:03.137243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb1b61-6a81-4a26-bbf9-c4bc30116088 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.153263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb1b61-6a81-4a26-bbf9-c4bc30116088 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:03.160734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:03.161757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.162262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.162698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.163509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:03.163816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:03.166623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:03.186508 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2869/5741] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:03.189509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74582b4a-be73-442f-9e04-1f0067310b43 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.202873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74582b4a-be73-442f-9e04-1f0067310b43 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:03.209681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:03.210533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.210854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.211385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.212504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:03.215374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:03.258394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2873/5742] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:03 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:03.260038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.273631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7d1b4-2e6a-43a0-a9e4-ff5cdee94c68 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:03.386825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:03.388132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.388603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.389123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.390691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:03.393155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:03.425609 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2870/5743] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:03 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:03.430014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.445230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa76676-3928-487e-8752-a0ac75e927af admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:03.452488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:03.453270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.453547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.453816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.454557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:03.456562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:03.472889 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2874/5744] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:03.477365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.495175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b2758-2837-4415-b92e-e62e33bc11c7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:03.503131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c8040-32c0-4d63-8d65-8d5b29907ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:03.504248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c8040-32c0-4d63-8d65-8d5b29907ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.504747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c8040-32c0-4d63-8d65-8d5b29907ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.505442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c8040-32c0-4d63-8d65-8d5b29907ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.506336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c8040-32c0-4d63-8d65-8d5b29907ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:03.508381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c8040-32c0-4d63-8d65-8d5b29907ed1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:03.526292 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2871/5745] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:11:03.533350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.550388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b4f5-b3ac-403e-8ae1-a5b852d571b5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:03.557465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:03.558496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.558642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.558895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] PATH_INFO: `/v3/projects/2d055111e2e44ed7afe3b335094df69a/users/3f03d688d0ee4d2b951828a3f496c914/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.581906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d055111e2e44ed7afe3b335094df69a, user_id=3f03d688d0ee4d2b951828a3f496c914, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:03.584776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:03.603238 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2875/5746] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:03 2022] PUT /identity/v3/projects/2d055111e2e44ed7afe3b335094df69a/users/3f03d688d0ee4d2b951828a3f496c914/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:03.610073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.610697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.611255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.645386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:03.646696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:03.724711 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2872/5747] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:03.735432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.737321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:03.806065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb858be7-d438-47c1-8fc1-6e8a80f440ef admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:03.810774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:03.812151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:03.812603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:03.812839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:03.824045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:03.825475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:03.830853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:03.838496 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2876/5748] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:11:03.956146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:03.959491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:03.999470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:04.085639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25c09-6462-464c-bc0a-4bfaeff8e163 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:04.095858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068cd6b-f53a-428b-a94d-2763b7387d5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:04.098224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068cd6b-f53a-428b-a94d-2763b7387d5c service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:04.099062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068cd6b-f53a-428b-a94d-2763b7387d5c service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:04.099062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068cd6b-f53a-428b-a94d-2763b7387d5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:04.101695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f068cd6b-f53a-428b-a94d-2763b7387d5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:04.121268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068cd6b-f53a-428b-a94d-2763b7387d5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:04.124571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068cd6b-f53a-428b-a94d-2763b7387d5c service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:04.141693 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2873/5749] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:11.721294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:11.722819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:11.906252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc709b-eee6-4cb1-a4f0-49fe29c00265 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:11.934012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:11.934657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:11.935327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:11.935692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:11.940571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:12.107156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:12.109505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:12.138509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2877/5750] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:12.728245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd4ab7-75c3-465e-bf6e-828e14c3c88b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:12.728245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd4ab7-75c3-465e-bf6e-828e14c3c88b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:12.728245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd4ab7-75c3-465e-bf6e-828e14c3c88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:12.770396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-37dd4ab7-75c3-465e-bf6e-828e14c3c88b None None] MFA Rules not processed for user `3f03d688d0ee4d2b951828a3f496c914`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:12.771746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-37dd4ab7-75c3-465e-bf6e-828e14c3c88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:12.880715 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2874/5751] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:12.891483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:12.904536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4cc52-447a-4c6e-a82c-10311fb0f796 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:12.913375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:12.915869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:12.916162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:12.916612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:12.927501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:12.929246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:12.940843 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2878/5752] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:17.992471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c90c5-f566-47d1-aa8d-30f90b2ab034 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:17.993421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c90c5-f566-47d1-aa8d-30f90b2ab034 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:17.993694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c90c5-f566-47d1-aa8d-30f90b2ab034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:18.052340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0e5c90c5-f566-47d1-aa8d-30f90b2ab034 None None] MFA Rules not processed for user `3f03d688d0ee4d2b951828a3f496c914`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:18.054959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0e5c90c5-f566-47d1-aa8d-30f90b2ab034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:18.068697 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2875/5753] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:20.491398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:20.493216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:20.511004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee169ce-98b5-4b08-a751-756371d7e968 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:20.517614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:20.519288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:20.519727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:20.520232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] PATH_INFO: `/v3/users/054335f339424a499a1c2a1e91ae3f55` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:20.521699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054335f339424a499a1c2a1e91ae3f55)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:20.523716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:20.584949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054335f339424a499a1c2a1e91ae3f55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:20.594600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054335f339424a499a1c2a1e91ae3f55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:20.595342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054335f339424a499a1c2a1e91ae3f55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:20.605593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054335f339424a499a1c2a1e91ae3f55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:20.606631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054335f339424a499a1c2a1e91ae3f55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:20.622125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:20.633982 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2879/5754] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:20 2022] DELETE /identity/v3/users/054335f339424a499a1c2a1e91ae3f55 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:20.637074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:20.637619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:20.637855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:20.669672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:20.673439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:20.786381 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2876/5755] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:20.797589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:20.799965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:20.895762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d074bd-0c31-44e7-b2e7-eb4401860684 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:20.900387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc94fc2-4fbd-4ff2-b655-7563c285b98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:20.901079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94fc2-4fbd-4ff2-b655-7563c285b98f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:20.902154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94fc2-4fbd-4ff2-b655-7563c285b98f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:20.902468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94fc2-4fbd-4ff2-b655-7563c285b98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:20.915719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc94fc2-4fbd-4ff2-b655-7563c285b98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:20.917683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc94fc2-4fbd-4ff2-b655-7563c285b98f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:20.930762 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2880/5756] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:11:21.408519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:21.410686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:21.427664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76c332-e250-476a-b8bb-922379db9a43 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:21.439176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:21.439956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:21.440308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:21.440660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] PATH_INFO: `/v3/projects/890c9456a15440f38f74ec0acdc4894e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:21.444071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=890c9456a15440f38f74ec0acdc4894e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:21.449895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:21.495924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:21.527253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890c9456a15440f38f74ec0acdc4894e', 'request_id': 'req-ab999d22-ef65-4147-bc6a-5a5b2b02f052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:21.535519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890c9456a15440f38f74ec0acdc4894e', 'request_id': 'req-ab999d22-ef65-4147-bc6a-5a5b2b02f052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:21.543243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890c9456a15440f38f74ec0acdc4894e', 'request_id': 'req-ab999d22-ef65-4147-bc6a-5a5b2b02f052'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:21.557865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] The token cache is being invalidate because project 890c9456a15440f38f74ec0acdc4894e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:21.558165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:21.558851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab999d22-ef65-4147-bc6a-5a5b2b02f052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:21.564113 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2877/5757] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:21 2022] DELETE /identity/v3/projects/890c9456a15440f38f74ec0acdc4894e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:21.566359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf8a6e-2c29-4243-b8b7-a112edf32613 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:21.566844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf8a6e-2c29-4243-b8b7-a112edf32613 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:21.567507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf8a6e-2c29-4243-b8b7-a112edf32613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:21.594517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b8cf8a6e-2c29-4243-b8b7-a112edf32613 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:21.596107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b8cf8a6e-2c29-4243-b8b7-a112edf32613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:21.721547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2881/5758] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:21.730168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8a21e-55fd-4e38-9e2e-b1a49711cfe2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:21.731180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8a21e-55fd-4e38-9e2e-b1a49711cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:21.732132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8a21e-55fd-4e38-9e2e-b1a49711cfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:21.757751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a7d8a21e-55fd-4e38-9e2e-b1a49711cfe2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:21.759025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a7d8a21e-55fd-4e38-9e2e-b1a49711cfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:21.770737 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2878/5759] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:21.775521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf8a6e-2c29-4243-b8b7-a112edf32613 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:21.802568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf8a6e-2c29-4243-b8b7-a112edf32613 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:21.814329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:21.815130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:21.815805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:21.815805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:21.816638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:21.817070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:21.819401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:21.839960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2882/5760] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:21.845049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8a21e-55fd-4e38-9e2e-b1a49711cfe2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:21.856812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8a21e-55fd-4e38-9e2e-b1a49711cfe2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:21.865228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:21.866152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:21.867277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:21.869790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:21.870878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:21.873514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:21.918392 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2879/5761] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:21 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:21.924311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:21.944636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686b015-7910-40ff-b659-8e3a28c40999 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:22.070325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:22.071622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.072554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.073535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.074467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:22.078261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:22.123756 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2883/5762] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:21 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:22.127857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:22.169431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0d479-3f43-44c7-acba-54f109a32924 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:22.180000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:22.181073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.181527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.182076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.183136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:22.186853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:22.209668 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2880/5763] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:22.216152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:22.229796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67692e-cf91-4df7-b1ec-458ce22714e9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:22.240030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cecc06-4dcb-4053-8545-2695e7413852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:22.241568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cecc06-4dcb-4053-8545-2695e7413852 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.242111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cecc06-4dcb-4053-8545-2695e7413852 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.242585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cecc06-4dcb-4053-8545-2695e7413852 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.243399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cecc06-4dcb-4053-8545-2695e7413852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:22.245855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cecc06-4dcb-4053-8545-2695e7413852 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:22.267884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2884/5764] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:22.271952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:22.295514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e5b5a-60b1-4a3c-ae01-b7714a5a96bc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:22.306329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b716ba-57ca-4416-8faa-3c9ac510a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:22.307158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b716ba-57ca-4416-8faa-3c9ac510a861 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.307515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b716ba-57ca-4416-8faa-3c9ac510a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.307892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b716ba-57ca-4416-8faa-3c9ac510a861 admin admin] PATH_INFO: `/v3/projects/72ebc5db5d674c988875f9b4832f066b/users/67c513a3d868454780a301cbff0e8254/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.344801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b716ba-57ca-4416-8faa-3c9ac510a861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ebc5db5d674c988875f9b4832f066b, user_id=67c513a3d868454780a301cbff0e8254, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:22.351170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b716ba-57ca-4416-8faa-3c9ac510a861 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:22.378876 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2881/5765] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:22 2022] PUT /identity/v3/projects/72ebc5db5d674c988875f9b4832f066b/users/67c513a3d868454780a301cbff0e8254/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:22.390332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.390887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.391401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.419520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:22.433618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.433933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.434196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.497421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] MFA Rules not processed for user `67c513a3d868454780a301cbff0e8254`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:22.499193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:22.535879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:22.537684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:22.605612 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2885/5766] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:22.638392 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2882/5767] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:22.647374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:22.649232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:22.739320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:22.742348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:22.800138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12abe5a-c12a-4907-940b-0fe1efab54ab None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:22.805472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:22.806478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.806768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.807043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.819594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:22.821322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:22.825499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:22.839153 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2886/5768] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:22.861466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:22.940215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c33fa4-e83f-40b9-80bb-dca844b8117e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:22.980899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbfcb0-eab8-4375-827f-d7c72347a1b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:22.981640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbfcb0-eab8-4375-827f-d7c72347a1b7 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:22.982085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbfcb0-eab8-4375-827f-d7c72347a1b7 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:22.983223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbfcb0-eab8-4375-827f-d7c72347a1b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:22.996421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbfcb0-eab8-4375-827f-d7c72347a1b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:22.999572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbfcb0-eab8-4375-827f-d7c72347a1b7 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:23.013111 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2883/5769] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:11:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:11:23.023722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1b2b1-aace-449f-9d64-810247ef3b0b service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:23.031058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:23.031866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:23.032192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:23.032537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] PATH_INFO: `/v3/users/67c513a3d868454780a301cbff0e8254` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:23.034395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c513a3d868454780a301cbff0e8254)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:23.036998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:23.163418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:23.206291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c513a3d868454780a301cbff0e8254'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:23.251497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c513a3d868454780a301cbff0e8254'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:23.252563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c513a3d868454780a301cbff0e8254'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:23.273070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c513a3d868454780a301cbff0e8254'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:23.279133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c513a3d868454780a301cbff0e8254'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:23.318139 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2887/5770] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:22 2022] DELETE /identity/v3/users/67c513a3d868454780a301cbff0e8254 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:23.319944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:23.320230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:23.320505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:23.348033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:23.349259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:23.538218 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2884/5771] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:23.550945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:23.555804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:23.777567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5ced-a5d8-474c-bb0f-f6953f010430 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:23.784249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad5ee3-e2a1-43bc-a187-92d56f353a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:23.784971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad5ee3-e2a1-43bc-a187-92d56f353a7d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:23.785249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad5ee3-e2a1-43bc-a187-92d56f353a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:23.785522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad5ee3-e2a1-43bc-a187-92d56f353a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:23.803176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad5ee3-e2a1-43bc-a187-92d56f353a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:23.805184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad5ee3-e2a1-43bc-a187-92d56f353a7d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:23.838610 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2888/5772] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:24.924985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:24.927019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:24.962117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c345937-b89e-4eca-ab48-d2eea90cc5f5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:24.974983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:24.975817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:24.976440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:24.976603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] PATH_INFO: `/v3/projects/72ebc5db5d674c988875f9b4832f066b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:24.988750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ebc5db5d674c988875f9b4832f066b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:24.995958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:25.127836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ebc5db5d674c988875f9b4832f066b', 'request_id': 'req-58573fa1-5b33-4069-a5d0-8f56372e29c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:25.137499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ebc5db5d674c988875f9b4832f066b', 'request_id': 'req-58573fa1-5b33-4069-a5d0-8f56372e29c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:25.152457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ebc5db5d674c988875f9b4832f066b', 'request_id': 'req-58573fa1-5b33-4069-a5d0-8f56372e29c7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:25.155708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe5708-9e1e-45dc-b094-bf4eecc183b9 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.156256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe5708-9e1e-45dc-b094-bf4eecc183b9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.156469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe5708-9e1e-45dc-b094-bf4eecc183b9 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.158708 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2889/5773] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:11:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 21:11:25.165698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017993-8063-42bd-8f6c-b452692205e9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.166146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017993-8063-42bd-8f6c-b452692205e9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.166420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017993-8063-42bd-8f6c-b452692205e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.174961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] The token cache is being invalidate because project 72ebc5db5d674c988875f9b4832f066b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:25.176352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:25.176352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-58573fa1-5b33-4069-a5d0-8f56372e29c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:25.182445 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2885/5774] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:24 2022] DELETE /identity/v3/projects/72ebc5db5d674c988875f9b4832f066b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:25.183952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.184363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.185040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.203829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8a017993-8063-42bd-8f6c-b452692205e9 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:25.205796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8a017993-8063-42bd-8f6c-b452692205e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:25.226427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:25.228301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:25.303735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:25.364395 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2890/5775] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:25.364998 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2886/5776] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:25.378147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef86e7-012f-4d3e-b719-14cb9d898789 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.378374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef86e7-012f-4d3e-b719-14cb9d898789 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.378568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef86e7-012f-4d3e-b719-14cb9d898789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.390243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:25.392840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:25.405622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-58ef86e7-012f-4d3e-b719-14cb9d898789 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:25.408662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-58ef86e7-012f-4d3e-b719-14cb9d898789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:25.421146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2891/5777] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:25.424622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aefc6-8b0f-460f-a0f4-9865e1ae4d46 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:25.425696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef86e7-012f-4d3e-b719-14cb9d898789 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:25.433484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:25.434089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.434390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.434652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.449131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef86e7-012f-4d3e-b719-14cb9d898789 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:25.453340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:25.454952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:25.464188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:25.464719 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2887/5778] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:25.465093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.466006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.466785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.467514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:25.468158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:25.471391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:25.492828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2892/5779] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:25.493332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:25.509346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ea7b1-ab93-4d05-9a3f-0eb8ce92d25c service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:25.516865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba8248-5f66-4be9-a0f5-91f31388763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:25.518781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba8248-5f66-4be9-a0f5-91f31388763d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.519237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba8248-5f66-4be9-a0f5-91f31388763d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.519690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba8248-5f66-4be9-a0f5-91f31388763d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.520516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba8248-5f66-4be9-a0f5-91f31388763d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:25.522634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba8248-5f66-4be9-a0f5-91f31388763d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:25.577394 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2888/5780] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:25 2022] POST /identity/v3/projects => generated 403 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:11:25.586724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:25.607485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63614230-28b3-4b2b-8e40-67806e002e17 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:25.662172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb25e-f357-4fe8-adee-46b3867154d8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.662430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb25e-f357-4fe8-adee-46b3867154d8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.662714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053eb25e-f357-4fe8-adee-46b3867154d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.690438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-053eb25e-f357-4fe8-adee-46b3867154d8 None None] MFA Rules not processed for user `54dcc78ee39c4119b08023d3cc692d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:25.692484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-053eb25e-f357-4fe8-adee-46b3867154d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:25.698063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:25.722967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:25.724219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.724683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.725186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.726028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:25.729966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:25.762532 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2893/5781] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:25 2022] POST /identity/v3/users => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:25.770555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:25.787528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fb0f5-e30d-4d81-be54-41ee56b5c0cd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:25.798139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:25.801440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.805088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.806642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.808477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:25.819177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:25.838522 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2889/5782] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:25.845104 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2894/5783] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:25.850498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053eb25e-f357-4fe8-adee-46b3867154d8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:25.870574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053eb25e-f357-4fe8-adee-46b3867154d8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:25.884870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b064d-8ef9-4b01-aea8-bdcf2217b392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:25.885734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b064d-8ef9-4b01-aea8-bdcf2217b392 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.886189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b064d-8ef9-4b01-aea8-bdcf2217b392 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.886567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b064d-8ef9-4b01-aea8-bdcf2217b392 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.888222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b064d-8ef9-4b01-aea8-bdcf2217b392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:25.894938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b064d-8ef9-4b01-aea8-bdcf2217b392 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:25.913985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2890/5784] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:25.914576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:25.928763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2b689-a87e-443f-9ff3-5847c5474c9b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:25.940634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:25.941329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:25.941669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:25.942016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] PATH_INFO: `/v3/projects/fc1ce6381322403599f89ab2965851fb/users/6502ed76b6204e4db289c37cc0d181aa/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:25.996468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1ce6381322403599f89ab2965851fb, user_id=6502ed76b6204e4db289c37cc0d181aa, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:26.006464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:26.041540 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2895/5785] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:25 2022] PUT /identity/v3/projects/fc1ce6381322403599f89ab2965851fb/users/6502ed76b6204e4db289c37cc0d181aa/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:11:26.053540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494e2cb-b5e5-4101-b28a-d9019d76c8b9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:26.054431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494e2cb-b5e5-4101-b28a-d9019d76c8b9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:26.054431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494e2cb-b5e5-4101-b28a-d9019d76c8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:26.118882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1494e2cb-b5e5-4101-b28a-d9019d76c8b9 None None] MFA Rules not processed for user `6502ed76b6204e4db289c37cc0d181aa`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:26.120226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1494e2cb-b5e5-4101-b28a-d9019d76c8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:26.261848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:26.269111 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2891/5786] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:26.373688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351eb0c-3031-436d-a991-ebe580d4c786 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:26.382582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:26.383294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:26.383761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:26.384657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:26.385272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:26.387394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:26.434011 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2896/5787] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:26 2022] POST /identity/v3/projects => generated 405 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:26.437616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494e2cb-b5e5-4101-b28a-d9019d76c8b9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:26.463814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494e2cb-b5e5-4101-b28a-d9019d76c8b9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:26.498387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:26.565497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:26.566723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:26.567708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:26.568086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:26.568761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:26.573068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:26.602455 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2892/5788] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:26 2022] POST /identity/v3/users => generated 309 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:26.606552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:26.620826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf730c4-8f56-4804-9b76-0d4ec27eabdb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:26.647783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:26.649088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:26.649422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:26.649759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:26.650520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:26.652427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:26.671297 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2897/5789] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:26.672740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:26.689427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760df08-39d5-415c-96d1-642d86c0f628 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:26.697480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:26.697480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:26.697480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:26.698025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] PATH_INFO: `/v3/projects/4e1bd16c4a334102b1ea89a568b9335c/users/e88e1041140441c8ba4cf4c4b35ba601/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:26.722287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1bd16c4a334102b1ea89a568b9335c, user_id=e88e1041140441c8ba4cf4c4b35ba601, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:26.725608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:26.747565 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2893/5790] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:26 2022] PUT /identity/v3/projects/4e1bd16c4a334102b1ea89a568b9335c/users/e88e1041140441c8ba4cf4c4b35ba601/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:26.762044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:26.790397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600458d2-ca38-4c3f-a902-8e444acf3645 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:26.891972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aeb8a-f59b-448e-804c-2bedf8ba6fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:26.892597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aeb8a-f59b-448e-804c-2bedf8ba6fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:26.892881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aeb8a-f59b-448e-804c-2bedf8ba6fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:26.893156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aeb8a-f59b-448e-804c-2bedf8ba6fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:26.893808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aeb8a-f59b-448e-804c-2bedf8ba6fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:26.896007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aeb8a-f59b-448e-804c-2bedf8ba6fc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:26.911132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2898/5791] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:11:26.916117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:26.931498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d0bc1-eccc-41ec-baa6-640aa0db010d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:26.948199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:26.948862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:26.949240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:26.949425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] PATH_INFO: `/v3/projects/4e1bd16c4a334102b1ea89a568b9335c/users/e88e1041140441c8ba4cf4c4b35ba601/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:26.954988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1bd16c4a334102b1ea89a568b9335c, user_id=e88e1041140441c8ba4cf4c4b35ba601, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:26.960039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:26.991699 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2894/5792] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:26 2022] PUT /identity/v3/projects/4e1bd16c4a334102b1ea89a568b9335c/users/e88e1041140441c8ba4cf4c4b35ba601/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:27.003267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27576c38-1796-4ed5-957d-2cd8a0600113 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:27.004456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27576c38-1796-4ed5-957d-2cd8a0600113 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:27.006317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27576c38-1796-4ed5-957d-2cd8a0600113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:27.069051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:27.077496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-27576c38-1796-4ed5-957d-2cd8a0600113 None None] MFA Rules not processed for user `e88e1041140441c8ba4cf4c4b35ba601`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:27.078650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-27576c38-1796-4ed5-957d-2cd8a0600113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:27.232821 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2899/5793] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:27.238334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:27.241755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:27.431428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccf3d2-b5af-43da-8776-832b80765f9f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:27.441274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:27.442181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:27.442789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:27.443152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:27.479033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:27.480822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:27.496657 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2895/5794] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:27 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:27.704449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27576c38-1796-4ed5-957d-2cd8a0600113 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:27.817507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27576c38-1796-4ed5-957d-2cd8a0600113 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:27.826509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:27.827742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:27.828045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:27.828291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] PATH_INFO: `/v3/users/6502ed76b6204e4db289c37cc0d181aa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:27.830330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6502ed76b6204e4db289c37cc0d181aa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:27.832689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:27.889423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:27.891634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:27.931783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6502ed76b6204e4db289c37cc0d181aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:27.948285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6502ed76b6204e4db289c37cc0d181aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:27.949214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6502ed76b6204e4db289c37cc0d181aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:27.964632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6502ed76b6204e4db289c37cc0d181aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:27.966272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0403676a-8f7c-4ccc-9bd4-9a5885c5c5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6502ed76b6204e4db289c37cc0d181aa'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:28.005819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f61f15-12ef-4069-a3aa-539500c4e226 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:28.014466 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2900/5795] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:27 2022] DELETE /identity/v3/users/6502ed76b6204e4db289c37cc0d181aa => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:28.014699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:28.014699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:28.014699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:28.014699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:28.017398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:28.019684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:28.020235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:28.020519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:28.030236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:28.036838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:28.038546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:28.058577 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2896/5796] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:28.061984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:28.063014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:28.082751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:28.214030 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2901/5797] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:28.224729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:28.229018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:28.477351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41233f-9615-4085-8953-dadc7d82aa11 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:28.557029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:28.564516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:28.564516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:28.564516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:28.577202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:28.579003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:28.595061 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2897/5798] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:29.125386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:29.128914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:29.179370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30459773-aa51-4a7c-821b-9d0cb676595c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:29.185319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:29.186236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:29.186649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:29.187029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] PATH_INFO: `/v3/projects/fc1ce6381322403599f89ab2965851fb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:29.188445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1ce6381322403599f89ab2965851fb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:29.190720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:29.252448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1ce6381322403599f89ab2965851fb', 'request_id': 'req-65755787-b06e-4365-83ed-861eaf9f3346'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.260793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1ce6381322403599f89ab2965851fb', 'request_id': 'req-65755787-b06e-4365-83ed-861eaf9f3346'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.276389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1ce6381322403599f89ab2965851fb', 'request_id': 'req-65755787-b06e-4365-83ed-861eaf9f3346'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.292278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] The token cache is being invalidate because project fc1ce6381322403599f89ab2965851fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:29.292760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.293716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.296934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2902/5799] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:29 2022] DELETE /identity/v3/projects/fc1ce6381322403599f89ab2965851fb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:29.301723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:29.318489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715afe5-95db-48c0-ab9b-86dd15c11c7c service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:29.453460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:29.454257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:29.454639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:29.455447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] PATH_INFO: `/v3/users/e88e1041140441c8ba4cf4c4b35ba601` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:29.481938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e88e1041140441c8ba4cf4c4b35ba601)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:29.505710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:29.678470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e88e1041140441c8ba4cf4c4b35ba601'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.690547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:29.704433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e88e1041140441c8ba4cf4c4b35ba601'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.721726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e88e1041140441c8ba4cf4c4b35ba601'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.746065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e88e1041140441c8ba4cf4c4b35ba601'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.746804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b665719-9d8a-442f-870e-736b19dc00cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e88e1041140441c8ba4cf4c4b35ba601'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:29.770878 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2898/5800] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:29 2022] DELETE /identity/v3/users/e88e1041140441c8ba4cf4c4b35ba601 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:11:30.695524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:30.697838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:30.721753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65755787-b06e-4365-83ed-861eaf9f3346 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:30.848953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:30.850301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:30.850681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:30.851186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] PATH_INFO: `/v3/projects/4e1bd16c4a334102b1ea89a568b9335c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:30.853736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1bd16c4a334102b1ea89a568b9335c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:30.857893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:30.932703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:30.943133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1bd16c4a334102b1ea89a568b9335c', 'request_id': 'req-da109c5b-1ae8-4776-ba07-aa467fe61d1d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:30.957053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1bd16c4a334102b1ea89a568b9335c', 'request_id': 'req-da109c5b-1ae8-4776-ba07-aa467fe61d1d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:30.981566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1bd16c4a334102b1ea89a568b9335c', 'request_id': 'req-da109c5b-1ae8-4776-ba07-aa467fe61d1d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:31.010322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] The token cache is being invalidate because project 4e1bd16c4a334102b1ea89a568b9335c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:31.010708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:31.011735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-da109c5b-1ae8-4776-ba07-aa467fe61d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:31.014574 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2903/5801] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:30 2022] DELETE /identity/v3/projects/4e1bd16c4a334102b1ea89a568b9335c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:31.019306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86041681-17a3-43cc-920b-c24da5b44cc7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.019952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86041681-17a3-43cc-920b-c24da5b44cc7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.019952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86041681-17a3-43cc-920b-c24da5b44cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.046075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-86041681-17a3-43cc-920b-c24da5b44cc7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:31.047620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-86041681-17a3-43cc-920b-c24da5b44cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:31.186974 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2899/5802] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:31.194778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951b612-db14-466a-8220-eb97eab0b8f8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.195511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951b612-db14-466a-8220-eb97eab0b8f8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.195805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951b612-db14-466a-8220-eb97eab0b8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.235702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5951b612-db14-466a-8220-eb97eab0b8f8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:31.238894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5951b612-db14-466a-8220-eb97eab0b8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:31.255067 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2904/5803] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:31.258904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86041681-17a3-43cc-920b-c24da5b44cc7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:31.278526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86041681-17a3-43cc-920b-c24da5b44cc7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:31.286027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:31.286852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.287136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.287530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.288379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:31.288755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:31.293856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:31.334823 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2900/5804] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:31.338504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951b612-db14-466a-8220-eb97eab0b8f8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:31.356591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951b612-db14-466a-8220-eb97eab0b8f8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:31.367472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:31.368253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.369687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.369994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.370975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:31.376299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:31.448057 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2905/5805] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:31 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:31.452648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:31.467473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829affd-fe99-4107-b37e-af93569542da admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:31.613053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:31.613664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.613975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.614242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.615041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:31.618012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:31.669382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2901/5806] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:31 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:31.674255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:31.696056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262221e-02e8-4da3-8aa9-36c6850d8e25 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:31.708793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:31.709577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.709959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.710453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.711360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:31.714402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:31.729892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2906/5807] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:31.734554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:31.751228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bac1d-dc50-40e7-969e-11c9c173aed2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:31.758066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac02823-6ff2-4fae-b59d-b2c1adccd150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:31.764980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac02823-6ff2-4fae-b59d-b2c1adccd150 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.765461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac02823-6ff2-4fae-b59d-b2c1adccd150 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.765609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac02823-6ff2-4fae-b59d-b2c1adccd150 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.766381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac02823-6ff2-4fae-b59d-b2c1adccd150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:31.768319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac02823-6ff2-4fae-b59d-b2c1adccd150 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:31.784866 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2902/5808] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:31.789029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:31.804943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4439b-1513-4c50-b2c7-04f451d25f58 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:31.813215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:31.814710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.815104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.815523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] PATH_INFO: `/v3/projects/567eac366c5a47daa5b70713b6168cb9/users/ecd325ca509e4a0db3b11a0019b86be3/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.838991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=567eac366c5a47daa5b70713b6168cb9, user_id=ecd325ca509e4a0db3b11a0019b86be3, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:31.841654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:31.869081 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2907/5809] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:31 2022] PUT /identity/v3/projects/567eac366c5a47daa5b70713b6168cb9/users/ecd325ca509e4a0db3b11a0019b86be3/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:31.870722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a093-ed3e-48c6-bd5f-9817a3d40545 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:31.871008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a093-ed3e-48c6-bd5f-9817a3d40545 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:31.871284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13a093-ed3e-48c6-bd5f-9817a3d40545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:31.909580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d13a093-ed3e-48c6-bd5f-9817a3d40545 None None] MFA Rules not processed for user `ecd325ca509e4a0db3b11a0019b86be3`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:31.911111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d13a093-ed3e-48c6-bd5f-9817a3d40545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:31.953732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:32.036342 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2903/5810] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:32.041171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:32.164595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1217e89-3099-43cf-b150-887f4a8b6547 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:32.174128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:32.175165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:32.175784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:32.175784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:32.176474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:32.179052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:32.240360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a093-ed3e-48c6-bd5f-9817a3d40545 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:32.251455 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2908/5811] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:32 2022] POST /identity/v3/projects => generated 389 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:32.257339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13a093-ed3e-48c6-bd5f-9817a3d40545 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:32.387764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:32.389984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:32.390615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:32.391158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:32.393060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:32.394408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:32.434132 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2904/5812] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:32 2022] POST /identity/v3/users => generated 301 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:32.438584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:32.455267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44e704-ae20-413f-87cb-8bddb0b9bae9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:32.462945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:32.463941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:32.464313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:32.464687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:32.465965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:32.469585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:32.488686 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2909/5813] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:32.493887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:32.506933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51741dd1-7713-4273-a04f-8ff04edea5c3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:32.517499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:32.518637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:32.519054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:32.519629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] PATH_INFO: `/v3/projects/6b81437a44944144876ea76e31a3cbad/users/2f7e6c55e1b64502bf6d6964f230dc18/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:32.547130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b81437a44944144876ea76e31a3cbad, user_id=2f7e6c55e1b64502bf6d6964f230dc18, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:32.550874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:32.581218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:32.581819 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2905/5814] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:32 2022] PUT /identity/v3/projects/6b81437a44944144876ea76e31a3cbad/users/2f7e6c55e1b64502bf6d6964f230dc18/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:32.595080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40443942-17d1-479e-bb83-c5aba71c4d00 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:32.660547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:32.713421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417a9fd-cc15-4e95-9f1e-45dcc6e1bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:32.715744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417a9fd-cc15-4e95-9f1e-45dcc6e1bb4f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:32.716845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417a9fd-cc15-4e95-9f1e-45dcc6e1bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:32.717482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417a9fd-cc15-4e95-9f1e-45dcc6e1bb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:32.719610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417a9fd-cc15-4e95-9f1e-45dcc6e1bb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:32.722958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417a9fd-cc15-4e95-9f1e-45dcc6e1bb4f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:32.742395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:32.743748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2910/5815] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:32.759795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304f288-bc22-403a-8573-c5af247860e7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:32.767717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:32.768516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:32.768901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:32.769284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] PATH_INFO: `/v3/projects/6b81437a44944144876ea76e31a3cbad/users/2f7e6c55e1b64502bf6d6964f230dc18/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:32.777641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b81437a44944144876ea76e31a3cbad, user_id=2f7e6c55e1b64502bf6d6964f230dc18, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:32.781677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:32.800387 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2906/5816] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:32 2022] PUT /identity/v3/projects/6b81437a44944144876ea76e31a3cbad/users/2f7e6c55e1b64502bf6d6964f230dc18/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:32.811592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85896fe2-da3e-4d6d-bda4-ab7015291632 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:32.811981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85896fe2-da3e-4d6d-bda4-ab7015291632 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:32.812351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85896fe2-da3e-4d6d-bda4-ab7015291632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:32.853573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-85896fe2-da3e-4d6d-bda4-ab7015291632 None None] MFA Rules not processed for user `2f7e6c55e1b64502bf6d6964f230dc18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:32.854917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-85896fe2-da3e-4d6d-bda4-ab7015291632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:32.967164 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2911/5817] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:32 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:32.979199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:32.983019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:33.102325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd75b7-c74d-4add-9cc5-48334ef1967b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:33.107914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38754fc-6cfc-4412-9bcb-dcb0559498c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:33.108787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38754fc-6cfc-4412-9bcb-dcb0559498c9 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:33.109309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38754fc-6cfc-4412-9bcb-dcb0559498c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:33.109794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38754fc-6cfc-4412-9bcb-dcb0559498c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:33.127091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38754fc-6cfc-4412-9bcb-dcb0559498c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:33.128680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38754fc-6cfc-4412-9bcb-dcb0559498c9 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:33.140320 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2907/5818] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:32 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:33.365074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85896fe2-da3e-4d6d-bda4-ab7015291632 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:33.443466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85896fe2-da3e-4d6d-bda4-ab7015291632 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:33.449379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:33.452634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:33.453201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:33.453714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] PATH_INFO: `/v3/users/ecd325ca509e4a0db3b11a0019b86be3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:33.455425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd325ca509e4a0db3b11a0019b86be3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:33.458090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:33.523858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd325ca509e4a0db3b11a0019b86be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:33.538948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd325ca509e4a0db3b11a0019b86be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:33.539702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd325ca509e4a0db3b11a0019b86be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:33.553618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd325ca509e4a0db3b11a0019b86be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:33.554584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd325ca509e4a0db3b11a0019b86be3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:33.558755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:33.589788 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2912/5819] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:33 2022] DELETE /identity/v3/users/ecd325ca509e4a0db3b11a0019b86be3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:33.594849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:33.595352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:33.595809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:33.625184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:33.626878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:33.671540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:33.768883 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2908/5820] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:33.779287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:33.781245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:33.869944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19480b-9aaa-438a-8783-6af891d6e1b8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:33.876958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:33.877876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:33.878658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:33.879031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:33.894266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:33.895960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:33.910596 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2913/5821] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:11:34.293205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:34.294779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:34.318716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3a18-1cc9-4917-9400-6206153308ef None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:34.331277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:34.334308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:34.336070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:34.337869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] PATH_INFO: `/v3/projects/567eac366c5a47daa5b70713b6168cb9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:34.344390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=567eac366c5a47daa5b70713b6168cb9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:34.349193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:34.436129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '567eac366c5a47daa5b70713b6168cb9', 'request_id': 'req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.448010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '567eac366c5a47daa5b70713b6168cb9', 'request_id': 'req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.458893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '567eac366c5a47daa5b70713b6168cb9', 'request_id': 'req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.483248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] The token cache is being invalidate because project 567eac366c5a47daa5b70713b6168cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:34.483949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.484873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.490488 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2909/5822] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:34 2022] DELETE /identity/v3/projects/567eac366c5a47daa5b70713b6168cb9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:34.491110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:34.505683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a679-d9d2-4c15-84cf-6423c65850ce service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:34.586899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:34.587673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:34.588175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:34.588776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] PATH_INFO: `/v3/users/2f7e6c55e1b64502bf6d6964f230dc18` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:34.590217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7e6c55e1b64502bf6d6964f230dc18)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:34.593140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:34.666150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7e6c55e1b64502bf6d6964f230dc18'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.681728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7e6c55e1b64502bf6d6964f230dc18'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.683061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7e6c55e1b64502bf6d6964f230dc18'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.700261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7e6c55e1b64502bf6d6964f230dc18'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.702651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cf38b27e-c988-4200-b709-b487bdc37fa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7e6c55e1b64502bf6d6964f230dc18'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:34.782249 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2914/5823] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:34 2022] DELETE /identity/v3/users/2f7e6c55e1b64502bf6d6964f230dc18 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:35.332891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:35.334582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:35.363598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736d2d4-9410-4cc8-b079-ee5e2b7a3ce7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:35.462239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:35.463063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:35.463453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:35.463834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] PATH_INFO: `/v3/projects/6b81437a44944144876ea76e31a3cbad` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:35.465698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b81437a44944144876ea76e31a3cbad)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:35.468101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:35.557327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b81437a44944144876ea76e31a3cbad', 'request_id': 'req-9e795643-6478-42d4-a977-cde9ebe8de2e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:35.559444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:35.570932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b81437a44944144876ea76e31a3cbad', 'request_id': 'req-9e795643-6478-42d4-a977-cde9ebe8de2e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:35.579032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b81437a44944144876ea76e31a3cbad', 'request_id': 'req-9e795643-6478-42d4-a977-cde9ebe8de2e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:35.599866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] The token cache is being invalidate because project 6b81437a44944144876ea76e31a3cbad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:35.600370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:35.601180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e795643-6478-42d4-a977-cde9ebe8de2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:35.603570 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2910/5824] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:35 2022] DELETE /identity/v3/projects/6b81437a44944144876ea76e31a3cbad => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:35.609096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990a68-52cc-4123-8e60-e86c23c5a96b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:35.609396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990a68-52cc-4123-8e60-e86c23c5a96b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:35.609673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990a68-52cc-4123-8e60-e86c23c5a96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:35.633738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:35.656933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8c990a68-52cc-4123-8e60-e86c23c5a96b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:35.658522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8c990a68-52cc-4123-8e60-e86c23c5a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:35.751070 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2915/5825] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:35.753398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22c76e-2596-4a41-a776-95779c631b78 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:35.753694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22c76e-2596-4a41-a776-95779c631b78 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:35.754015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22c76e-2596-4a41-a776-95779c631b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:35.785752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2e22c76e-2596-4a41-a776-95779c631b78 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:35.787989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2e22c76e-2596-4a41-a776-95779c631b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:35.798461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2911/5826] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:35.802606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c990a68-52cc-4123-8e60-e86c23c5a96b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:35.822691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c990a68-52cc-4123-8e60-e86c23c5a96b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:35.838081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:35.839051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:35.839507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:35.839966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:35.841843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:35.842359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:35.845162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:35.870939 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2916/5827] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:35.872642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22c76e-2596-4a41-a776-95779c631b78 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:35.894961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22c76e-2596-4a41-a776-95779c631b78 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:35.900723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:35.901632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:35.902192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:35.902748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:35.903457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:35.906339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:35.952087 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2912/5828] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:35 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:11:35.956161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:35.970939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c48b49-830a-4e17-869f-7ec85fd844ce admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.091822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.092661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.092956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.093240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.093924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:36.096114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:36.145909 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2917/5829] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:35 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:36.150013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:36.176239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a29c83-6d85-4361-80a4-caa16e2ed211 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.189951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.194006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.194006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.194006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.195753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:36.200902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:36.220950 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2913/5830] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:36.221551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:36.235162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b8c5d-ada0-4c5d-8cdd-beac199d88f6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.253048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa48fa3-18a4-4556-b908-1a100e6a4571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.253687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa48fa3-18a4-4556-b908-1a100e6a4571 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.256567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa48fa3-18a4-4556-b908-1a100e6a4571 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.256567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa48fa3-18a4-4556-b908-1a100e6a4571 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.256567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa48fa3-18a4-4556-b908-1a100e6a4571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:36.257077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa48fa3-18a4-4556-b908-1a100e6a4571 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:36.284827 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2918/5831] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:11:36.286170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:36.299424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa547b0-d877-48e3-8b47-9479e4c4e158 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.306885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.307790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.308224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.308669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] PATH_INFO: `/v3/projects/4e42d60330404b38b488345d107b36fe/users/e4473d57a24d4889b91c6c805c0b4052/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.342554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e42d60330404b38b488345d107b36fe, user_id=e4473d57a24d4889b91c6c805c0b4052, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:36.347829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:36.369500 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2914/5832] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:36 2022] PUT /identity/v3/projects/4e42d60330404b38b488345d107b36fe/users/e4473d57a24d4889b91c6c805c0b4052/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:36.380285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7efa5-4c33-447e-9e66-d9192ddd9b9d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.380777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7efa5-4c33-447e-9e66-d9192ddd9b9d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.381215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7efa5-4c33-447e-9e66-d9192ddd9b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.462534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-47c7efa5-4c33-447e-9e66-d9192ddd9b9d None None] MFA Rules not processed for user `e4473d57a24d4889b91c6c805c0b4052`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:36.464355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-47c7efa5-4c33-447e-9e66-d9192ddd9b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:36.565226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:36.571272 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2919/5833] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:36.578860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:36.675761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35f0d0-aebf-4b3a-86b0-e95683bd5a38 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.695988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.697569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.698066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.698512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.699159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:36.701290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:36.734440 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2915/5834] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:36 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:36.738533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7efa5-4c33-447e-9e66-d9192ddd9b9d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:36.759053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7efa5-4c33-447e-9e66-d9192ddd9b9d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.854149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.854814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.855114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.855583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.856175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:36.859407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:36.895222 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2920/5835] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:36 2022] POST /identity/v3/users => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:36.899817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:36.916637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997447d5-d99a-47c0-9807-976d7945a38b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.924132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.924918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.926519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.926907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:36.927746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:36.929671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:36.955461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2916/5836] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:36.959312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:36.980404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935156c-292d-48c4-9edf-6ba4186a588e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:36.991924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:36.992940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:36.993524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:36.993853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] PATH_INFO: `/v3/projects/4365556f23124a98809187cab67f3f5c/users/703e3847076949e59d3470ed03fc4103/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:37.016624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4365556f23124a98809187cab67f3f5c, user_id=703e3847076949e59d3470ed03fc4103, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:37.020215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:37.040545 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2921/5837] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:36 2022] PUT /identity/v3/projects/4365556f23124a98809187cab67f3f5c/users/703e3847076949e59d3470ed03fc4103/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:37.046241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:37.064182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8264-fe4d-4ca4-8d8a-16568ace919a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:37.189263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:37.212417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d06c5-467e-410c-81a8-4f4409ac0490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:37.213051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d06c5-467e-410c-81a8-4f4409ac0490 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:37.213329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d06c5-467e-410c-81a8-4f4409ac0490 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:37.213601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d06c5-467e-410c-81a8-4f4409ac0490 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:37.214299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d06c5-467e-410c-81a8-4f4409ac0490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:37.218223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d06c5-467e-410c-81a8-4f4409ac0490 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:37.235883 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2917/5838] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:37.241100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:37.254989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da400e-2a61-4b9b-a7b7-353391501ec3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:37.265700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:37.266438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:37.266729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:37.267009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] PATH_INFO: `/v3/projects/4365556f23124a98809187cab67f3f5c/users/703e3847076949e59d3470ed03fc4103/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:37.269035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4365556f23124a98809187cab67f3f5c, user_id=703e3847076949e59d3470ed03fc4103, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:37.271767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:37.291909 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2922/5839] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:37 2022] PUT /identity/v3/projects/4365556f23124a98809187cab67f3f5c/users/703e3847076949e59d3470ed03fc4103/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:37.304069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc07141-d155-4cc3-af37-c0823b5aa9f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:37.304404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc07141-d155-4cc3-af37-c0823b5aa9f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:37.304717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc07141-d155-4cc3-af37-c0823b5aa9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:37.348615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fbc07141-d155-4cc3-af37-c0823b5aa9f0 None None] MFA Rules not processed for user `703e3847076949e59d3470ed03fc4103`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:37.350336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fbc07141-d155-4cc3-af37-c0823b5aa9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:37.491606 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2918/5840] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:37.503274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:37.505564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:37.616442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc87b8-39fb-48aa-8bfe-f653ed92cf69 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:37.621776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:37.622731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:37.623404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:37.624722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:37.646609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:37.648608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:37.712932 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2923/5841] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:37.843095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc07141-d155-4cc3-af37-c0823b5aa9f0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:37.855362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc07141-d155-4cc3-af37-c0823b5aa9f0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:37.865155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099091d2-8ce8-4d69-b9be-859e0b8ae620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:37.866038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099091d2-8ce8-4d69-b9be-859e0b8ae620 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:37.866430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099091d2-8ce8-4d69-b9be-859e0b8ae620 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:37.866808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099091d2-8ce8-4d69-b9be-859e0b8ae620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:37.959126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099091d2-8ce8-4d69-b9be-859e0b8ae620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:37.960991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099091d2-8ce8-4d69-b9be-859e0b8ae620 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:37.976019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2919/5842] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:38.290337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:38.351605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:38.358718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c497ef-5432-4841-8476-a150d7709852 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:38.364746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:38.365379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:38.365659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:38.365968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] PATH_INFO: `/v3/users/e4473d57a24d4889b91c6c805c0b4052` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:38.367154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4473d57a24d4889b91c6c805c0b4052)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:38.369189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:38.417911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4473d57a24d4889b91c6c805c0b4052'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:38.425361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4473d57a24d4889b91c6c805c0b4052'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:38.426054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4473d57a24d4889b91c6c805c0b4052'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:38.433308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4473d57a24d4889b91c6c805c0b4052'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:38.434126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4473d57a24d4889b91c6c805c0b4052'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:38.452692 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2924/5843] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:38 2022] DELETE /identity/v3/users/e4473d57a24d4889b91c6c805c0b4052 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:38.457075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:38.457532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:38.457649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:38.476259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:38.477453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:38.534784 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2920/5844] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:38.545006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:38.546425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:38.604725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa25f69-5d43-4b11-98ee-975e1245e9d2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:38.608670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:38.609270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:38.609551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:38.609847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:38.618910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:38.620339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:38.630457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2925/5845] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:39.016534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:39.018055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:39.033628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78dcf56-dc6e-4b17-8393-50e85dff4d1e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:39.038686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:39.039470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:39.039826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:39.040101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] PATH_INFO: `/v3/projects/4e42d60330404b38b488345d107b36fe` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:39.041433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e42d60330404b38b488345d107b36fe)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:39.043409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:39.046636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:39.092456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e42d60330404b38b488345d107b36fe', 'request_id': 'req-c5067bc4-4a80-4029-bb49-fef585dc3ac4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.098096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e42d60330404b38b488345d107b36fe', 'request_id': 'req-c5067bc4-4a80-4029-bb49-fef585dc3ac4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.104877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e42d60330404b38b488345d107b36fe', 'request_id': 'req-c5067bc4-4a80-4029-bb49-fef585dc3ac4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.117442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] The token cache is being invalidate because project 4e42d60330404b38b488345d107b36fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:39.117735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.118468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.120110 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2921/5846] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:39 2022] DELETE /identity/v3/projects/4e42d60330404b38b488345d107b36fe => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:39.126784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:39.141337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595e98c-83bd-43b0-a3f9-ffc74f17e963 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:39.212173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:39.212742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:39.213021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:39.213301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] PATH_INFO: `/v3/users/703e3847076949e59d3470ed03fc4103` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:39.214444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=703e3847076949e59d3470ed03fc4103)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:39.216476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:39.243132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:39.272985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '703e3847076949e59d3470ed03fc4103'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.280089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '703e3847076949e59d3470ed03fc4103'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.280795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '703e3847076949e59d3470ed03fc4103'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.288109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '703e3847076949e59d3470ed03fc4103'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.288874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d843d327-5e31-4119-9e87-2b7204e43989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '703e3847076949e59d3470ed03fc4103'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.305956 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2926/5847] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:39 2022] DELETE /identity/v3/users/703e3847076949e59d3470ed03fc4103 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:11:39.578205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:39.579718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:39.593054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5067bc4-4a80-4029-bb49-fef585dc3ac4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:39.657557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:39.658353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:39.658677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:39.659177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] PATH_INFO: `/v3/projects/4365556f23124a98809187cab67f3f5c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:39.660295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4365556f23124a98809187cab67f3f5c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:39.662325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:39.711642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4365556f23124a98809187cab67f3f5c', 'request_id': 'req-41101693-c8a2-4793-a009-b5c709bd230c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.717279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4365556f23124a98809187cab67f3f5c', 'request_id': 'req-41101693-c8a2-4793-a009-b5c709bd230c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.723546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4365556f23124a98809187cab67f3f5c', 'request_id': 'req-41101693-c8a2-4793-a009-b5c709bd230c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.735944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] The token cache is being invalidate because project 4365556f23124a98809187cab67f3f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:39.736234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.736859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-41101693-c8a2-4793-a009-b5c709bd230c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:39.738383 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2922/5848] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:39 2022] DELETE /identity/v3/projects/4365556f23124a98809187cab67f3f5c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:39.743230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317082b4-78ca-47bd-b7d5-257be4e7b4e0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:39.743520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317082b4-78ca-47bd-b7d5-257be4e7b4e0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:39.743795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317082b4-78ca-47bd-b7d5-257be4e7b4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:39.765207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-317082b4-78ca-47bd-b7d5-257be4e7b4e0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:39.766498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-317082b4-78ca-47bd-b7d5-257be4e7b4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:39.824549 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2927/5849] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:39.833602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a8c0b-671b-4dfe-b389-7fa2941c22bd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:39.833934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a8c0b-671b-4dfe-b389-7fa2941c22bd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:39.834221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a8c0b-671b-4dfe-b389-7fa2941c22bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:39.854107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d6a8c0b-671b-4dfe-b389-7fa2941c22bd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:39.855314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d6a8c0b-671b-4dfe-b389-7fa2941c22bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:39.863859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2923/5850] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:39.868047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317082b4-78ca-47bd-b7d5-257be4e7b4e0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:39.878589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317082b4-78ca-47bd-b7d5-257be4e7b4e0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:39.883188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:39.883781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:39.884063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:39.884335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:39.884978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:39.885281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:39.886916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:39.898386 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2928/5851] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:39.902570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a8c0b-671b-4dfe-b389-7fa2941c22bd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:39.912587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a8c0b-671b-4dfe-b389-7fa2941c22bd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:39.917313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:39.917946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:39.918235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:39.918513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:39.919136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:39.921897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:39.936721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:39.949874 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2924/5852] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:39 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:39.954081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:39.964982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d624e-3722-4dbd-9e96-c262f40405eb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:40.042067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:40.042703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:40.042985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:40.043263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:40.043910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:40.046674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:40.078856 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2929/5853] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:39 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:40.080772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:40.093157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d82cf-b05a-42c2-8023-bfc41c4c585f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:40.101437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:40.102360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:40.102791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:40.103350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:40.104029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:40.105681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:40.117059 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2925/5854] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:40.121423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:40.132875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb00b5e-d030-4348-b4aa-61ce987e4f13 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:40.139396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b970b4c-263b-4a35-a647-fa9afec6a544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:40.140030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b970b4c-263b-4a35-a647-fa9afec6a544 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:40.140302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b970b4c-263b-4a35-a647-fa9afec6a544 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:40.140571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b970b4c-263b-4a35-a647-fa9afec6a544 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:40.141233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b970b4c-263b-4a35-a647-fa9afec6a544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:40.143018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b970b4c-263b-4a35-a647-fa9afec6a544 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:40.162391 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2930/5855] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:40.166515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:40.177547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166870c-b37c-43a4-9050-56708dd86ba0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:40.182374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:40.183087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:40.183354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:40.183631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] PATH_INFO: `/v3/projects/5afe4950a255418196bbeadcc979078c/users/3f91cec7330c44cd9b616299a2a5fc72/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:40.201858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afe4950a255418196bbeadcc979078c, user_id=3f91cec7330c44cd9b616299a2a5fc72, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:40.204684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:40.222193 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2926/5856] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:40 2022] PUT /identity/v3/projects/5afe4950a255418196bbeadcc979078c/users/3f91cec7330c44cd9b616299a2a5fc72/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:11:40.232394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7538d-ce5a-42c0-833c-69f4d45308ab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:40.232792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7538d-ce5a-42c0-833c-69f4d45308ab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:40.233139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7538d-ce5a-42c0-833c-69f4d45308ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:40.240981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:40.269909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-31b7538d-ce5a-42c0-833c-69f4d45308ab None None] MFA Rules not processed for user `3f91cec7330c44cd9b616299a2a5fc72`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:40.271526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-31b7538d-ce5a-42c0-833c-69f4d45308ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:40.349410 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2931/5857] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:40.362426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:40.365462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:40.438953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0327682-2c6a-47d0-943a-95eb9e413a26 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:40.444969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:40.445715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:40.446106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:40.446427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:40.456870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:40.458344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:40.467704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2927/5858] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:44.639261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3d839-f2c2-49e5-ab3c-9ebc4b94e947 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:44.639791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3d839-f2c2-49e5-ab3c-9ebc4b94e947 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:44.639900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3d839-f2c2-49e5-ab3c-9ebc4b94e947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:44.660232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8de3d839-f2c2-49e5-ab3c-9ebc4b94e947 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:44.661355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8de3d839-f2c2-49e5-ab3c-9ebc4b94e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:44.720794 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2932/5859] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:44.733808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:44.735303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:44.815212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1bd767-6f32-40ff-b4cf-ee06307d2546 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:44.820190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cbe3f-fdc7-459f-8fb9-93d08788e6b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:44.820822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbe3f-fdc7-459f-8fb9-93d08788e6b1 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:44.821108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbe3f-fdc7-459f-8fb9-93d08788e6b1 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:44.821384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbe3f-fdc7-459f-8fb9-93d08788e6b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:44.830946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cbe3f-fdc7-459f-8fb9-93d08788e6b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:44.832540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cbe3f-fdc7-459f-8fb9-93d08788e6b1 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:44.841318 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2928/5860] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:11:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:11:46.730686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3d839-f2c2-49e5-ab3c-9ebc4b94e947 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:46.825184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3d839-f2c2-49e5-ab3c-9ebc4b94e947 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:46.839611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:46.840397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:46.840769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:46.841288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] PATH_INFO: `/v3/users/3f91cec7330c44cd9b616299a2a5fc72` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:46.842911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f91cec7330c44cd9b616299a2a5fc72)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:46.844963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:46.898314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:46.908144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f91cec7330c44cd9b616299a2a5fc72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:46.915972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f91cec7330c44cd9b616299a2a5fc72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:46.916796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f91cec7330c44cd9b616299a2a5fc72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:46.925076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f91cec7330c44cd9b616299a2a5fc72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:46.925767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f91cec7330c44cd9b616299a2a5fc72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:46.945638 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2933/5861] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:46 2022] DELETE /identity/v3/users/3f91cec7330c44cd9b616299a2a5fc72 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:46.951061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:46.952099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:46.952608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:46.979298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:46.981343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:47.038826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:47.058179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2929/5862] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:47.069069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:47.070679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:47.166250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4fd0a-1d8c-481b-b7f1-ee8b8ba1600c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:47.173067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a56c5-01e0-468f-8ea7-4da5ae478487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:47.173948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a56c5-01e0-468f-8ea7-4da5ae478487 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:47.177598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a56c5-01e0-468f-8ea7-4da5ae478487 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:47.177598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a56c5-01e0-468f-8ea7-4da5ae478487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:47.196077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a56c5-01e0-468f-8ea7-4da5ae478487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:47.197570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a56c5-01e0-468f-8ea7-4da5ae478487 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:47.209526 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2934/5863] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:47.569218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:47.570814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:47.583654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f91c6-3269-48e4-898d-a4da01028f33 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:47.588841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:47.589617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:47.590023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:47.590462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] PATH_INFO: `/v3/projects/5afe4950a255418196bbeadcc979078c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:47.592332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afe4950a255418196bbeadcc979078c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:47.594391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:47.654960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afe4950a255418196bbeadcc979078c', 'request_id': 'req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:47.660976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afe4950a255418196bbeadcc979078c', 'request_id': 'req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:47.668197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afe4950a255418196bbeadcc979078c', 'request_id': 'req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:47.688235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] The token cache is being invalidate because project 5afe4950a255418196bbeadcc979078c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:47.688465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:47.689105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac7ee4fb-16da-4ea1-b982-e1f5aeee6507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:47.690912 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2930/5864] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:47 2022] DELETE /identity/v3/projects/5afe4950a255418196bbeadcc979078c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:47.695699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fc83-0ad9-425f-a9de-fc3eed9184b7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:47.695987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fc83-0ad9-425f-a9de-fc3eed9184b7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:47.696261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fc83-0ad9-425f-a9de-fc3eed9184b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:47.720728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d450fc83-0ad9-425f-a9de-fc3eed9184b7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:47.722962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d450fc83-0ad9-425f-a9de-fc3eed9184b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:47.790078 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2935/5865] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:47.799089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d83f6-f553-4f38-82a6-b60e5c5ab25f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:47.799089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d83f6-f553-4f38-82a6-b60e5c5ab25f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:47.799089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d83f6-f553-4f38-82a6-b60e5c5ab25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:47.820340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6b0d83f6-f553-4f38-82a6-b60e5c5ab25f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:47.820340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b0d83f6-f553-4f38-82a6-b60e5c5ab25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:47.838767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450fc83-0ad9-425f-a9de-fc3eed9184b7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:47.839324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2931/5866] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:47.850753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450fc83-0ad9-425f-a9de-fc3eed9184b7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:47.855811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:47.856418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:47.856777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:47.857108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:47.857766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:11:47.858104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:47.859702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:47.874639 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2936/5867] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:11:47.879278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d83f6-f553-4f38-82a6-b60e5c5ab25f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:47.891837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d83f6-f553-4f38-82a6-b60e5c5ab25f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:47.897081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:47.897841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:47.898204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:47.898544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:47.899248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:47.901400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:47.932682 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2932/5868] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:11:47 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:47.938788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:47.950845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3b29-9924-4b2a-a967-75586a91f9c0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:48.017850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:48.034308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:48.035121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:48.035482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:48.035825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:48.036539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:48.038816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:48.069917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:48.070551 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2937/5869] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:11:47 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:11:48.082466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a24257-9a3e-483d-9ea4-47aa9b5aa507 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:48.088087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:48.088932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:48.089212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:48.089484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:48.090209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:48.092805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:48.104882 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2933/5870] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:48.109089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:48.119869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdfe8f-e719-4558-a93c-23be0e5990ce admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:48.129500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca39f65-aa11-49ad-a3a9-b9998729ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:48.130172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca39f65-aa11-49ad-a3a9-b9998729ec2a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:48.130455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca39f65-aa11-49ad-a3a9-b9998729ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:48.130879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca39f65-aa11-49ad-a3a9-b9998729ec2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:48.131523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca39f65-aa11-49ad-a3a9-b9998729ec2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:48.133120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca39f65-aa11-49ad-a3a9-b9998729ec2a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:48.152500 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2938/5871] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:11:48.154519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:48.168799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42cc6-66c3-4b78-946e-dc1f6af2bca6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:48.178165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:48.179147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:48.179622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:48.179896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] PATH_INFO: `/v3/projects/23ac56e87f6242a39be3dbd6cca65cbe/users/be02cb70cbfb4be49a59344e95f3cbea/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:48.201502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ac56e87f6242a39be3dbd6cca65cbe, user_id=be02cb70cbfb4be49a59344e95f3cbea, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:48.204200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:48.222469 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2934/5872] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:11:48 2022] PUT /identity/v3/projects/23ac56e87f6242a39be3dbd6cca65cbe/users/be02cb70cbfb4be49a59344e95f3cbea/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:48.232998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28078dbe-6c19-487a-911b-2720d46d2e48 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:48.233295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28078dbe-6c19-487a-911b-2720d46d2e48 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:48.234547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28078dbe-6c19-487a-911b-2720d46d2e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:48.264913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-28078dbe-6c19-487a-911b-2720d46d2e48 None None] MFA Rules not processed for user `be02cb70cbfb4be49a59344e95f3cbea`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:48.266176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-28078dbe-6c19-487a-911b-2720d46d2e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:48.353892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2939/5873] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:11:48.373135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:48.375916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:48.446790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d5e7-2657-47b3-b62f-9945eaa5f071 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:48.454416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f7a5b-5eb4-4c5d-9f12-9fa4a9b8922d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:48.455476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f7a5b-5eb4-4c5d-9f12-9fa4a9b8922d service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:48.455476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f7a5b-5eb4-4c5d-9f12-9fa4a9b8922d service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:48.455476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f7a5b-5eb4-4c5d-9f12-9fa4a9b8922d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:48.465602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f7a5b-5eb4-4c5d-9f12-9fa4a9b8922d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:48.467787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f7a5b-5eb4-4c5d-9f12-9fa4a9b8922d service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:48.482536 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2935/5874] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:58.348017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:58.477645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:59.073537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28078dbe-6c19-487a-911b-2720d46d2e48 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:59.137063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28078dbe-6c19-487a-911b-2720d46d2e48 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:59.142391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:59.143014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:59.143296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:59.143573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] PATH_INFO: `/v3/users/be02cb70cbfb4be49a59344e95f3cbea` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:59.144705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] RBAC: Authorizing `identity:delete_user(user_id=be02cb70cbfb4be49a59344e95f3cbea)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:59.146663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:59.193714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be02cb70cbfb4be49a59344e95f3cbea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.201420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be02cb70cbfb4be49a59344e95f3cbea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.202132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be02cb70cbfb4be49a59344e95f3cbea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.209362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be02cb70cbfb4be49a59344e95f3cbea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.210062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be02cb70cbfb4be49a59344e95f3cbea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.228606 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2940/5875] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:11:59 2022] DELETE /identity/v3/users/be02cb70cbfb4be49a59344e95f3cbea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:59.233394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:59.233682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:59.233991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:59.252935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:59.254172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:59.313952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2936/5876] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:59.324342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:59.326329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:59.383732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecb395-dadb-4c9b-ac83-1975fd40028e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:59.387445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce66ed-bb51-4c49-8710-f11956053dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:59.388045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce66ed-bb51-4c49-8710-f11956053dfc service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:59.388306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce66ed-bb51-4c49-8710-f11956053dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:59.388575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce66ed-bb51-4c49-8710-f11956053dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:59.397466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce66ed-bb51-4c49-8710-f11956053dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:59.398917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce66ed-bb51-4c49-8710-f11956053dfc service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:59.408888 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2941/5877] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:11:59.694482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:11:59.696062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:59.712379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6d478-af32-49c9-a2d4-baeb519508f4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:11:59.719405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:11:59.720064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:59.720346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:59.720623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] PATH_INFO: `/v3/projects/23ac56e87f6242a39be3dbd6cca65cbe` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:59.721738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ac56e87f6242a39be3dbd6cca65cbe)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:11:59.723726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:11:59.772836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ac56e87f6242a39be3dbd6cca65cbe', 'request_id': 'req-0518e6ab-f15c-4243-a3a9-1bb863ceb074'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.780120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ac56e87f6242a39be3dbd6cca65cbe', 'request_id': 'req-0518e6ab-f15c-4243-a3a9-1bb863ceb074'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.787326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ac56e87f6242a39be3dbd6cca65cbe', 'request_id': 'req-0518e6ab-f15c-4243-a3a9-1bb863ceb074'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.800529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] The token cache is being invalidate because project 23ac56e87f6242a39be3dbd6cca65cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:11:59.800807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.801442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0518e6ab-f15c-4243-a3a9-1bb863ceb074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:11:59.802991 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2937/5878] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:11:59 2022] DELETE /identity/v3/projects/23ac56e87f6242a39be3dbd6cca65cbe => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:11:59.808144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324084-dee9-4f60-9212-5c41bd772b25 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:59.808434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324084-dee9-4f60-9212-5c41bd772b25 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:59.808705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324084-dee9-4f60-9212-5c41bd772b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:59.829003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e1324084-dee9-4f60-9212-5c41bd772b25 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:59.830824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e1324084-dee9-4f60-9212-5c41bd772b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:59.850436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:11:59.941964 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2942/5879] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:59.946503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b30443-ae7a-4b5f-99f3-672b06203598 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:11:59.946918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b30443-ae7a-4b5f-99f3-672b06203598 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:11:59.947286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b30443-ae7a-4b5f-99f3-672b06203598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:11:59.970721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-72b30443-ae7a-4b5f-99f3-672b06203598 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:11:59.972162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-72b30443-ae7a-4b5f-99f3-672b06203598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:11:59.985388 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2938/5880] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:11:59.986314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1324084-dee9-4f60-9212-5c41bd772b25 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.003024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1324084-dee9-4f60-9212-5c41bd772b25 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.007730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.008334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.008617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.008894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.009542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:12:00.009869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.011481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.024525 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2943/5881] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:12:00.029413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b30443-ae7a-4b5f-99f3-672b06203598 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.039671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b30443-ae7a-4b5f-99f3-672b06203598 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.045075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.045687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.046027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.046311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.046954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.048865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.082698 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2939/5882] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:12:00 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:12:00.083854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.095658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a4183-cc9b-46ef-9a71-b27a280f19d4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.167565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ccba6-e18d-4996-81d6-db3aa30f5f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.168342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ccba6-e18d-4996-81d6-db3aa30f5f19 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.168659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ccba6-e18d-4996-81d6-db3aa30f5f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.169450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ccba6-e18d-4996-81d6-db3aa30f5f19 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.170162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ccba6-e18d-4996-81d6-db3aa30f5f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.172109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ccba6-e18d-4996-81d6-db3aa30f5f19 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.196917 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2944/5883] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:12:00 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:12:00.201220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.212717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c244-0a8c-4cb1-9f32-24255d44919b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.218302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.218969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.219326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.219641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.220423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.221908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.236702 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2940/5884] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:12:00.236922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.236922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.236922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.238074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.251390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94dd65-9276-4767-8e5c-14a9e7cad453 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.256760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.257374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.257659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.257976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.258612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.260414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:00.260567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.261292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:00.271485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2941/5885] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:12:00.275565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.288811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e15cd-2aad-435c-86b9-a9e8712721cd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.294408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fdf87-260b-4a7b-b2bf-70877a2a678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.294648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fdf87-260b-4a7b-b2bf-70877a2a678e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.295002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fdf87-260b-4a7b-b2bf-70877a2a678e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.295283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fdf87-260b-4a7b-b2bf-70877a2a678e admin admin] PATH_INFO: `/v3/projects/4fefb25df4ba46079af639c918a5daac/users/45db92d38fff41c48144815864a90297/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.311140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fdf87-260b-4a7b-b2bf-70877a2a678e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fefb25df4ba46079af639c918a5daac, user_id=45db92d38fff41c48144815864a90297, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.313799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fdf87-260b-4a7b-b2bf-70877a2a678e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.328322 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2942/5886] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:12:00 2022] PUT /identity/v3/projects/4fefb25df4ba46079af639c918a5daac/users/45db92d38fff41c48144815864a90297/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:12:00.333305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.333594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.333922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.336010 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2945/5887] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:12:00.348868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.350676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:00.355076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:00.356263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:00.360575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:00.401817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:00.419850 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2943/5888] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:00.429631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1028a-79c7-40ae-9af8-c5a7d8f445ec None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.430338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:00.431752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:00.491622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e633654-b12c-47f1-ad1b-8cb631cfb5dc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:00.494737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.495372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.495493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32103f-0a9f-4b95-8a24-712522064f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:00.495651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.495920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.496040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32103f-0a9f-4b95-8a24-712522064f6e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:00.496386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32103f-0a9f-4b95-8a24-712522064f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:00.496673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32103f-0a9f-4b95-8a24-712522064f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:00.506660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32103f-0a9f-4b95-8a24-712522064f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.508213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32103f-0a9f-4b95-8a24-712522064f6e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.518733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2944/5889] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:12:00.559678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:00.561386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:00.570604 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2946/5890] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:12:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 21:12:06.294899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444060-004e-4ab7-bb30-dfbbf53e4be1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:06.295538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444060-004e-4ab7-bb30-dfbbf53e4be1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:06.295538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33444060-004e-4ab7-bb30-dfbbf53e4be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:06.332736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-33444060-004e-4ab7-bb30-dfbbf53e4be1 None None] MFA Rules not processed for user `45db92d38fff41c48144815864a90297`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:06.334203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-33444060-004e-4ab7-bb30-dfbbf53e4be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:06.437395 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2945/5891] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:06.441596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:06.454564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca452c4-a78a-44a7-ad29-e050e7693929 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:06.471608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:06.472183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:06.472467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:06.472743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:06.473394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:06.475336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:06.513956 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2947/5892] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:12:06 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:12:06.515432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444060-004e-4ab7-bb30-dfbbf53e4be1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:06.534385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33444060-004e-4ab7-bb30-dfbbf53e4be1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:06.600161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:06.634114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:06.634826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:06.635116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:06.635418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:06.636032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:06.639409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:06.677008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2946/5893] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:12:06 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:12:06.681115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:06.696640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72794142-e563-437a-ab80-7f614dc80f22 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:06.704699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:06.705632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:06.706065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:06.706439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:06.707257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:06.709013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:06.726521 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2948/5894] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:12:06.727870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:06.744577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6027fe-341d-4204-af20-d07c9c1a03f8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:06.752345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:06.753178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:06.753575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:06.753989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] PATH_INFO: `/v3/projects/7e758fe68e744654bf1c07d7f52d3357/users/5b7f8f4e31e9457d860e9afe4583803c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:06.776246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e758fe68e744654bf1c07d7f52d3357, user_id=5b7f8f4e31e9457d860e9afe4583803c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:06.779689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:06.801753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2947/5895] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:12:06 2022] PUT /identity/v3/projects/7e758fe68e744654bf1c07d7f52d3357/users/5b7f8f4e31e9457d860e9afe4583803c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:06.805448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:06.820260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee02e8-b851-4ac1-9050-b194a5539017 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:06.994006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:07.006613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f4358-dac1-430e-a5ea-1f1519e6ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:07.007572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4358-dac1-430e-a5ea-1f1519e6ce59 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:07.008633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4358-dac1-430e-a5ea-1f1519e6ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:07.009146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f4358-dac1-430e-a5ea-1f1519e6ce59 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:07.009932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f4358-dac1-430e-a5ea-1f1519e6ce59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:07.011686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f4358-dac1-430e-a5ea-1f1519e6ce59 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:07.030146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2949/5896] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:12:07.031442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:07.042924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61def0e8-1b6a-43a1-9b32-3dbf345ba55d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:07.049010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:07.049856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:07.050238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:07.050614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] PATH_INFO: `/v3/projects/7e758fe68e744654bf1c07d7f52d3357/users/5b7f8f4e31e9457d860e9afe4583803c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:07.052672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e758fe68e744654bf1c07d7f52d3357, user_id=5b7f8f4e31e9457d860e9afe4583803c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:07.057024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:07.073519 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2948/5897] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:12:07 2022] PUT /identity/v3/projects/7e758fe68e744654bf1c07d7f52d3357/users/5b7f8f4e31e9457d860e9afe4583803c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:14.424105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:14.424616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:14.424743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:14.474030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] MFA Rules not processed for user `5b7f8f4e31e9457d860e9afe4583803c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:14.475331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:14.595486 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2950/5898] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:14.602871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:14.605535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:14.711316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266750a-ac35-461f-b614-c16b660a266a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:14.719075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a3efa-01e3-4835-a3c1-e065c85fb539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:14.719854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a3efa-01e3-4835-a3c1-e065c85fb539 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:14.720201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a3efa-01e3-4835-a3c1-e065c85fb539 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:14.720498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a3efa-01e3-4835-a3c1-e065c85fb539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:14.735605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a3efa-01e3-4835-a3c1-e065c85fb539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:14.737243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a3efa-01e3-4835-a3c1-e065c85fb539 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:14.749986 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2949/5899] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:12:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:12:15.042295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:15.044822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:15.145532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272277d-ec6a-47b2-9fb3-45ca865895e3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:15.150896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:15.151734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:15.152175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:15.152615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:15.155788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:15.171363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:15.172978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:15.188958 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2951/5900] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:12:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:12:17.091153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f3a3a-b3ba-4bd3-b121-e5dc74d1aca4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:17.092503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f3a3a-b3ba-4bd3-b121-e5dc74d1aca4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:17.093152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f3a3a-b3ba-4bd3-b121-e5dc74d1aca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:17.119660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-022f3a3a-b3ba-4bd3-b121-e5dc74d1aca4 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:17.120900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-022f3a3a-b3ba-4bd3-b121-e5dc74d1aca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:17.217123 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2950/5901] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:12:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:17.224907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:17.333488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ff023-43cb-4a51-aca7-e262fb57e589 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:17.342996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:17.343927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:17.344686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:17.345248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:17.359297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:17.363178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:17.365155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:17.380013 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2952/5902] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:12:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:12:18.804507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b879-ba4a-49c6-bc88-ed310e370446 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:18.805053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b879-ba4a-49c6-bc88-ed310e370446 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:18.805174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3b879-ba4a-49c6-bc88-ed310e370446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:18.833960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-87f3b879-ba4a-49c6-bc88-ed310e370446 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:18.835933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87f3b879-ba4a-49c6-bc88-ed310e370446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:18.856372 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2951/5903] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:12:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:18.869738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:18.886218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12332818-0951-4dd4-9abf-c2db9f4faf44 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:18.895441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:18.896837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:18.907693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:18.908338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:18.923773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:18.926378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:18.959734 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2953/5904] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:12:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:12:19.513716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2e4e-7121-4c29-be23-b525488fd9cd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:19.514255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2e4e-7121-4c29-be23-b525488fd9cd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:19.514697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2e4e-7121-4c29-be23-b525488fd9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:19.553775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6adc2e4e-7121-4c29-be23-b525488fd9cd None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:19.565748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6adc2e4e-7121-4c29-be23-b525488fd9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:19.599865 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2952/5905] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:19.617879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:19.633819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc546a-c2cc-46df-8307-2b90657f31e3 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:19.648665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:19.649269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:19.649747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:19.650264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:19.673794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:19.675630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:19.688629 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2954/5906] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:12:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:12:20.534267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc2e4e-7121-4c29-be23-b525488fd9cd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:20.551892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc2e4e-7121-4c29-be23-b525488fd9cd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:20.557785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770965c-855c-4fd7-bcd6-32fa8b9d920f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:20.558581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770965c-855c-4fd7-bcd6-32fa8b9d920f service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:20.559476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770965c-855c-4fd7-bcd6-32fa8b9d920f service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:20.559786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770965c-855c-4fd7-bcd6-32fa8b9d920f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:20.561954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5770965c-855c-4fd7-bcd6-32fa8b9d920f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:20.576491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770965c-855c-4fd7-bcd6-32fa8b9d920f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:20.578126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770965c-855c-4fd7-bcd6-32fa8b9d920f service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:20.582938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:20.602200 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2953/5907] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:12:26.094607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:26.195992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d126163-d9b5-49b6-8e99-fdf350bf210d service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:26.201055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:26.201696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:26.202029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:26.202337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] PATH_INFO: `/v3/users/45db92d38fff41c48144815864a90297` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:26.203470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] RBAC: Authorizing `identity:delete_user(user_id=45db92d38fff41c48144815864a90297)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:26.205565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:26.263142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45db92d38fff41c48144815864a90297'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.271428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45db92d38fff41c48144815864a90297'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.272228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45db92d38fff41c48144815864a90297'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.280977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45db92d38fff41c48144815864a90297'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.281794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45db92d38fff41c48144815864a90297'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.312045 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2955/5908] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:12:26 2022] DELETE /identity/v3/users/45db92d38fff41c48144815864a90297 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:26.384777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5457-56a6-4637-b442-aacbe74f57f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:26.385638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5457-56a6-4637-b442-aacbe74f57f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:26.386128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5457-56a6-4637-b442-aacbe74f57f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:26.424448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9eff5457-56a6-4637-b442-aacbe74f57f0 None None] MFA Rules not processed for user `d83b6ceb599746b394e379d52683b70c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:26.425844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9eff5457-56a6-4637-b442-aacbe74f57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:26.489977 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2954/5909] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:26.515383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:26.517221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:26.590811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:26.662939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfb16a-7a50-4b91-8208-545a60dd397c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:26.668768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:26.669702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:26.670032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:26.670316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] PATH_INFO: `/v3/projects/4fefb25df4ba46079af639c918a5daac` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:26.671745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fefb25df4ba46079af639c918a5daac)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:26.673692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:26.732167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fefb25df4ba46079af639c918a5daac', 'request_id': 'req-c4f2aeb0-74f7-4868-ac2c-faec9b700860'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.740774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fefb25df4ba46079af639c918a5daac', 'request_id': 'req-c4f2aeb0-74f7-4868-ac2c-faec9b700860'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.750902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fefb25df4ba46079af639c918a5daac', 'request_id': 'req-c4f2aeb0-74f7-4868-ac2c-faec9b700860'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.765522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] The token cache is being invalidate because project 4fefb25df4ba46079af639c918a5daac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:12:26.766071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.766747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:26.773974 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2956/5910] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:12:26 2022] DELETE /identity/v3/projects/4fefb25df4ba46079af639c918a5daac => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:26.775561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff5457-56a6-4637-b442-aacbe74f57f0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:26.843279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff5457-56a6-4637-b442-aacbe74f57f0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:26.951574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:26.953067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:26.953654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:26.954074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] PATH_INFO: `/v3/users/5b7f8f4e31e9457d860e9afe4583803c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:26.968790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7f8f4e31e9457d860e9afe4583803c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:26.973029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:27.075803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7f8f4e31e9457d860e9afe4583803c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.102775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7f8f4e31e9457d860e9afe4583803c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.117514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7f8f4e31e9457d860e9afe4583803c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.122825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:27.132795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7f8f4e31e9457d860e9afe4583803c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.133968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-da055ba5-2b2b-465c-b00c-99f92fb53ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7f8f4e31e9457d860e9afe4583803c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.145995 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2955/5911] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:12:26 2022] DELETE /identity/v3/users/5b7f8f4e31e9457d860e9afe4583803c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:27.483857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:27.486083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:27.509092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2aeb0-74f7-4868-ac2c-faec9b700860 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:27.615939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:27.616943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:27.617520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:27.618363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] PATH_INFO: `/v3/projects/7e758fe68e744654bf1c07d7f52d3357` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:27.619343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e758fe68e744654bf1c07d7f52d3357)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:27.622043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:27.671099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:27.685353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e758fe68e744654bf1c07d7f52d3357', 'request_id': 'req-27b13b8d-d372-4c22-be75-70de25eb1fab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.694311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e758fe68e744654bf1c07d7f52d3357', 'request_id': 'req-27b13b8d-d372-4c22-be75-70de25eb1fab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.707645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e758fe68e744654bf1c07d7f52d3357', 'request_id': 'req-27b13b8d-d372-4c22-be75-70de25eb1fab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.721890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] The token cache is being invalidate because project 7e758fe68e744654bf1c07d7f52d3357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:12:27.722192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.722854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-27b13b8d-d372-4c22-be75-70de25eb1fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:12:27.725007 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2957/5912] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:12:27 2022] DELETE /identity/v3/projects/7e758fe68e744654bf1c07d7f52d3357 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:27.729960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0a06c-796d-40e0-bf61-f08753553a66 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:27.730450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0a06c-796d-40e0-bf61-f08753553a66 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:27.731086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0a06c-796d-40e0-bf61-f08753553a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:27.784760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-78e0a06c-796d-40e0-bf61-f08753553a66 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:27.786278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-78e0a06c-796d-40e0-bf61-f08753553a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:27.881717 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2956/5913] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:12:27.883468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d8ff5-27b4-4d87-a17f-a47f54a3fdc5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:27.883797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d8ff5-27b4-4d87-a17f-a47f54a3fdc5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:27.884077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d8ff5-27b4-4d87-a17f-a47f54a3fdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:27.910254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d28d8ff5-27b4-4d87-a17f-a47f54a3fdc5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:27.911497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d28d8ff5-27b4-4d87-a17f-a47f54a3fdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:27.920040 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2958/5914] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:27.924362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0a06c-796d-40e0-bf61-f08753553a66 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:27.936055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0a06c-796d-40e0-bf61-f08753553a66 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:27.941124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:27.941850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:27.942487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:27.942860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:27.943754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:12:27.944076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:27.945687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:27.959613 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2957/5915] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:12:27.963977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d8ff5-27b4-4d87-a17f-a47f54a3fdc5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:27.974185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d8ff5-27b4-4d87-a17f-a47f54a3fdc5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:27.984917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:27.985547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:27.985863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:27.986262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:27.986894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:28.001382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:28.034084 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2959/5916] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:12:27 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:12:28.038372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:28.049186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ceffb-6b5d-4044-9aa6-876f8957e47b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:28.125140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:28.125871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:28.126252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:28.126596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:28.127315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:28.129343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:28.157991 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2958/5917] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:12:28 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:12:28.162127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:28.177747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df519654-28f5-4ade-8923-f25671833f74 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:28.183887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:28.185086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:28.185525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:28.185937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:28.186997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:28.188768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:28.201221 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2960/5918] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:12:28.205459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:28.217668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb548f-dcb9-47da-9070-9c896a7e445f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:28.223778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595d0fc-df53-4f8e-a195-9b3d68bb8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:28.224031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595d0fc-df53-4f8e-a195-9b3d68bb8fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:28.224388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595d0fc-df53-4f8e-a195-9b3d68bb8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:28.224734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595d0fc-df53-4f8e-a195-9b3d68bb8fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:28.225479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595d0fc-df53-4f8e-a195-9b3d68bb8fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:28.227360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595d0fc-df53-4f8e-a195-9b3d68bb8fe8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:28.241652 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2959/5919] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:12:28.245586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:28.262370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1b2d2-3649-443a-9486-3cb6ca3d3523 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:28.279591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:28.280278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:28.280557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:28.280841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] PATH_INFO: `/v3/projects/84a8662ae28642d1bd1f404c391ff23e/users/5346063245704d50b918a54c29bc297a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:28.301671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a8662ae28642d1bd1f404c391ff23e, user_id=5346063245704d50b918a54c29bc297a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:28.304727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:28.321855 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2961/5920] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:12:28 2022] PUT /identity/v3/projects/84a8662ae28642d1bd1f404c391ff23e/users/5346063245704d50b918a54c29bc297a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:28.328023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae2215-5f86-4948-9b23-a955366c9585 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:28.328404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae2215-5f86-4948-9b23-a955366c9585 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:28.328802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae2215-5f86-4948-9b23-a955366c9585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:28.352892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b8ae2215-5f86-4948-9b23-a955366c9585 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:28.354622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b8ae2215-5f86-4948-9b23-a955366c9585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:28.422948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:28.450616 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2960/5921] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:28.461659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:28.463936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:28.613742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b3bb9-e557-4a69-bd62-5b2e60063295 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:28.619859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:28.620847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:28.621173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:28.621486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:28.654671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:28.656128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:28.667957 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2962/5922] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:12:34.292760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76231cbb-9c4a-4dbe-9dc8-75aeffe296f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:34.298211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76231cbb-9c4a-4dbe-9dc8-75aeffe296f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:34.298527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76231cbb-9c4a-4dbe-9dc8-75aeffe296f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:34.361596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-76231cbb-9c4a-4dbe-9dc8-75aeffe296f0 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:34.363388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-76231cbb-9c4a-4dbe-9dc8-75aeffe296f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:34.486577 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2961/5923] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:12:34.534976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:34.536719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:34.666028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:34.735173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28861fca-558b-4d29-8beb-25041157bdea service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:34.746217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:34.746934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:34.747602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:34.748140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:34.760355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:34.762995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:34.774303 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2963/5924] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:12:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:12:36.206058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d945f5-a3f2-4574-b87e-4c13a4e1e153 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:36.206631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d945f5-a3f2-4574-b87e-4c13a4e1e153 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:36.206631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d945f5-a3f2-4574-b87e-4c13a4e1e153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:36.309556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-68d945f5-a3f2-4574-b87e-4c13a4e1e153 None None] MFA Rules not processed for user `5346063245704d50b918a54c29bc297a`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:36.318325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-68d945f5-a3f2-4574-b87e-4c13a4e1e153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:36.522638 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2962/5925] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:36.528147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:36.544620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d913e5-75b4-43d3-aacc-f91302af3655 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:36.560620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:36.561257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:36.561539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:36.561813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:36.564553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:36.566521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:36.641465 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2964/5926] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:12:36 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:12:36.645797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d945f5-a3f2-4574-b87e-4c13a4e1e153 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:36.661515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d945f5-a3f2-4574-b87e-4c13a4e1e153 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:36.810780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:36.811305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:36.811595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:36.811863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:36.812636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:36.814836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:36.845322 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2963/5927] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:12:36 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:12:36.850739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:36.891931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba200-08fa-4cca-ba5f-0f7e1a642a27 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:36.905891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:36.906790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:36.907071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:36.907343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:36.908298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:36.910106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:36.934664 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2965/5928] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:12:36.938931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:36.954827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f442f4a-97a7-4caa-aea5-a27fd00dcdf3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:37.011342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:37.013498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:37.014057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:37.014580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] PATH_INFO: `/v3/projects/284406fb5bca447e99e96da850338927/users/a52844a1f8e142849762f7394dd0010a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:37.031942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:37.043958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284406fb5bca447e99e96da850338927, user_id=a52844a1f8e142849762f7394dd0010a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:37.047198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:37.083047 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2964/5929] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:12:36 2022] PUT /identity/v3/projects/284406fb5bca447e99e96da850338927/users/a52844a1f8e142849762f7394dd0010a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:37.083986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:37.100058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ff55c-d4a9-4dcb-8c62-6ceff5087174 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:37.221063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf939813-3d7c-42de-8247-3f8e27778c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:37.222313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939813-3d7c-42de-8247-3f8e27778c0a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:37.222823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939813-3d7c-42de-8247-3f8e27778c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:37.223557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf939813-3d7c-42de-8247-3f8e27778c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:37.224422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf939813-3d7c-42de-8247-3f8e27778c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:37.231656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf939813-3d7c-42de-8247-3f8e27778c0a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:37.257686 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2966/5930] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:12:37.262712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:37.276186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee178f-c767-41b0-aa20-33485a0c07c8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:37.285674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:37.288045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:37.288578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:37.289663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] PATH_INFO: `/v3/projects/284406fb5bca447e99e96da850338927/users/a52844a1f8e142849762f7394dd0010a/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:37.295743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284406fb5bca447e99e96da850338927, user_id=a52844a1f8e142849762f7394dd0010a, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:37.299504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:37.315809 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2965/5931] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:12:37 2022] PUT /identity/v3/projects/284406fb5bca447e99e96da850338927/users/a52844a1f8e142849762f7394dd0010a/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:12:47.250624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=58,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:47.308745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=10,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:52.597424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb260bd-8798-45ec-8b24-c20407bf7aea None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:52.599088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb260bd-8798-45ec-8b24-c20407bf7aea None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:52.599603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb260bd-8798-45ec-8b24-c20407bf7aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:52.676879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6fb260bd-8798-45ec-8b24-c20407bf7aea None None] MFA Rules not processed for user `a52844a1f8e142849762f7394dd0010a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:52.678145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6fb260bd-8798-45ec-8b24-c20407bf7aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:52.897772 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2967/5932] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:52.953374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:52.955485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:53.235297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac068e0-c743-407c-a417-9723271cbec4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:53.249763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273cf37-7aaa-4707-9817-c2b773530f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:53.252492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273cf37-7aaa-4707-9817-c2b773530f4e service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:53.253035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273cf37-7aaa-4707-9817-c2b773530f4e service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:53.253680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273cf37-7aaa-4707-9817-c2b773530f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:53.282237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273cf37-7aaa-4707-9817-c2b773530f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:53.284045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273cf37-7aaa-4707-9817-c2b773530f4e service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:53.313442 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2966/5933] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:12:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:12:53.406508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:53.407245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:53.407702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:53.466885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] MFA Rules not processed for user `5346063245704d50b918a54c29bc297a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:53.469466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:53.684887 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2968/5934] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:54.062766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:54.063906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:54.065060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:54.102565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] MFA Rules not processed for user `54dcc78ee39c4119b08023d3cc692d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:12:54.105607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:54.233188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2967/5935] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:12:55.441681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:12:55.444199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:55.614800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644bc4f-42e7-4d53-bdbe-64b41affdd83 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:12:55.633545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:12:55.634524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:12:55.635014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:12:55.635514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:12:55.640136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:12:55.702481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:12:55.738719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:12:55.741460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:12:55.766805 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2969/5936] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:02.076244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:02.078730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:02.186828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34de9-3bc7-452a-b6cb-3315653641bc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:02.190755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:02.191285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:02.191754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:02.192191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:02.194310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:02.213785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:02.216892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:02.249172 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2968/5937] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:02.303238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:02.305541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:02.334799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31544e-e671-4155-9a10-06b867acb96b service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:02.341135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:02.342924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:02.344019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:02.344545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] PATH_INFO: `/v3/users/13e5077421c44eb7abcda40582c7ea9a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:02.346769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e5077421c44eb7abcda40582c7ea9a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:02.350337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:02.363347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:02.404851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57c0bc-d368-4fca-8a4f-cbd4f970dc64 service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:02.409209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:02.410611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:02.412857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:02.414130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:02.416267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:02.418985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e5077421c44eb7abcda40582c7ea9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.428970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e5077421c44eb7abcda40582c7ea9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.429696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e5077421c44eb7abcda40582c7ea9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.432093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:02.434428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:02.438870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e5077421c44eb7abcda40582c7ea9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.440193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e5077421c44eb7abcda40582c7ea9a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.446778 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2969/5938] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:13:02.466807 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2970/5939] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:02 2022] DELETE /identity/v3/users/13e5077421c44eb7abcda40582c7ea9a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:02.688589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:02.697093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:02.705263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26e1b-be9c-4579-b9c1-85dfc7577466 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:02.734264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:02.735881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:02.802725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:02.803395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:02.803679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:02.803957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] PATH_INFO: `/v3/projects/9b196d42c2d6416c81818922af07b8a3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:02.805114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b196d42c2d6416c81818922af07b8a3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:02.807055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:02.818046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb14e6-a5c1-4018-b23f-40cb454c8a30 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:02.822053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03f01c-773a-4afb-ac66-b70139287793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:02.822656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f01c-773a-4afb-ac66-b70139287793 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:02.822937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f01c-773a-4afb-ac66-b70139287793 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:02.823207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f01c-773a-4afb-ac66-b70139287793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:02.825756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9e03f01c-773a-4afb-ac66-b70139287793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:02.859689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b196d42c2d6416c81818922af07b8a3', 'request_id': 'req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.865369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b196d42c2d6416c81818922af07b8a3', 'request_id': 'req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.871691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b196d42c2d6416c81818922af07b8a3', 'request_id': 'req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.884809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:02.885051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] The token cache is being invalidate because project 9b196d42c2d6416c81818922af07b8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:02.885167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.886322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-271c2bdf-4b8e-4b52-93f0-50ecba8dfc6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:02.887943 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2970/5940] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:02 2022] DELETE /identity/v3/projects/9b196d42c2d6416c81818922af07b8a3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:02.893438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f115d0e-520a-43e0-8e24-93ef49d5ce10 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:02.893798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f115d0e-520a-43e0-8e24-93ef49d5ce10 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:02.894110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f115d0e-520a-43e0-8e24-93ef49d5ce10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:02.912052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03f01c-773a-4afb-ac66-b70139287793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:02.913506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03f01c-773a-4afb-ac66-b70139287793 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:02.915536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9f115d0e-520a-43e0-8e24-93ef49d5ce10 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:02.916734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9f115d0e-520a-43e0-8e24-93ef49d5ce10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:02.927404 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2971/5941] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:02.980767 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2971/5942] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:02.988730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784746-492f-4b35-bf39-acec4f50db8f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:02.989378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784746-492f-4b35-bf39-acec4f50db8f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:02.989666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784746-492f-4b35-bf39-acec4f50db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.012374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-21784746-492f-4b35-bf39-acec4f50db8f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:03.013684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21784746-492f-4b35-bf39-acec4f50db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:03.025479 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2972/5943] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:03.030048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f115d0e-520a-43e0-8e24-93ef49d5ce10 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.040538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f115d0e-520a-43e0-8e24-93ef49d5ce10 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.046635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.047249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.047669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.047955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.048603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:03.048907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.050553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.066789 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2972/5944] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:03.067946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21784746-492f-4b35-bf39-acec4f50db8f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.079896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21784746-492f-4b35-bf39-acec4f50db8f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.085578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.086275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.086652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.086843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.087458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.089594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.122935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2973/5945] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:03 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:03.127050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.138015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb14011-b52f-4a3a-8bb3-e407cac6c64c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.161503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:03.215198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.216279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.216905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.217426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.218631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.221264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.247374 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2973/5946] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:03 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:03.251871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.264470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd8404-7c91-42f0-8ebd-d004a3973261 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.269899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.270601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.270950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.271324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.272060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.273738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.289178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2974/5947] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:03.291264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.304095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b25fe-c8b3-43d0-a896-136375d68655 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.311967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288503b-6985-42de-b2fd-81072d0b9b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.312950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288503b-6985-42de-b2fd-81072d0b9b99 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.314300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288503b-6985-42de-b2fd-81072d0b9b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.314931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288503b-6985-42de-b2fd-81072d0b9b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.316168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288503b-6985-42de-b2fd-81072d0b9b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.337866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288503b-6985-42de-b2fd-81072d0b9b99 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.352332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2974/5948] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:03.356935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.369253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8ceaf-14cc-4fd8-bb80-75b874f263ae admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.376122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.377029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.377508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.378210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] PATH_INFO: `/v3/projects/acfab651c3454dedba6c2a92bb6bfba5/users/177acfc023fb4a8493495104a23b7f4a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.409211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfab651c3454dedba6c2a92bb6bfba5, user_id=177acfc023fb4a8493495104a23b7f4a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.412059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.456727 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2975/5949] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:03 2022] PUT /identity/v3/projects/acfab651c3454dedba6c2a92bb6bfba5/users/177acfc023fb4a8493495104a23b7f4a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:03.458290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29a6f4-dae3-446b-9808-6a4cde027be9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.458769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29a6f4-dae3-446b-9808-6a4cde027be9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.459043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29a6f4-dae3-446b-9808-6a4cde027be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.490933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d29a6f4-dae3-446b-9808-6a4cde027be9 None None] MFA Rules not processed for user `177acfc023fb4a8493495104a23b7f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:03.495759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d29a6f4-dae3-446b-9808-6a4cde027be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:03.608651 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2975/5950] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:03.613240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.734982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425dcdd-8197-4415-ba1b-a293d57667ad admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.739759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.740380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.740660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.740931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.741589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.743537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.782935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2976/5951] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:03 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:03.786792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29a6f4-dae3-446b-9808-6a4cde027be9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:03.810778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29a6f4-dae3-446b-9808-6a4cde027be9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:03.950315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:03.951058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:03.951425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:03.951781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:03.952679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:03.954896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:03.998942 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2976/5952] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:03 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:04.003328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.018432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd7ed4-2b1f-4b50-acfd-042d662d346c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.024838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.025720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.026184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.026535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.027650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.030043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.047180 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2977/5953] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:04.051057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.082898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa709ae3-338f-4284-8065-06b2fa20247c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.092332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.095414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.096370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.097481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] PATH_INFO: `/v3/projects/fc104676560c4353ace0a0f669417dcf/users/2697909d8c164468ae364f0ba7a8a06c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.137439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc104676560c4353ace0a0f669417dcf, user_id=2697909d8c164468ae364f0ba7a8a06c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.143298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.146750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:04.169264 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2977/5954] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:04 2022] PUT /identity/v3/projects/fc104676560c4353ace0a0f669417dcf/users/2697909d8c164468ae364f0ba7a8a06c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:04.172899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.186568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d41542-9af2-488f-93f8-e8dd450040f2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.287856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:04.303925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60fcbf-1172-435a-8f29-c4ec7ca248c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.304798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fcbf-1172-435a-8f29-c4ec7ca248c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.305181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fcbf-1172-435a-8f29-c4ec7ca248c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.306146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fcbf-1172-435a-8f29-c4ec7ca248c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.306820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60fcbf-1172-435a-8f29-c4ec7ca248c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.309066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60fcbf-1172-435a-8f29-c4ec7ca248c0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.350852 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2978/5955] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:04.356987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.374871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa68a5-dede-4520-bb0a-47077149692b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.394986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.397810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.402032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.402537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] PATH_INFO: `/v3/projects/fc104676560c4353ace0a0f669417dcf/users/2697909d8c164468ae364f0ba7a8a06c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.408052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc104676560c4353ace0a0f669417dcf, user_id=2697909d8c164468ae364f0ba7a8a06c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.410923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.445689 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2978/5956] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:04 2022] PUT /identity/v3/projects/fc104676560c4353ace0a0f669417dcf/users/2697909d8c164468ae364f0ba7a8a06c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:04.458079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112916b8-2697-485b-afa7-06b54412517d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.458689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112916b8-2697-485b-afa7-06b54412517d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.458689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112916b8-2697-485b-afa7-06b54412517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.502741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-112916b8-2697-485b-afa7-06b54412517d None None] MFA Rules not processed for user `2697909d8c164468ae364f0ba7a8a06c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:04.504494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-112916b8-2697-485b-afa7-06b54412517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:04.609034 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2979/5957] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:04.613055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.624945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e06913-d283-42cd-a53c-32a041ccd3da admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.630088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.630830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.631403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.631858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.632513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.634717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.680348 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2979/5958] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:13:04 2022] POST /identity/v3/projects => generated 349 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:04.684459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112916b8-2697-485b-afa7-06b54412517d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.694151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112916b8-2697-485b-afa7-06b54412517d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.749887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.750530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.750818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.751094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.751754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.753677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.787476 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2980/5959] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:04 2022] POST /identity/v3/projects => generated 351 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:04.791712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.803826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde686e0-9316-4f3d-aca7-e1ded3dfab56 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.866474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.867113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.867499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.867696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/users/177acfc023fb4a8493495104a23b7f4a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.868782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:get_user(user_id=177acfc023fb4a8493495104a23b7f4a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.871627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.873253 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2980/5960] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:13:04 2022] GET /identity/v3/users/177acfc023fb4a8493495104a23b7f4a => generated 287 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:04.878916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.893269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac92ed7-5cea-4c4a-8dbd-e14beddeb35c tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.899603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.900250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.900541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.900861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.901638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.904341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.931565 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2981/5961] 10.222.0.33 () {60 vars in 1162 bytes} [Sat Sep 10 21:13:04 2022] POST /identity/v3/credentials => generated 431 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:04.932475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:04.949843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5872cd-44db-4e28-a71f-37d850e0f097 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:04.954979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:04.955733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:04.956118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:04.956501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials/954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:04.959035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.960896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.964896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:04.967691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:04.985579 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2981/5962] 10.222.0.33 () {60 vars in 1358 bytes} [Sat Sep 10 21:13:04 2022] PATCH /identity/v3/credentials/954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9 => generated 436 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:04.986737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.003972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e3ef-87ab-4853-983c-46c9d0bc0f43 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.013365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.014296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.014779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.015223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials/954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.017417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.019374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.025258 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2982/5963] 10.222.0.33 () {58 vars in 1335 bytes} [Sat Sep 10 21:13:04 2022] GET /identity/v3/credentials/954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9 => generated 436 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:05.026937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.038757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6faca-daec-4436-a9ef-412c336c07da tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.045044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.045902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.046254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.046633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials/954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.048775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.050632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.075679 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2982/5964] 10.222.0.33 () {58 vars in 1338 bytes} [Sat Sep 10 21:13:05 2022] DELETE /identity/v3/credentials/954c9629c6cda9bf8f1e5b60d54dfe4de894a1b2e485c526e5840e2f753a5bf9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:05.081004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.096220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751877c-6e46-48db-85a7-11c344802209 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.101177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.101886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.102302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.102610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.103258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.105513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.122864 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2983/5965] 10.222.0.33 () {60 vars in 1162 bytes} [Sat Sep 10 21:13:05 2022] POST /identity/v3/credentials => generated 431 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:05.126976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.138440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74263eeb-1ed0-459d-8669-7dcc148bffc2 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.143475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.144306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.144737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.145245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.145914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.147642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.167630 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2983/5966] 10.222.0.33 () {60 vars in 1162 bytes} [Sat Sep 10 21:13:05 2022] POST /identity/v3/credentials => generated 430 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:05.171743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.182026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ad8f-b21f-420e-8afd-539eed3c0d11 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.187312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.187934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.188215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.188498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.189171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.191721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.200817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.202564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.204110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.205672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.207555 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2984/5967] 10.222.0.33 () {58 vars in 1140 bytes} [Sat Sep 10 21:13:05 2022] GET /identity/v3/credentials => generated 946 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:05.211822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.223051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c360-2180-4a2e-8d7e-ea0043198dae tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.227757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.228410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.228957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.229231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials/15f4bf27f0fc3f62d959feec4b7b536926822d4816f1575ac0abb66c61d2137c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.231659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=15f4bf27f0fc3f62d959feec4b7b536926822d4816f1575ac0abb66c61d2137c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.233380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.249533 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2984/5968] 10.222.0.33 () {58 vars in 1338 bytes} [Sat Sep 10 21:13:05 2022] DELETE /identity/v3/credentials/15f4bf27f0fc3f62d959feec4b7b536926822d4816f1575ac0abb66c61d2137c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:05.254063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.265168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e677f-cc85-4d04-b209-76c4dd80d261 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.270027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.270655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.270994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.271271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/credentials/27686efe7e3b29f7abd641dab49dfbef42340dc7dc7930b974134689c65d8019` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.273647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=27686efe7e3b29f7abd641dab49dfbef42340dc7dc7930b974134689c65d8019)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.275355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.293306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2985/5969] 10.222.0.33 () {58 vars in 1338 bytes} [Sat Sep 10 21:13:05 2022] DELETE /identity/v3/credentials/27686efe7e3b29f7abd641dab49dfbef42340dc7dc7930b974134689c65d8019 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:05.297931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.310365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52427-3c00-48d3-8497-1184d5f95681 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.315958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.317369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.317921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.318381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/projects/098147c46770478bab09efa81ee13b98` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.319545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:delete_project(project_id=098147c46770478bab09efa81ee13b98)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.322173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.380439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098147c46770478bab09efa81ee13b98', 'request_id': 'req-a170636f-a848-4f49-a26d-54389c939c66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.388600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098147c46770478bab09efa81ee13b98', 'request_id': 'req-a170636f-a848-4f49-a26d-54389c939c66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.395164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:05.396268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098147c46770478bab09efa81ee13b98', 'request_id': 'req-a170636f-a848-4f49-a26d-54389c939c66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.413375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] The token cache is being invalidate because project 098147c46770478bab09efa81ee13b98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:05.413675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.414593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.417914 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2985/5970] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:05 2022] DELETE /identity/v3/projects/098147c46770478bab09efa81ee13b98 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:05.420714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.422575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:05.437510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dfcba-3a9b-48e0-b15a-0300e1929c7b tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.446494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:05.515876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2697909d8c164468ae364f0ba7a8a06c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc104676560c4353ace0a0f669417dcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.516598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.516788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.517065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] PATH_INFO: `/v3/projects/a2c8fdcd343d4bc9873ccd1378aa3f9e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.518240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a2c8fdcd343d4bc9873ccd1378aa3f9e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.521143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.574640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c8fdcd343d4bc9873ccd1378aa3f9e', 'request_id': 'req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.582013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c8fdcd343d4bc9873ccd1378aa3f9e', 'request_id': 'req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.591523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c8fdcd343d4bc9873ccd1378aa3f9e', 'request_id': 'req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.602398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] The token cache is being invalidate because project a2c8fdcd343d4bc9873ccd1378aa3f9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:05.602748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.603492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-23b32ab3-2d17-4be3-9bd4-c5a9216dd92e tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.605611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2986/5971] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:05 2022] DELETE /identity/v3/projects/a2c8fdcd343d4bc9873ccd1378aa3f9e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:05.609711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.611325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:05.683977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170636f-a848-4f49-a26d-54389c939c66 tempest-CredentialsTestJSON-297268769 tempest-CredentialsTestJSON-297268769-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:05.744717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:05.745942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.746227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.746499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] PATH_INFO: `/v3/users/177acfc023fb4a8493495104a23b7f4a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.747682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=177acfc023fb4a8493495104a23b7f4a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:05.749576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:05.803377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '177acfc023fb4a8493495104a23b7f4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.818572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '177acfc023fb4a8493495104a23b7f4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.827516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '177acfc023fb4a8493495104a23b7f4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.836173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '177acfc023fb4a8493495104a23b7f4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.837069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '177acfc023fb4a8493495104a23b7f4a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:05.845058 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2986/5972] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:05 2022] DELETE /identity/v3/users/177acfc023fb4a8493495104a23b7f4a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:05.849415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:05.849705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:05.850035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:05.874263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:05.875461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:05.944276 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2987/5973] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:05.954661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:05.956068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:05.988446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:06.036953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb20c4-0f2f-43ed-915a-8949255c4966 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:06.040827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:06.041516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:06.042428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:06.042811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:06.052678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:06.054256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:06.065305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2987/5974] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:06.423968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:06.425745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:06.439728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b3a7d-76ce-42bd-a897-e7f03eb96f04 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:06.444890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:06.445526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:06.445925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:06.446269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] PATH_INFO: `/v3/projects/acfab651c3454dedba6c2a92bb6bfba5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:06.447439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acfab651c3454dedba6c2a92bb6bfba5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:06.449749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:06.491706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:06.508501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acfab651c3454dedba6c2a92bb6bfba5', 'request_id': 'req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.517652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acfab651c3454dedba6c2a92bb6bfba5', 'request_id': 'req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.531482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acfab651c3454dedba6c2a92bb6bfba5', 'request_id': 'req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.547294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] The token cache is being invalidate because project acfab651c3454dedba6c2a92bb6bfba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:06.547776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.550161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.554131 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2988/5975] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:06 2022] DELETE /identity/v3/projects/acfab651c3454dedba6c2a92bb6bfba5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:06.558516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:06.571920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4479d-bbda-4dc9-8177-69bc892b32a9 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:06.662568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:06.664932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:06.665216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:06.665491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] PATH_INFO: `/v3/users/2697909d8c164468ae364f0ba7a8a06c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:06.670045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2697909d8c164468ae364f0ba7a8a06c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:06.672587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:06.763432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2697909d8c164468ae364f0ba7a8a06c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.775557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2697909d8c164468ae364f0ba7a8a06c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.783280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2697909d8c164468ae364f0ba7a8a06c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.791103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2697909d8c164468ae364f0ba7a8a06c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.791784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9e17bf1f-b8f5-4c00-9796-c5d590639886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2697909d8c164468ae364f0ba7a8a06c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:06.798738 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2988/5976] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:06 2022] DELETE /identity/v3/users/2697909d8c164468ae364f0ba7a8a06c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:07.326658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:07.328141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:07.342584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1815a0-ee78-4b0f-94e2-4c6dd0aa4a82 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:07.406863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:07.407507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.407816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.408091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] PATH_INFO: `/v3/projects/fc104676560c4353ace0a0f669417dcf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.409202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc104676560c4353ace0a0f669417dcf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:07.411197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:07.479043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc104676560c4353ace0a0f669417dcf', 'request_id': 'req-967d61fa-15bf-458d-85b7-336989b8192f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:07.487054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc104676560c4353ace0a0f669417dcf', 'request_id': 'req-967d61fa-15bf-458d-85b7-336989b8192f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:07.495933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc104676560c4353ace0a0f669417dcf', 'request_id': 'req-967d61fa-15bf-458d-85b7-336989b8192f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:07.508990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] The token cache is being invalidate because project fc104676560c4353ace0a0f669417dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:07.509288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:07.510003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-967d61fa-15bf-458d-85b7-336989b8192f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:07.512079 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2989/5977] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:07 2022] DELETE /identity/v3/projects/fc104676560c4353ace0a0f669417dcf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:07.517302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e2dd-06d4-49a3-810c-42a006954edc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.517756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e2dd-06d4-49a3-810c-42a006954edc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.518547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e2dd-06d4-49a3-810c-42a006954edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.545474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3ad8e2dd-06d4-49a3-810c-42a006954edc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:07.547066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3ad8e2dd-06d4-49a3-810c-42a006954edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:07.602309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:07.640285 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2989/5978] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:07.648083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958bb1-4ea0-4ae8-8f7e-5232c5d4169a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.648596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958bb1-4ea0-4ae8-8f7e-5232c5d4169a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.648723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958bb1-4ea0-4ae8-8f7e-5232c5d4169a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.670017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d9958bb1-4ea0-4ae8-8f7e-5232c5d4169a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:07.671257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d9958bb1-4ea0-4ae8-8f7e-5232c5d4169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:07.679631 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2990/5979] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:07.684360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8e2dd-06d4-49a3-810c-42a006954edc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:07.694235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8e2dd-06d4-49a3-810c-42a006954edc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:07.700065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:07.700685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.700973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.701255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.702260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:07.702260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:07.703846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:07.716417 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2990/5980] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:07.720715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9958bb1-4ea0-4ae8-8f7e-5232c5d4169a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:07.730921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9958bb1-4ea0-4ae8-8f7e-5232c5d4169a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:07.735721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:07.736325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.736635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.736920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.737547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:07.739504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:07.774385 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2991/5981] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:07 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:07.779057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:07.791637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c6d56-1026-4523-b1ce-474b6d236226 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:07.857636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:07.858265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.858547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.858821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.859443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:07.861390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:07.884826 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2991/5982] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:07 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:07.889563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:07.900649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7550b53-fc61-4783-a117-ef996a59667f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:07.906066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:07.906715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.907003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.907848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.908504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:07.910150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:07.923854 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2992/5983] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:07.928020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:07.944092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58039-5289-4407-bbc5-f4a07a6f3804 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:07.950166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36afa7f-1c78-4bfa-983b-e9c777d49742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:07.950852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36afa7f-1c78-4bfa-983b-e9c777d49742 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.951202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36afa7f-1c78-4bfa-983b-e9c777d49742 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.951551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36afa7f-1c78-4bfa-983b-e9c777d49742 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.952262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36afa7f-1c78-4bfa-983b-e9c777d49742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:07.953960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36afa7f-1c78-4bfa-983b-e9c777d49742 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:07.969649 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2992/5984] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:07.970158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:07.983466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607904b-6fe6-4128-9dc8-ac34b583a022 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:07.988254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942696f-6ebf-4180-b77a-6abe34e8992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:07.988915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942696f-6ebf-4180-b77a-6abe34e8992c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.989291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942696f-6ebf-4180-b77a-6abe34e8992c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.989466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942696f-6ebf-4180-b77a-6abe34e8992c admin admin] PATH_INFO: `/v3/projects/6ca520e0f2f847e9b12523a7dbebc13c/users/872de0d813a1487d9e343c8c328bca05/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:07.992146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:07.992477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:07.992690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.007040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:08.016055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:08.017431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:08.019967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942696f-6ebf-4180-b77a-6abe34e8992c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca520e0f2f847e9b12523a7dbebc13c, user_id=872de0d813a1487d9e343c8c328bca05, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:08.024474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942696f-6ebf-4180-b77a-6abe34e8992c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:08.047928 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2993/5985] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:07 2022] PUT /identity/v3/projects/6ca520e0f2f847e9b12523a7dbebc13c/users/872de0d813a1487d9e343c8c328bca05/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:08.058050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f878d-4590-4be6-ab80-2cdbc6ed8e17 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.058894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f878d-4590-4be6-ab80-2cdbc6ed8e17 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.058894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f878d-4590-4be6-ab80-2cdbc6ed8e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.097403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-395f878d-4590-4be6-ab80-2cdbc6ed8e17 None None] MFA Rules not processed for user `872de0d813a1487d9e343c8c328bca05`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:08.099294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-395f878d-4590-4be6-ab80-2cdbc6ed8e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:08.123755 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2993/5986] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:08.140306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:08.142068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:08.244594 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2994/5987] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:08.245249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:08.249897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f878d-4590-4be6-ab80-2cdbc6ed8e17 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:08.283621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d0354-03a7-4bc7-a186-52e0976f76e2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:08.348958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f878d-4590-4be6-ab80-2cdbc6ed8e17 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:08.365110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:08.365934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.366221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.366494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.367393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:08.370835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:08.382246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:08.383187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.383570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.383944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.412000 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2995/5988] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:08 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:08.415688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:08.500103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:08.505659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:08.506298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf29e5f-6561-4699-a027-20605ea3b054 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:08.511891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:08.512574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.512858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.513177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.513960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:08.515954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:08.527739 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2994/5989] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:08.529580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:08.554391 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2996/5990] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:08 2022] POST /identity/v3/users => generated 301 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:08.559153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:08.583584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec06b97-7ea2-4782-8ce5-f13c53dff555 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:08.591179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:08.591981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.599264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.599842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.605265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:08.607303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:08.627412 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2995/5991] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:08.627642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:08.646195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b34836-9c9d-4460-9419-fc8fb31bce45 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:08.661656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:08.662912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.663358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.663689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] PATH_INFO: `/v3/projects/4289920029f34ed9baa14d34dc3b1c30/users/34fae5192ae84cd3acf56c4a9d753634/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.709032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4289920029f34ed9baa14d34dc3b1c30, user_id=34fae5192ae84cd3acf56c4a9d753634, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:08.718253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:08.739797 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2997/5992] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:08 2022] PUT /identity/v3/projects/4289920029f34ed9baa14d34dc3b1c30/users/34fae5192ae84cd3acf56c4a9d753634/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:08.745941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:08.758155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe96b6-2913-40e8-895c-e47358b8da40 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:08.899887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47bab8-d01f-4a14-862e-2a3cd67d2dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:08.900922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47bab8-d01f-4a14-862e-2a3cd67d2dbe admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.901474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47bab8-d01f-4a14-862e-2a3cd67d2dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.902012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47bab8-d01f-4a14-862e-2a3cd67d2dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.902916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47bab8-d01f-4a14-862e-2a3cd67d2dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:08.907505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47bab8-d01f-4a14-862e-2a3cd67d2dbe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:08.935967 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2996/5993] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:08.940181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:08.956661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633c4a5-3f3f-4927-927c-c0383846046c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:08.977544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:08.978262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:08.978546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:08.979091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] PATH_INFO: `/v3/projects/4289920029f34ed9baa14d34dc3b1c30/users/34fae5192ae84cd3acf56c4a9d753634/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:08.991590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4289920029f34ed9baa14d34dc3b1c30, user_id=34fae5192ae84cd3acf56c4a9d753634, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.002638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.047039 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2998/5994] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:08 2022] PUT /identity/v3/projects/4289920029f34ed9baa14d34dc3b1c30/users/34fae5192ae84cd3acf56c4a9d753634/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:09.059473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb186228-7a2f-4b22-ba88-1c8620d675c6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.060607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb186228-7a2f-4b22-ba88-1c8620d675c6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.061989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb186228-7a2f-4b22-ba88-1c8620d675c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.166677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bb186228-7a2f-4b22-ba88-1c8620d675c6 None None] MFA Rules not processed for user `34fae5192ae84cd3acf56c4a9d753634`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:09.172918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bb186228-7a2f-4b22-ba88-1c8620d675c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:09.189257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:09.351693 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2997/5995] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:09.358667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.378445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d742-643b-4fd5-b6fe-30e3615efd3c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.383670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.384742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.385285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.386032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.387378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.389801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.415234 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 2999/5996] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:13:09 2022] POST /identity/v3/domains => generated 277 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:09.419932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb186228-7a2f-4b22-ba88-1c8620d675c6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.430766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb186228-7a2f-4b22-ba88-1c8620d675c6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.528891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.530125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.530742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.531340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943/config` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.532408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=8543b1b9388b43708898eb1fc99a8943)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.534244 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:09.534244 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:09.534244 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:09.535902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.611179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2998/5997] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:13:09 2022] PUT /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943/config => generated 138 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:09.614896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.630484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6664e-d583-4974-8b83-89f1010b58f7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.636401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.637342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.637685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.637970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.639216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=8543b1b9388b43708898eb1fc99a8943, group=identity)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.641606 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:09.641606 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:09.641606 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:09.641606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.667477 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3000/5998] 10.222.0.33 () {58 vars in 1275 bytes} [Sat Sep 10 21:13:09 2022] GET /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/identity => generated 45 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:09.668070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.689819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474c95c-0cac-4b49-b586-2fa18a770651 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.702611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.703379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.703587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.703868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/identity/driver` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.705256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=8543b1b9388b43708898eb1fc99a8943, group=identity, option=driver)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.706210 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:09.706210 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:09.706210 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:09.707128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.714590 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 2999/5999] 10.222.0.33 () {58 vars in 1296 bytes} [Sat Sep 10 21:13:09 2022] GET /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/identity/driver => generated 31 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:09.718678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.734647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e3e0e-63a1-4a1c-8b4e-fd8939161084 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.742004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.742979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.743637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.744132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/ldap` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.745577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=8543b1b9388b43708898eb1fc99a8943, group=ldap)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.748601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.754860 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3001/6000] 10.222.0.33 () {58 vars in 1263 bytes} [Sat Sep 10 21:13:09 2022] GET /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/ldap => generated 106 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:13:09.758802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.774488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edfc36-83db-4ba9-a613-7627ea31173c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.779888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.780881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.781492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.782038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/ldap/url` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.783469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=8543b1b9388b43708898eb1fc99a8943, group=ldap, option=url)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.785300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.791175 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3000/6001] 10.222.0.33 () {58 vars in 1275 bytes} [Sat Sep 10 21:13:09 2022] GET /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/ldap/url => generated 46 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:09.803357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.830020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6cfde-6763-4dcc-a610-df7b849be1a2 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.835652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.836585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.837090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.837603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/ldap/user_tree_dn` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.839993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=8543b1b9388b43708898eb1fc99a8943, group=ldap, option=user_tree_dn)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.841872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.849975 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3002/6002] 10.222.0.33 () {58 vars in 1302 bytes} [Sat Sep 10 21:13:09 2022] GET /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943/config/ldap/user_tree_dn => generated 63 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:09.855623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.869272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e61f-a6ae-4495-8a9c-3a0c77886aa1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.875570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.876722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.877256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.877766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943/config` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.879317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=8543b1b9388b43708898eb1fc99a8943)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.880835 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:09.880835 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:09.880835 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:09.882179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.896740 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3001/6003] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:13:09 2022] DELETE /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943/config => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:09.903007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:09.914922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2355-3fbe-472b-9e68-437321765322 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:09.921509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:09.922544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:09.923054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:09.923547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:09.925084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=8543b1b9388b43708898eb1fc99a8943)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:09.927188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:09.987170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '8543b1b9388b43708898eb1fc99a8943'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:09.988087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '8543b1b9388b43708898eb1fc99a8943'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.039506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '8543b1b9388b43708898eb1fc99a8943'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.040257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '8543b1b9388b43708898eb1fc99a8943'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.044740 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3003/6004] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:09 2022] PATCH /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943 => generated 278 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:10.048770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:10.050938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:10.074835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70ace4-80ab-4314-b9d3-cd74710dabae tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:10.205535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:10.207086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:10.216104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:10.249778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:10.250671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:10.251059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:10.251605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:10.283846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=8543b1b9388b43708898eb1fc99a8943)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:10.285674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:10.342820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '8543b1b9388b43708898eb1fc99a8943'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.355657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '8543b1b9388b43708898eb1fc99a8943'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.398202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db1bfb-44cb-449e-b472-ce1c10b6dc13 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:10.407518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:10.409293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:10.409588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:10.411236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:10.414647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:10.461797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8543b1b9388b43708898eb1fc99a8943', 'request_id': 'req-48b14d19-4150-4f08-89b3-50a3437feef1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.475602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8543b1b9388b43708898eb1fc99a8943', 'request_id': 'req-48b14d19-4150-4f08-89b3-50a3437feef1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.491322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8543b1b9388b43708898eb1fc99a8943', 'request_id': 'req-48b14d19-4150-4f08-89b3-50a3437feef1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.517459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] The token cache is being invalidate because project 8543b1b9388b43708898eb1fc99a8943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:10.517933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.518910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.538107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '8543b1b9388b43708898eb1fc99a8943', 'request_id': 'req-48b14d19-4150-4f08-89b3-50a3437feef1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.539572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '8543b1b9388b43708898eb1fc99a8943', 'request_id': 'req-48b14d19-4150-4f08-89b3-50a3437feef1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.543248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=5,keystone:SELECT=94,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:10.549937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '8543b1b9388b43708898eb1fc99a8943', 'request_id': 'req-48b14d19-4150-4f08-89b3-50a3437feef1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:10.575203 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3002/6005] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:10 2022] DELETE /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:10.576656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:10.581015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:10.613700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:10.615308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:10.663995 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3004/6006] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:13:10.724366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b14d19-4150-4f08-89b3-50a3437feef1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:10.729978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:10.731217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:10.731217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:10.731217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:10.731823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:10.733443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:10.760895 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3003/6007] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:13:10 2022] POST /identity/v3/domains => generated 276 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:10.764965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:10.782009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0049-07f9-4e49-b201-f726e33558bf service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:10.900136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:10.900894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:10.901844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:10.902225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55/config` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:10.903095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:10.904223 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:10.904223 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:10.904223 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:10.905298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:10.960472 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3005/6008] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:13:10 2022] PUT /identity/v3/domains/c8d882f2baa144c898572b405c07ad55/config => generated 138 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:10.961273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:10.973702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6b5cd-569f-431b-a0d8-7a3b45760234 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:10.978507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:10.980095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:10.980254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:10.980537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55/config` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:10.981219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:10.982258 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:10.982258 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:10.982258 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:10.983159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:10.994325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:11.019657 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3004/6009] 10.222.0.33 () {60 vars in 1271 bytes} [Sat Sep 10 21:13:10 2022] PATCH /identity/v3/domains/c8d882f2baa144c898572b405c07ad55/config => generated 142 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:11.033851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.044834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38181b8-6d2d-48eb-b11e-533b93799478 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:11.048020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:11.048988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.049372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.049957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55/config` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.054542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:11.056512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:11.071171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3006/6010] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:13:11 2022] GET /identity/v3/domains/c8d882f2baa144c898572b405c07ad55/config => generated 142 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:11.072224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.087286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51921-6f63-49c0-b727-a969093dbe61 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:11.099202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:11.101314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.101782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.105038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55/config` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.106158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:11.110490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:11.141969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3005/6011] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:13:11 2022] DELETE /identity/v3/domains/c8d882f2baa144c898572b405c07ad55/config => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:11.146441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.162593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3dcd8-0304-40aa-9a75-1cf478396d1b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:11.172355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:11.173015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.173304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.173606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55/config` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.174789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:11.176419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Could not find any options in domain configuration for domain c8d882f2baa144c898572b405c07ad55.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain c8d882f2baa144c898572b405c07ad55. Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 10 21:13:11.185629 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain c8d882f2baa144c898572b405c07ad55. Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:11.190332 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3007/6012] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:13:11 2022] GET /identity/v3/domains/c8d882f2baa144c898572b405c07ad55/config => generated 151 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:11.192825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.211115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c621f-2c07-403f-a4c7-8e4fd6b7180e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:11.224658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:11.225702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.226197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.226682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55/config` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.227447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:11.230122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:11.238247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3006/6013] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:13:11 2022] DELETE /identity/v3/domains/c8d882f2baa144c898572b405c07ad55/config => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:11.242629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.257500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098ddb7-d70b-40e5-8971-a584875e9238 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:11.264782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:11.266193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.267480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.268013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.269601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:11.271612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:11.320893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c8d882f2baa144c898572b405c07ad55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.322046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c8d882f2baa144c898572b405c07ad55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.406334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c8d882f2baa144c898572b405c07ad55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.407013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b0514cd-813f-4c48-a89d-54f62566c199 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c8d882f2baa144c898572b405c07ad55'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.413048 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3008/6014] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:11 2022] PATCH /identity/v3/domains/c8d882f2baa144c898572b405c07ad55 => generated 277 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:11.417519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.430480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:11.469027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c95ff-a575-4088-9fad-8a3ffaec8a1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:11.572128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:11.573688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.578933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.579328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.601724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c8d882f2baa144c898572b405c07ad55)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:11.603936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:11.613268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.613557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.613874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.633738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] MFA Rules not processed for user `5346063245704d50b918a54c29bc297a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:11.635003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:11.644132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c8d882f2baa144c898572b405c07ad55'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.648174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c8d882f2baa144c898572b405c07ad55'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.676516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:11.721118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d882f2baa144c898572b405c07ad55', 'request_id': 'req-8b1290a3-e797-4a4e-8ac2-fa152f596e71'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.723200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3009/6015] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:11.728985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d882f2baa144c898572b405c07ad55', 'request_id': 'req-8b1290a3-e797-4a4e-8ac2-fa152f596e71'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.750661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d882f2baa144c898572b405c07ad55', 'request_id': 'req-8b1290a3-e797-4a4e-8ac2-fa152f596e71'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.770390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.771871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:11.792991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] The token cache is being invalidate because project c8d882f2baa144c898572b405c07ad55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:11.793269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.794326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.810576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c8d882f2baa144c898572b405c07ad55', 'request_id': 'req-8b1290a3-e797-4a4e-8ac2-fa152f596e71'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.811496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c8d882f2baa144c898572b405c07ad55', 'request_id': 'req-8b1290a3-e797-4a4e-8ac2-fa152f596e71'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.815390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c8d882f2baa144c898572b405c07ad55', 'request_id': 'req-8b1290a3-e797-4a4e-8ac2-fa152f596e71'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:11.839929 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3007/6016] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:11 2022] DELETE /identity/v3/domains/c8d882f2baa144c898572b405c07ad55 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:11.847109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:11.848831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:11.855669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=3,keystone:DELETE=17,keystone:SELECT=80 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:11.886419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f9d55-308e-4a74-9cfc-7dcdbbbbb701 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:11.967690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:11.968577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:11.968961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:11.969463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:11.971454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:12.056864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1290a3-e797-4a4e-8ac2-fa152f596e71 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.065241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.066473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.066986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.067668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.068655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.070763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.099853 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3008/6017] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:13:11 2022] POST /identity/v3/domains => generated 276 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:12.100861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:12.102612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:12.137223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.138712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.165840 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3010/6018] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:13:12.167548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:12.232469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a8972-37c1-4165-bc1f-e4ebdb5e70f9 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.251741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c15fa-5f4c-4f02-81a3-4cffd567d6c4 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.264774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.265555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.268680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.269083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.271114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:12.407235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:12.421318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.423122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.434651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.435494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.435763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.436213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.436736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.438498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.445020 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3009/6019] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:12.517058 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3011/6020] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:13:12 2022] PUT /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config => generated 138 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:12.521412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:12.552303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab68c0-ba9e-4e27-9783-64d232ea89a1 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.560786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.561655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.561968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.562246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.563218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517, group=identity)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.568626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.641361 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3010/6021] 10.222.0.33 () {60 vars in 1297 bytes} [Sat Sep 10 21:13:12 2022] PATCH /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity => generated 176 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:12.645392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:12.665928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63813cf3-23a0-480d-8ece-886370463454 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.681749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.682820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.683705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.684202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity/driver` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.687028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517, group=identity, option=driver)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.688316 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:12.688316 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:12.688316 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:12.691999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.766677 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3012/6022] 10.222.0.33 () {60 vars in 1318 bytes} [Sat Sep 10 21:13:12 2022] PATCH /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity/driver => generated 177 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:12.771543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:12.779633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:12.817925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46728f4d-3171-4b59-b1ac-1bb6ca0dc773 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.830645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.831760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.832437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.833008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity/driver` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.835331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517, group=identity, option=driver)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.838772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.874385 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3011/6023] 10.222.0.33 () {58 vars in 1299 bytes} [Sat Sep 10 21:13:12 2022] DELETE /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity/driver => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:12.878304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:12.893536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae8938-0f91-437a-a1c8-6ea23df00d64 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.900359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.903432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.903736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.904017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity/driver` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.905250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517, group=identity, option=driver)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.907661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Could not find option driver in group identity in domain configuration for domain f12df48146884e84b0a9fb83ea250517.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain f12df48146884e84b0a9fb83ea250517. Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:12.934488 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain f12df48146884e84b0a9fb83ea250517. Sep 10 21:13:12.935663 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:12.940769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:12.959273 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3013/6024] 10.222.0.33 () {58 vars in 1296 bytes} [Sat Sep 10 21:13:12 2022] GET /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity/driver => generated 171 bytes in 81 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:13:12.970222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72859e4d-cacf-4698-be7b-0679926c25e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:12.977758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:12.979890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:12.979890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:12.980104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:12.980649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517, group=identity)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:12.982320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:13.045459 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3012/6025] 10.222.0.33 () {58 vars in 1278 bytes} [Sat Sep 10 21:13:12 2022] DELETE /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:13.046646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:13.064881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c010796-66ed-4c93-befd-c19f8663e8a5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:13.071042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:13.072232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:13.072519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:13.072798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:13.074712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517, group=identity)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:13.077310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Could not find group identity in domain configuration for domain f12df48146884e84b0a9fb83ea250517.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain f12df48146884e84b0a9fb83ea250517. Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:13.092728 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain f12df48146884e84b0a9fb83ea250517. Sep 10 21:13:13.093694 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:13.094726 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3014/6026] 10.222.0.33 () {58 vars in 1275 bytes} [Sat Sep 10 21:13:13 2022] GET /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config/identity => generated 154 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:13.099497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:13.116862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af913fc9-749f-455c-a724-a794b9127c07 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:13.138688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:13.143586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:13.145532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:13.145854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517/config` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:13.148804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f12df48146884e84b0a9fb83ea250517)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:13.159089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:13.180002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:13.185932 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3013/6027] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:13:13 2022] DELETE /identity/v3/domains/f12df48146884e84b0a9fb83ea250517/config => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:13.235321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c2d91-4f79-4761-8ad9-4e8c4613e303 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:13.267109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:13.267362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:13.267640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:13.267856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:13.269235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f12df48146884e84b0a9fb83ea250517)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:13.270873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:13.331468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f12df48146884e84b0a9fb83ea250517'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.334963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f12df48146884e84b0a9fb83ea250517'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.409580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f12df48146884e84b0a9fb83ea250517'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.410538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f12df48146884e84b0a9fb83ea250517'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.417285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3015/6028] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:13 2022] PATCH /identity/v3/domains/f12df48146884e84b0a9fb83ea250517 => generated 277 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:13:13.421152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:13.423249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:13.458156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f88fb4-7933-4858-84e0-abd88e7cf961 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:13.605102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:13.606151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:13.606442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:13.606722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:13.620575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f12df48146884e84b0a9fb83ea250517)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:13.622232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:13.670499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f12df48146884e84b0a9fb83ea250517'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.676199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f12df48146884e84b0a9fb83ea250517'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.779509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12df48146884e84b0a9fb83ea250517', 'request_id': 'req-a7e496b3-b81b-4f53-becb-822d23515ea9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.787876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12df48146884e84b0a9fb83ea250517', 'request_id': 'req-a7e496b3-b81b-4f53-becb-822d23515ea9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.797504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12df48146884e84b0a9fb83ea250517', 'request_id': 'req-a7e496b3-b81b-4f53-becb-822d23515ea9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.817969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] The token cache is being invalidate because project f12df48146884e84b0a9fb83ea250517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:13.818737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.820031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.829359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:13.831894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f12df48146884e84b0a9fb83ea250517', 'request_id': 'req-a7e496b3-b81b-4f53-becb-822d23515ea9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.832818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f12df48146884e84b0a9fb83ea250517', 'request_id': 'req-a7e496b3-b81b-4f53-becb-822d23515ea9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.855649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f12df48146884e84b0a9fb83ea250517', 'request_id': 'req-a7e496b3-b81b-4f53-becb-822d23515ea9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:13.877004 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3014/6029] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:13 2022] DELETE /identity/v3/domains/f12df48146884e84b0a9fb83ea250517 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:13.877335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:13.890235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a17445-2aec-4285-8477-1e46bb38ec1a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.100013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.101508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.102152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.102733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.107786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.109293 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:14.109293 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:14.109293 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:14.112414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.129671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.132235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:14.153038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3016/6030] 10.222.0.33 () {58 vars in 1173 bytes} [Sat Sep 10 21:13:13 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:14.161691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e496b3-b81b-4f53-becb-822d23515ea9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.170908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.172787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.173499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.174043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.174960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.183587 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:14.183587 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:14.183587 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:14.185212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.190803 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3015/6031] 10.222.0.33 () {58 vars in 1200 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.192079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.216457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323530a-96e9-4497-afc0-c8318fe7215e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.242582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.247218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.248006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.248542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.250342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.253934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.264711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.265749 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3017/6032] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.280420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91c12d-5ac6-4bb0-9738-70f2a1f21993 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.293085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.298653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.299065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.299472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.300343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.306495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.315091 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3016/6033] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.317983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.336443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729beff-e4ff-44d7-8e39-1b595a2330da tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.348503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.349698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.350152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.351500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.351500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.358754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.367766 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3018/6034] 10.222.0.33 () {58 vars in 1188 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:14.369425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.386133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528b8af-37c8-4803-a56e-e983d0778ff0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.399500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.400750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.401747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.402320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.403216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.407490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.413216 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3017/6035] 10.222.0.33 () {58 vars in 1200 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.416383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.431173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689cdf7-6923-4f88-9180-434daaa16ef0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.446323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.447070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.447461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.447841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.448699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.450634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.457144 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3019/6036] 10.222.0.33 () {58 vars in 1203 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:14.461508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.474693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dd6d0-2ba6-4dcf-8540-c257a39df161 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.486041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.487131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.488775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.489256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.490969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.494740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.512295 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3018/6037] 10.222.0.33 () {58 vars in 1209 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.516700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.550615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74706d8-a9f8-4dc3-b6f5-c0fdb134627c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.595893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.596910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.597432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.597873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.599026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.602107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.605973 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3020/6038] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.609251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.630874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4a74d-5591-4a82-b797-83a503ede4c3 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.644935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.645618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.653427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.653427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.653427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.653427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.654901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.662893 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3019/6039] 10.222.0.33 () {58 vars in 1218 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:14.672308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448f474-8ebe-4f2f-8500-0976d7c8a274 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.685862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.686954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.687477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.688055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.688877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.690778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.695210 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3021/6040] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.700354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.712868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11378bc-b53a-4bb9-87e4-e6e6e0fb8224 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.724721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.725513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.725805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.726155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.727417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.733352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.741355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.750371 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3020/6041] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.761313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91dbc2-7d6d-4893-9ce7-d3ef17f4c676 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.783727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.784403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.786375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.786375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.786375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.787928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.788685 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3022/6042] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:14.794034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.809172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e616eaa-f2e4-4c87-b545-8d099ece290e tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.820988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.822883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.825930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.826733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.827367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.831499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.839325 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3021/6043] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.841940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.865188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5169d6-5c40-4dd3-9718-702166c1743b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.879920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.880844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.881134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.881429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.882270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.884266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.894644 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3023/6044] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.895400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.915210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b78f6f-256e-48b5-8df1-dde09128392a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:14.927396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:14.929732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:14.930042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:14.930225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:14.936315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:14.948686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:14.959994 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3022/6045] 10.222.0.33 () {58 vars in 1239 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:14.960167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:14.994503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3d37c-806f-4871-a471-6ecdb1016a8a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.028557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.029621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.030392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.031874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.032721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.042519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.046152 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3024/6046] 10.222.0.33 () {58 vars in 1242 bytes} [Sat Sep 10 21:13:14 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:15.050971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.084658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e158b-9a4f-402e-a7b5-a7f4013126c5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.110218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.110915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.114333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.114893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.115863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.126123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.133304 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3023/6047] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.134262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.164187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadbe28-f7ae-42ff-9cb9-c9ab45741281 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.177692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.178579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.178975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.179351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.180040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.185207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.198631 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3025/6048] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:15.200340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.220821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e79b4-e061-4724-80d9-84659f8e017a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.229647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.232543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.232543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.233750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.234839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.241537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.248711 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3024/6049] 10.222.0.33 () {58 vars in 1269 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.249979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.255422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:15.273451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3316-3706-475d-beaa-8a7ff32432c0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.293845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.294585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.294943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.295294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.295962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.297810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.303492 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3026/6050] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.306164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.338269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1779d5-3fb2-412e-96d6-46d1096c15df tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.355238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.355908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.356260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.356615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.357290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.359011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.363261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3025/6051] 10.222.0.33 () {58 vars in 1257 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.364697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.382096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cced95-34b0-424f-b348-36aed3cfdbea tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.392957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.393937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.394229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.394504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.395383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.398926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.403663 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3027/6052] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.404891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.430139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89689684-8a3d-46a6-b379-b699b0c00ed5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.439048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.440142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.440673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.441186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.442953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.444913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.451276 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3026/6053] 10.222.0.33 () {58 vars in 1242 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.452151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.466063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71993573-d297-423f-aeef-8d484a9ea6a1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.471929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.473393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.473676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.474027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.474824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.481097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.485484 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3028/6054] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.486836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.529248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a69cb-6907-4a02-b8c6-963b2adfc93b tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.562636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.563366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.564034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.564327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.565062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.566899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.574256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3027/6055] 10.222.0.33 () {58 vars in 1254 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.578741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.621430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a180482-d97a-4e49-bbf7-825534bae903 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.645148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.645815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.646184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.646448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.647352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.656243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.657445 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3029/6056] 10.222.0.33 () {58 vars in 1290 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.663170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.681368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d9dfa-b1e4-4934-a401-9c33905ce6e6 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.688005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.688935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.689397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.689876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.690649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.692611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.694934 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3028/6057] 10.222.0.33 () {58 vars in 1257 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:15.701891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.714584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddba86e-a24d-4c4f-a9e9-31cc0394325d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.723999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.725040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.726249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.727463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.731175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.733132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.736573 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3030/6058] 10.222.0.33 () {58 vars in 1266 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.742325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.761593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc2bc0-4572-471f-b96f-e20a14051946 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.770124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.773999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.774534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.775047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.775888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.781179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.783969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3029/6059] 10.222.0.33 () {58 vars in 1308 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.799690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.814591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c23da-dc69-45ba-9b61-a3d3f5791417 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.831525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.832814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.833430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.834038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.835185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.843727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.847958 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3031/6060] 10.222.0.33 () {58 vars in 1290 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:15.852154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.880611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b37a2-60c9-4549-b89b-53f9151d11c7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.902177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.903651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.904157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.904744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.905637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.908566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.911071 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3030/6061] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:15.916349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.934582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed5948-e38c-4836-a07b-8cb610ada0dd tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:15.948541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:15.949704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:15.950460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:15.950975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:15.951860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:15.956157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:15.959699 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3032/6062] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:15.967856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:15.994146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fe033-e0ca-4949-89ff-1598e20de8ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.001537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.002706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.003218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.003778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.004702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.007432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.010201 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3031/6063] 10.222.0.33 () {58 vars in 1242 bytes} [Sat Sep 10 21:13:15 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.016303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.032929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974305fd-70f4-44bf-8f98-35807d24ca5d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.041521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.043073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.043614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.044129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.045175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.047065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.049536 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3033/6064] 10.222.0.33 () {58 vars in 1245 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.054857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.070811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d49ef9-68d1-4afd-9820-790373699cc1 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.078827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.079988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.080587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.081177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.082671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.084712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.087538 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3032/6065] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.092853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.115137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb961a95-7d15-4aaf-a77d-4af8a58592e7 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.138459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.140291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.140816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.141314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.142333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.146599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.154928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.157801 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3034/6066] 10.222.0.33 () {58 vars in 1254 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.176713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692b49-b2da-4a63-8012-69f83962c7ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.200039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.201007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.201712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.202653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.204009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.206410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.212095 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3033/6067] 10.222.0.33 () {58 vars in 1257 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:16.215271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.258145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70a657-4c6a-4afd-b4a0-8174bc2ec662 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.285431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.286114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.286405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.286776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.288127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.290360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.296649 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3035/6068] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.298144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.321429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54744-1476-4093-814d-1b4d6b30b20d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.338606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.339472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.343462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.343904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.344736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.349241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.351557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3034/6069] 10.222.0.33 () {58 vars in 1257 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:13:16.356522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.370467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5514d-4452-493c-859a-30a1da01de96 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.385747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.386745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.387032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.387308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.387899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.389567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.390941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3036/6070] 10.222.0.33 () {58 vars in 1293 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.396896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.417886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e24c-f639-46a1-90fa-3cd01fa66bf5 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.433406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.433406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.436632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.436754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.439640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.444671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.447026 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3035/6071] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.451119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.469531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b6109-9c3e-4c34-bec1-7794dd1c987c tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.488840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.489512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.489813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.490174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.490687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.492311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.497277 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3037/6072] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.498601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.520486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad818e4d-d9eb-4431-b799-b73780bfd361 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.540266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.540936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.541201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.541481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.542702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.544316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.548490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3036/6073] 10.222.0.33 () {58 vars in 1212 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.549709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.562401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed0f22-47ee-41ec-98f9-99fe1a1984fe tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.576075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.576869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.577158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.577539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.578167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.582808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.587308 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3038/6074] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.589894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.616318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed053c-52d5-4794-bce8-fc8876436f25 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.632309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.642761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.642761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.642761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.642761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.643763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.643763 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3037/6075] 10.222.0.33 () {58 vars in 1215 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.643978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.658088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1c307-9e9c-4027-be9c-cc63e927fc81 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.671235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.671964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.672249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.672614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.673370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.675890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.681076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.689115 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3039/6076] 10.222.0.33 () {58 vars in 1218 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.701405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9388b48-da23-43f8-98c4-9857c372f169 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.725130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.725946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.726238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.726575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.727127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.729953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.731770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3038/6077] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.735817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.748351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120eb5-dc90-4336-9019-c6c8b8553cfb tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.772131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.774298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.775964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.781319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.784727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.789714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.799642 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3040/6078] 10.222.0.33 () {58 vars in 1239 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.800414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.826106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ff22a-675c-4179-b379-657c57753a92 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.837070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.837862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.838188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.838634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.839586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.841249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.848768 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3039/6079] 10.222.0.33 () {58 vars in 1260 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.852266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.888376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebeb2d6-a150-4edb-be32-912eacd1ecb0 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.907377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.909250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.909250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.909250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.909624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.912092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.918512 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3041/6080] 10.222.0.33 () {58 vars in 1263 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.919560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.939265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bce59-1817-4226-bbd4-e95cdba8a28a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.946677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.950367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.950828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:16.951132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:16.952134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:16.955567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:16.965100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:16.971625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3040/6081] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:16.977421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cdebb-0506-4505-b5c4-ba824c739859 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:16.997298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:16.998711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:16.999024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:17.001535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:17.002332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:17.009568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:17.020194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:17.023706 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3042/6082] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:16 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:17.064488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db871ba6-1389-4726-a32d-db8b82af6d80 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:17.072241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:17.074086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:17.074440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:17.074961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:17.075863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:17.077645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:17.086365 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3041/6083] 10.222.0.33 () {58 vars in 1278 bytes} [Sat Sep 10 21:13:17 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 10 21:13:17.091969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:17.116481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb85b7-929b-4831-825a-1ecdaf8a384d tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:17.130112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:17.130855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:17.131208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:17.133860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/f12df48146884e84b0a9fb83ea250517` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:17.167046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Could not find domain: f12df48146884e84b0a9fb83ea250517.: keystone.exception.DomainNotFound: Could not find domain: f12df48146884e84b0a9fb83ea250517. Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f12df48146884e84b0a9fb83ea250517. Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:17.180354 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:17.181754 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:13:17.183550 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f12df48146884e84b0a9fb83ea250517. Sep 10 21:13:17.184971 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:17.192484 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3043/6084] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:17 2022] PATCH /identity/v3/domains/f12df48146884e84b0a9fb83ea250517 => generated 112 bytes in 100 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:17.192803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:17.210594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630c7c4-709f-4d13-bb84-8958de4fb7ff tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:17.224636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:17.226006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:17.227250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:17.227643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/c8d882f2baa144c898572b405c07ad55` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:17.248333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Could not find domain: c8d882f2baa144c898572b405c07ad55.: keystone.exception.DomainNotFound: Could not find domain: c8d882f2baa144c898572b405c07ad55. Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c8d882f2baa144c898572b405c07ad55. Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:17.262518 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:17.263979 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:13:17.265432 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c8d882f2baa144c898572b405c07ad55. Sep 10 21:13:17.266933 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:17.270104 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3042/6085] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:17 2022] PATCH /identity/v3/domains/c8d882f2baa144c898572b405c07ad55 => generated 112 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:13:17.275214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:17.288379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723692a-7588-4e38-9754-ccdac19d8a3a tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:17.302410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7c375-4eb5-4516-b52b-83cbda8c09ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fae5192ae84cd3acf56c4a9d753634', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4289920029f34ed9baa14d34dc3b1c30', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:17.303066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c375-4eb5-4516-b52b-83cbda8c09ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:17.303357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c375-4eb5-4516-b52b-83cbda8c09ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:17.303638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c375-4eb5-4516-b52b-83cbda8c09ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] PATH_INFO: `/v3/domains/8543b1b9388b43708898eb1fc99a8943` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:17.343535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7c375-4eb5-4516-b52b-83cbda8c09ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-fde7c375-4eb5-4516-b52b-83cbda8c09ce tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Could not find domain: 8543b1b9388b43708898eb1fc99a8943.: keystone.exception.DomainNotFound: Could not find domain: 8543b1b9388b43708898eb1fc99a8943. Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 8543b1b9388b43708898eb1fc99a8943. Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:17.350712 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:17.352185 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:13:17.353499 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 8543b1b9388b43708898eb1fc99a8943. Sep 10 21:13:17.355836 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:17.385594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:17.386128 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3044/6086] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:17 2022] PATCH /identity/v3/domains/8543b1b9388b43708898eb1fc99a8943 => generated 112 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:17.387636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:17.636098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68dd0f-f9bb-4ae8-8237-6332275825d9 tempest-DomainConfigurationTestJSON-313449088 tempest-DomainConfigurationTestJSON-313449088-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:17.643237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:17.644561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:17.645097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:17.645601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] PATH_INFO: `/v3/users/872de0d813a1487d9e343c8c328bca05` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:17.647275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=872de0d813a1487d9e343c8c328bca05)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:17.649494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:17.716840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:17.776993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872de0d813a1487d9e343c8c328bca05'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:17.803758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872de0d813a1487d9e343c8c328bca05'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:17.833014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872de0d813a1487d9e343c8c328bca05'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:17.867321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872de0d813a1487d9e343c8c328bca05'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:17.868934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872de0d813a1487d9e343c8c328bca05'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:17.883676 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3043/6087] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:17 2022] DELETE /identity/v3/users/872de0d813a1487d9e343c8c328bca05 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:17.891405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d682fd-de4b-40ec-9dd9-3a949eda5131 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:17.891756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d682fd-de4b-40ec-9dd9-3a949eda5131 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:17.892131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d682fd-de4b-40ec-9dd9-3a949eda5131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:17.948661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-70d682fd-de4b-40ec-9dd9-3a949eda5131 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:17.950162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70d682fd-de4b-40ec-9dd9-3a949eda5131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:18.117470 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3045/6088] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:18.177045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:18.180647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:18.255996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:18.257170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:18.257842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:18.316460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] MFA Rules not processed for user `d83b6ceb599746b394e379d52683b70c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:18.318402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:18.391876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6de68-faf2-401b-a97a-873f01b0e536 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:18.404959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:18.405975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:18.406515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:18.406962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:18.428213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:18.443183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:18.486676 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3046/6089] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:18.499215 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3044/6090] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:19.984684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:19.986534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:20.003578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fdd91-59f9-4e6e-a86e-10ae91e9b51d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:20.010747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:20.011666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:20.012169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:20.012646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] PATH_INFO: `/v3/projects/6ca520e0f2f847e9b12523a7dbebc13c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:20.014198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca520e0f2f847e9b12523a7dbebc13c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:20.017362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:20.034894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:20.085617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca520e0f2f847e9b12523a7dbebc13c', 'request_id': 'req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.094636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca520e0f2f847e9b12523a7dbebc13c', 'request_id': 'req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.103541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca520e0f2f847e9b12523a7dbebc13c', 'request_id': 'req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.117581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] The token cache is being invalidate because project 6ca520e0f2f847e9b12523a7dbebc13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:20.117930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.118730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.120750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3047/6091] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:19 2022] DELETE /identity/v3/projects/6ca520e0f2f847e9b12523a7dbebc13c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:20.125347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:20.137025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c994-3ba9-412c-8642-2da0edd7dc2c service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:20.201449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:20.202239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:20.202527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:20.202800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] PATH_INFO: `/v3/users/34fae5192ae84cd3acf56c4a9d753634` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:20.203950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34fae5192ae84cd3acf56c4a9d753634)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:20.205905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:20.268979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34fae5192ae84cd3acf56c4a9d753634'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.270267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:20.285150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34fae5192ae84cd3acf56c4a9d753634'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.293450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34fae5192ae84cd3acf56c4a9d753634'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.301741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34fae5192ae84cd3acf56c4a9d753634'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.302525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a68dc1c-25dc-4562-8ec6-1aa06d2cb407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34fae5192ae84cd3acf56c4a9d753634'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.311442 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3045/6092] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:20 2022] DELETE /identity/v3/users/34fae5192ae84cd3acf56c4a9d753634 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:20.641696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:20.643777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:20.689578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17406f-34d0-4c73-a1b3-bbbfab481cc5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:20.771713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:20.772359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:20.772650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:20.772928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] PATH_INFO: `/v3/projects/4289920029f34ed9baa14d34dc3b1c30` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:20.774685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4289920029f34ed9baa14d34dc3b1c30)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:20.777678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:20.831462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4289920029f34ed9baa14d34dc3b1c30', 'request_id': 'req-878f7603-4cd3-4103-9ca9-e991709885c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.838041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4289920029f34ed9baa14d34dc3b1c30', 'request_id': 'req-878f7603-4cd3-4103-9ca9-e991709885c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.849747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4289920029f34ed9baa14d34dc3b1c30', 'request_id': 'req-878f7603-4cd3-4103-9ca9-e991709885c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.874131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] The token cache is being invalidate because project 4289920029f34ed9baa14d34dc3b1c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:20.874516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.875216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-878f7603-4cd3-4103-9ca9-e991709885c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:20.876802 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3048/6093] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:20 2022] DELETE /identity/v3/projects/4289920029f34ed9baa14d34dc3b1c30 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:20.882308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201000d8-fecc-43fc-9e1a-8fbf10ac1d46 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:20.882603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201000d8-fecc-43fc-9e1a-8fbf10ac1d46 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:20.882885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201000d8-fecc-43fc-9e1a-8fbf10ac1d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:20.907768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-201000d8-fecc-43fc-9e1a-8fbf10ac1d46 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:20.909563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-201000d8-fecc-43fc-9e1a-8fbf10ac1d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:20.984474 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3046/6094] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:20.992439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdaca3-8716-479c-9deb-d1bb56d1f34e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:20.992930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdaca3-8716-479c-9deb-d1bb56d1f34e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:20.993032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdaca3-8716-479c-9deb-d1bb56d1f34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.015308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-07cdaca3-8716-479c-9deb-d1bb56d1f34e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:21.016530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-07cdaca3-8716-479c-9deb-d1bb56d1f34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:21.024646 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3049/6095] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:21.030361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201000d8-fecc-43fc-9e1a-8fbf10ac1d46 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.040566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201000d8-fecc-43fc-9e1a-8fbf10ac1d46 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.045419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.046153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.046500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.046869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.047753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:21.048142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.049796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.066077 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3047/6096] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:13:21.068531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdaca3-8716-479c-9deb-d1bb56d1f34e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.080681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdaca3-8716-479c-9deb-d1bb56d1f34e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.101292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.104927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.105293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.105577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.106630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.111394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.164306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:21.172795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3050/6097] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:21 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:21.176778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.189186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91c691-2e78-43de-9b83-abb5ff2db8cb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.260483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.262264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.262867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.263142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.263948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.265921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.294933 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3048/6098] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:21 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:21.299438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.310557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba52ac8-04c6-459c-a582-dc2837559218 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.315575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.316261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.316550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.316825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.317461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.319343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.330076 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3051/6099] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:21.334131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.345122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c302410-cb41-4821-b9eb-0d8bb6e786b1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.349798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6c98c-4dfe-4a79-82ad-3aee1a924c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.350454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c98c-4dfe-4a79-82ad-3aee1a924c5b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.350723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c98c-4dfe-4a79-82ad-3aee1a924c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.350994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6c98c-4dfe-4a79-82ad-3aee1a924c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.351681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6c98c-4dfe-4a79-82ad-3aee1a924c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.353230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6c98c-4dfe-4a79-82ad-3aee1a924c5b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.365477 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3049/6100] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:21.369948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.380820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0ea22-0b20-48f2-b59d-e5b0fa549400 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.391068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.391976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.392265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.392539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] PATH_INFO: `/v3/projects/a41d0de701514bf2811fafef020be4c3/users/61551634eb934557b3ace3f726268a53/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.413413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41d0de701514bf2811fafef020be4c3, user_id=61551634eb934557b3ace3f726268a53, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.416109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.430483 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3052/6101] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:21 2022] PUT /identity/v3/projects/a41d0de701514bf2811fafef020be4c3/users/61551634eb934557b3ace3f726268a53/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:21.442856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c27b6-fd4c-4d0a-bcda-59d0d4d22259 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.443201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c27b6-fd4c-4d0a-bcda-59d0d4d22259 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.443474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c27b6-fd4c-4d0a-bcda-59d0d4d22259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.474431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-132c27b6-fd4c-4d0a-bcda-59d0d4d22259 None None] MFA Rules not processed for user `61551634eb934557b3ace3f726268a53`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:21.475647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-132c27b6-fd4c-4d0a-bcda-59d0d4d22259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:21.496449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:21.549587 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3050/6102] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:21.554786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.621947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e891d92-efa9-4fd0-b553-3b8f0e523c17 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.627091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.627747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.628337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.628649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.629327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.631519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.660252 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3053/6103] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:21 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:21.664289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c27b6-fd4c-4d0a-bcda-59d0d4d22259 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.675724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c27b6-fd4c-4d0a-bcda-59d0d4d22259 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.742785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.743404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.743690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.743967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.744658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.747255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.770329 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3051/6104] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:21 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:21.774030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.785054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42716079-a647-433a-a0ec-306210d012e9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.791014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.791014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.791014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.792013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.793095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.794584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.809715 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3054/6105] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:21.814233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.825850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bde50-3c5e-44a9-a2ec-88eb95b9e4d4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.831289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.832162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.832606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.832973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] PATH_INFO: `/v3/projects/009b708b8b564746b2441387aeaf26aa/users/4185eb3cfc0c40d2a197bb053efca252/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.853379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=009b708b8b564746b2441387aeaf26aa, user_id=4185eb3cfc0c40d2a197bb053efca252, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.857273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:21.885632 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3052/6106] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:21 2022] PUT /identity/v3/projects/009b708b8b564746b2441387aeaf26aa/users/4185eb3cfc0c40d2a197bb053efca252/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:21.889877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:21.901703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe56f0-4cbf-4e4e-b42b-9696ba917b6e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:21.988864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935326e4-da32-447a-b804-693bd8eaa2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:21.989496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935326e4-da32-447a-b804-693bd8eaa2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:21.989784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935326e4-da32-447a-b804-693bd8eaa2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:21.990153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935326e4-da32-447a-b804-693bd8eaa2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:21.990736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935326e4-da32-447a-b804-693bd8eaa2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:21.993757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935326e4-da32-447a-b804-693bd8eaa2a3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.017644 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3055/6107] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:22.022481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.037333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0b3c6-cc52-49f0-8e13-98de33feda9d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.046824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.049982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.050343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.050751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] PATH_INFO: `/v3/projects/009b708b8b564746b2441387aeaf26aa/users/4185eb3cfc0c40d2a197bb053efca252/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.053404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=009b708b8b564746b2441387aeaf26aa, user_id=4185eb3cfc0c40d2a197bb053efca252, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.060085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.075732 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3053/6108] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:22 2022] PUT /identity/v3/projects/009b708b8b564746b2441387aeaf26aa/users/4185eb3cfc0c40d2a197bb053efca252/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:22.086136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ef2a-30c6-4b19-87e8-d101f91948a3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.086407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ef2a-30c6-4b19-87e8-d101f91948a3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.086708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ef2a-30c6-4b19-87e8-d101f91948a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.134800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5d85ef2a-30c6-4b19-87e8-d101f91948a3 None None] MFA Rules not processed for user `4185eb3cfc0c40d2a197bb053efca252`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:22.136801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5d85ef2a-30c6-4b19-87e8-d101f91948a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:22.191441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:22.268104 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3056/6109] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:22.272362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.285381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07176e6b-111d-462e-9f09-ce50f2d2eafd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.292280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.293024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.293414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.293792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.294594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.295714 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:22.295714 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:22.295714 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:22.296906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.320418 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3054/6110] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:22 2022] POST /identity/v3/services => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:22.325860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85ef2a-30c6-4b19-87e8-d101f91948a3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.336583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85ef2a-30c6-4b19-87e8-d101f91948a3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.357629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.358471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.358867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.359250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.360144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.361849 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:22.361849 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:22.361849 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:22.374963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: On instance['enabled']: Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: On instance['enabled']: Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: 'False' Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise error Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application 'False' Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.388028 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:13:22.389918 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application 'False' Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.396703 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3057/6111] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:13:22 2022] POST /identity/v3/endpoints => generated 283 bytes in 66 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 10 21:13:22.397679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.423582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dad82-bad2-451c-970b-5a88f6cf77ba tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.432774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.433546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.433974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.434353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.435142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.436285 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:22.436285 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:22.436285 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:22.437445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: On instance['enabled']: Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: On instance['enabled']: Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: 'True' Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise error Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application 'True' Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.445426 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:13:22.447258 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application 'True' Sep 10 21:13:22.450617 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.454077 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3055/6112] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:13:22 2022] POST /identity/v3/endpoints => generated 281 bytes in 58 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 10 21:13:22.458919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.472820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ae679-a9b4-4aa2-a800-e48bff0ec432 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.477850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.478612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.480325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.480612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.481310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.482986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.503232 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1142923893. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:13:22.503232 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:13:22.503232 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:22.536076 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3058/6113] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:13:22 2022] POST /identity/v3/endpoints => generated 361 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:22.540144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.552682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67425207-2d4a-4d88-8972-2fc03363abbf tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.570722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.571721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.572109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.572620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/regions/tempest-region-1142923893` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.573535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1142923893)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.575047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.577426 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3056/6114] 10.222.0.33 () {58 vars in 1206 bytes} [Sat Sep 10 21:13:22 2022] GET /identity/v3/regions/tempest-region-1142923893 => generated 180 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:22.582789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.603102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc14c5e-31cb-4134-a4e8-4d6f823cdb20 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.610843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.611494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.611780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.612057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints/1b6416cf002248e1afd1c8bf6b9528b1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.620721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=1b6416cf002248e1afd1c8bf6b9528b1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.622144 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:22.622144 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:22.622144 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:22.624141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: On instance['enabled']: Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: On instance['enabled']: Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: 'False' Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise error Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application 'False' Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.631742 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:13:22.634184 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application 'False' Sep 10 21:13:22.635642 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:22.637725 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3059/6115] 10.222.0.33 () {60 vars in 1255 bytes} [Sat Sep 10 21:13:22 2022] PATCH /identity/v3/endpoints/1b6416cf002248e1afd1c8bf6b9528b1 => generated 283 bytes in 55 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 10 21:13:22.640240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.653054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe80a49-bbed-4bae-9967-0082bc43c8de tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.659728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.660570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.660930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.661958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints/1b6416cf002248e1afd1c8bf6b9528b1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.664117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=1b6416cf002248e1afd1c8bf6b9528b1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.665724 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:22.665724 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:22.665724 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:22.668304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.680281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '1b6416cf002248e1afd1c8bf6b9528b1', 'request_id': 'req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:22.692602 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3057/6116] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:13:22 2022] DELETE /identity/v3/endpoints/1b6416cf002248e1afd1c8bf6b9528b1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:22.697337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.708597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d4a7c-e1c8-4e20-b031-2d969590cb78 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.733243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.734063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.734423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.734773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/regions/tempest-region-1142923893` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.735437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1142923893)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.737204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.773142 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3060/6117] 10.222.0.33 () {58 vars in 1209 bytes} [Sat Sep 10 21:13:22 2022] DELETE /identity/v3/regions/tempest-region-1142923893 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:22.779266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.790613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9685f97-7d79-4d7c-bff9-cf85ebaa9c05 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.814701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.815317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.815827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.816098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.816779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.818585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.829251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:22.842396 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-215660584. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:13:22.842396 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:13:22.842396 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:22.876858 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3058/6118] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:13:22 2022] POST /identity/v3/endpoints => generated 358 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:22.878113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.889238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcfec1-601e-4f8d-b176-c3c3107b0f53 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.905904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.906579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.906926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.907265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/regions/tempest-region-215660584` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.907919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-215660584)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.909050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.913936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3061/6119] 10.222.0.33 () {58 vars in 1203 bytes} [Sat Sep 10 21:13:22 2022] GET /identity/v3/regions/tempest-region-215660584 => generated 178 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:22.915425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.927087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c32a5-c1a6-4443-a19e-7a4f68dce31e tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.932791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.933616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.934077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.934369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints/b7c96d4255604db88d14c4d1895d4d48` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.940582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=b7c96d4255604db88d14c4d1895d4d48)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.941737 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:22.941737 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:22.941737 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:22.942719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: On instance['enabled']: Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: On instance['enabled']: Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: 'True' Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise error Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application 'True' Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.952818 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:13:22.954786 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['enabled']: Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application 'True' Sep 10 21:13:22.955981 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:22.957764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3059/6120] 10.222.0.33 () {60 vars in 1255 bytes} [Sat Sep 10 21:13:22 2022] PATCH /identity/v3/endpoints/b7c96d4255604db88d14c4d1895d4d48 => generated 281 bytes in 42 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 10 21:13:22.958328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:22.977905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41debf9-7995-4666-9821-4b79bb5ced9c tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:22.985047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:22.985691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:22.986014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:22.986289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/endpoints/b7c96d4255604db88d14c4d1895d4d48` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:22.987444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=b7c96d4255604db88d14c4d1895d4d48)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:22.988996 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:22.988996 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:22.988996 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:22.989853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:23.002131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'b7c96d4255604db88d14c4d1895d4d48', 'request_id': 'req-d90a9e8c-9638-4129-891f-dadb26073cb3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:23.014260 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3062/6121] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:13:22 2022] DELETE /identity/v3/endpoints/b7c96d4255604db88d14c4d1895d4d48 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:23.018593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:23.030315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97aaaaf-0571-4b5e-b308-85c3b204d483 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:23.058966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:23.059778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:23.060191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:23.060542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/regions/tempest-region-215660584` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:23.061198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-215660584)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:23.063110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:23.085493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3060/6122] 10.222.0.33 () {58 vars in 1206 bytes} [Sat Sep 10 21:13:23 2022] DELETE /identity/v3/regions/tempest-region-215660584 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:23.093814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:23.107484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a9e8c-9638-4129-891f-dadb26073cb3 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:23.122913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990c229-3aef-4cae-9309-fadd1773f3a9 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4185eb3cfc0c40d2a197bb053efca252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '009b708b8b564746b2441387aeaf26aa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:23.123546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990c229-3aef-4cae-9309-fadd1773f3a9 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:23.123953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990c229-3aef-4cae-9309-fadd1773f3a9 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:23.124231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990c229-3aef-4cae-9309-fadd1773f3a9 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] PATH_INFO: `/v3/services/b4a54b4ccdac41c3b034c547bb18923a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:23.124808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990c229-3aef-4cae-9309-fadd1773f3a9 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorizing `identity:delete_service(service_id=b4a54b4ccdac41c3b034c547bb18923a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:23.125857 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-6990c229-3aef-4cae-9309-fadd1773f3a9 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:23.125857 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:23.125857 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:23.126786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990c229-3aef-4cae-9309-fadd1773f3a9 tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:23.154504 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3063/6123] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:23 2022] DELETE /identity/v3/services/b4a54b4ccdac41c3b034c547bb18923a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:23.158651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:23.242038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d3ab2-164e-4f88-b40b-ed31f041315f tempest-EndpointsNegativeTestJSON-887472492 tempest-EndpointsNegativeTestJSON-887472492-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:23.262899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:23.263548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:23.263889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:23.264166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] PATH_INFO: `/v3/users/61551634eb934557b3ace3f726268a53` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:23.265316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61551634eb934557b3ace3f726268a53)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:23.267629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:23.321215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61551634eb934557b3ace3f726268a53'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:23.334113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61551634eb934557b3ace3f726268a53'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:23.341995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61551634eb934557b3ace3f726268a53'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:23.349362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61551634eb934557b3ace3f726268a53'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:23.350036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61551634eb934557b3ace3f726268a53'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:23.352563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:23.358185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3061/6124] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:23 2022] DELETE /identity/v3/users/61551634eb934557b3ace3f726268a53 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:23.362989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:23.363335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:23.363610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:23.384821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:23.386100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:23.405445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:23.457698 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3064/6125] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:23.486685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:23.488194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:23.567512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca6e8f-8007-429f-98a2-5bf38d6587c1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:23.572146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:23.573102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:23.573478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:23.573783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:23.583804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:23.585240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:23.597217 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3062/6126] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:23.913205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:23.916007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:23.932032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e72c0-1434-4abc-b451-3207ea9ed6be None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:23.937465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:23.938332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:23.938698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:23.939048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] PATH_INFO: `/v3/projects/a41d0de701514bf2811fafef020be4c3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:23.940323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41d0de701514bf2811fafef020be4c3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:23.942369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:24.024569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41d0de701514bf2811fafef020be4c3', 'request_id': 'req-51b7d3e9-9240-4270-90c5-c7029c577366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.036604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41d0de701514bf2811fafef020be4c3', 'request_id': 'req-51b7d3e9-9240-4270-90c5-c7029c577366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.046115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41d0de701514bf2811fafef020be4c3', 'request_id': 'req-51b7d3e9-9240-4270-90c5-c7029c577366'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.059203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] The token cache is being invalidate because project a41d0de701514bf2811fafef020be4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:24.059567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.060147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.063417 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3065/6127] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:23 2022] DELETE /identity/v3/projects/a41d0de701514bf2811fafef020be4c3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:24.068328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:24.081384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373045a-c15b-49e0-a88a-4a5fea62beef service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:24.145426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:24.146104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:24.146387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:24.146662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] PATH_INFO: `/v3/users/4185eb3cfc0c40d2a197bb053efca252` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:24.147791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4185eb3cfc0c40d2a197bb053efca252)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:24.149761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:24.198323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4185eb3cfc0c40d2a197bb053efca252'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.210819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4185eb3cfc0c40d2a197bb053efca252'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.217747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4185eb3cfc0c40d2a197bb053efca252'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.225459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4185eb3cfc0c40d2a197bb053efca252'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.226108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6418003a-20f4-4faa-a593-2d8e16f621f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4185eb3cfc0c40d2a197bb053efca252'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.233109 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3063/6128] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:24 2022] DELETE /identity/v3/users/4185eb3cfc0c40d2a197bb053efca252 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:24.534628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:24.536254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:24.563773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7d3e9-9240-4270-90c5-c7029c577366 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:24.655913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:24.656556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:24.656838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:24.657161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] PATH_INFO: `/v3/projects/009b708b8b564746b2441387aeaf26aa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:24.658350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009b708b8b564746b2441387aeaf26aa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:24.660256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:24.708575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:24.710649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009b708b8b564746b2441387aeaf26aa', 'request_id': 'req-eab237b6-68a7-4b14-a137-a8ad341f14a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.720060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009b708b8b564746b2441387aeaf26aa', 'request_id': 'req-eab237b6-68a7-4b14-a137-a8ad341f14a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.728272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009b708b8b564746b2441387aeaf26aa', 'request_id': 'req-eab237b6-68a7-4b14-a137-a8ad341f14a1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.742338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] The token cache is being invalidate because project 009b708b8b564746b2441387aeaf26aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:24.742634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.743421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eab237b6-68a7-4b14-a137-a8ad341f14a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:24.744946 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3066/6129] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:24 2022] DELETE /identity/v3/projects/009b708b8b564746b2441387aeaf26aa => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:24.749910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2dc94-71a5-48d7-86e7-8808ea084db7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:24.750431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2dc94-71a5-48d7-86e7-8808ea084db7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:24.750532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2dc94-71a5-48d7-86e7-8808ea084db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:24.771752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cdd2dc94-71a5-48d7-86e7-8808ea084db7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:24.772951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cdd2dc94-71a5-48d7-86e7-8808ea084db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:24.782566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:24.845349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3064/6130] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:24.854570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e00b6-846b-4514-9e80-bcb88ffc261b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:24.855671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e00b6-846b-4514-9e80-bcb88ffc261b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:24.855946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e00b6-846b-4514-9e80-bcb88ffc261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:24.886948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-230e00b6-846b-4514-9e80-bcb88ffc261b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:24.888151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-230e00b6-846b-4514-9e80-bcb88ffc261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:24.896032 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3067/6131] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:24.900465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2dc94-71a5-48d7-86e7-8808ea084db7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:24.911128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2dc94-71a5-48d7-86e7-8808ea084db7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:24.915817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:24.916499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:24.917415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:24.917994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:24.918636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:24.918942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:24.920529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:24.936945 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3065/6132] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:24.938195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e00b6-846b-4514-9e80-bcb88ffc261b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:24.948368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e00b6-846b-4514-9e80-bcb88ffc261b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:24.953402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:24.954056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:24.954369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:24.954644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:24.955400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:24.957319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:24.992585 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3068/6133] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:24 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:24.993558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.005583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83135c-319c-4c11-9880-f57bc51ab238 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.064303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.064910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.065196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.065476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.066136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.068104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.094185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3066/6134] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:24 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:25.095692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.106713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925fc6f-3192-42f8-9858-c64068a6f4c9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.111541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.112147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.112844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.112844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.113333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.114969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.126000 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3069/6135] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:25.130131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.141264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410fcd5-13e7-4bd5-b743-97a4965f2fb8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.146190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ddadf-e362-4e2b-bb5f-00e857e91e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.147352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ddadf-e362-4e2b-bb5f-00e857e91e63 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.147757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ddadf-e362-4e2b-bb5f-00e857e91e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.148121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ddadf-e362-4e2b-bb5f-00e857e91e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.148685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ddadf-e362-4e2b-bb5f-00e857e91e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.150328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ddadf-e362-4e2b-bb5f-00e857e91e63 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.161928 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3067/6136] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:13:25.165708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.179889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0ac02-f4af-4200-a939-c9e95e038f0e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.185899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.186616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.186911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.187364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] PATH_INFO: `/v3/projects/5caea13722034559ac5318ed37296e84/users/093bfd9ca8a9453fb05e332ca8c38729/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.206635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5caea13722034559ac5318ed37296e84, user_id=093bfd9ca8a9453fb05e332ca8c38729, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.209526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.225907 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3070/6137] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:25 2022] PUT /identity/v3/projects/5caea13722034559ac5318ed37296e84/users/093bfd9ca8a9453fb05e332ca8c38729/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:25.238637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6406b4-7a17-4735-8a0f-ac39f764db1b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.238859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6406b4-7a17-4735-8a0f-ac39f764db1b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.239131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6406b4-7a17-4735-8a0f-ac39f764db1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.268744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2e6406b4-7a17-4735-8a0f-ac39f764db1b None None] MFA Rules not processed for user `093bfd9ca8a9453fb05e332ca8c38729`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:25.269996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2e6406b4-7a17-4735-8a0f-ac39f764db1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:25.336445 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3068/6138] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:25.341767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.404341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3474-0439-44c9-b5db-e6e282e5b9d1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.408954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.409542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.409846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.410120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.410732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.412618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.439233 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3071/6139] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:25 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:25.443728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6406b4-7a17-4735-8a0f-ac39f764db1b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.455061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6406b4-7a17-4735-8a0f-ac39f764db1b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.459793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:25.512728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.513262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.513544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.513820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.514483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.516441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.538035 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3069/6140] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:25 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:25.542168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.552938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee73de6-9a6e-4579-9010-9b419b6fd6bf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.559739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.560461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.560746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.561023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.561669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.563310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.574893 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3072/6141] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:25.579339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.590698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9c016-ef12-47a9-b75b-9dbccfbe9a40 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.595634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.596301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.596619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.596884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] PATH_INFO: `/v3/projects/06a87748e68c42489e49214189197a0b/users/542101eb7a3e42be99c101a4d5230701/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.611963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a87748e68c42489e49214189197a0b, user_id=542101eb7a3e42be99c101a4d5230701, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.614688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.630122 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3070/6142] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:25 2022] PUT /identity/v3/projects/06a87748e68c42489e49214189197a0b/users/542101eb7a3e42be99c101a4d5230701/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:25.634020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.644855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ed11-07c3-484a-86f8-1a41cbc2b329 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.690002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:25.706276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd64a-ca8b-4fab-a51b-43cdc712e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.706915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd64a-ca8b-4fab-a51b-43cdc712e412 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.707335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd64a-ca8b-4fab-a51b-43cdc712e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.707611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd64a-ca8b-4fab-a51b-43cdc712e412 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.708250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd64a-ca8b-4fab-a51b-43cdc712e412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.709865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd64a-ca8b-4fab-a51b-43cdc712e412 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.720025 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3073/6143] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:13:25.724143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.735732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a6080-acc0-4d66-a862-f107bdc0fedc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.740833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.741573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.741976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.742325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] PATH_INFO: `/v3/projects/06a87748e68c42489e49214189197a0b/users/542101eb7a3e42be99c101a4d5230701/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.745339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a87748e68c42489e49214189197a0b, user_id=542101eb7a3e42be99c101a4d5230701, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.748118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.762493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3071/6144] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:25 2022] PUT /identity/v3/projects/06a87748e68c42489e49214189197a0b/users/542101eb7a3e42be99c101a4d5230701/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:25.773474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9cd2a-8c14-4347-8fe3-6b229ea6f8b9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.773759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9cd2a-8c14-4347-8fe3-6b229ea6f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.774010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9cd2a-8c14-4347-8fe3-6b229ea6f8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.808338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c5d9cd2a-8c14-4347-8fe3-6b229ea6f8b9 None None] MFA Rules not processed for user `542101eb7a3e42be99c101a4d5230701`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:25.809813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c5d9cd2a-8c14-4347-8fe3-6b229ea6f8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:25.889554 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3074/6145] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:25.891106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.903791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d7664-09b8-46b4-80b9-a94c58ad5ddf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.908510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.909119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.909409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.909691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.910728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.912079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.931568 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3072/6146] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:25 2022] POST /identity/v3/roles => generated 235 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:25.935977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9cd2a-8c14-4347-8fe3-6b229ea6f8b9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:25.950223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9cd2a-8c14-4347-8fe3-6b229ea6f8b9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:25.955076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:25.955698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:25.955989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:25.956272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:25.956939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:25.958703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:25.996188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3075/6147] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:25 2022] POST /identity/v3/roles => generated 235 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:26.000314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.014106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cca0a-e2ee-4ec7-b477-ba04b79dcce2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.019348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.019991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.020273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.020548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.021217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.023026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.039434 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3073/6148] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:25 2022] POST /identity/v3/roles => generated 234 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:26.044147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.054267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b15fca-099b-408d-b65e-11e611b9292f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.059115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.059713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.059997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.060274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.060937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.062992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.080012 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3076/6149] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:13:26 2022] POST /identity/v3/domains => generated 276 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:26.084021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.095294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385020f7-edba-4907-ad6a-1281d7aaec5b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.151970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.152573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.152856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.153129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.153756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.155676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.195943 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3074/6150] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:26 2022] POST /identity/v3/projects => generated 398 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:26.197367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.208332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c181-7b19-43cc-8673-b0b8ff7bbf48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.268320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.268936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.269289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.269580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.270259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.272173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.290367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3077/6151] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:13:26 2022] POST /identity/v3/groups => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:26.292448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.303644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32220819-e877-4c34-80b0-ba3223308962 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.309238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.309873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.310168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.310521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.311260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.313050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.322755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:26.331772 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3075/6152] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:26 2022] POST /identity/v3/roles => generated 234 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:26.335864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.347646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28991d-db22-4ad5-8759-c3aed4272af3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.353277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.354896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.355271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.355650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.356454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.358685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.382790 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3078/6153] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:26 2022] POST /identity/v3/users => generated 436 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:26.388000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.406279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa1f5f-2d02-4a46-8d18-c8ea03407ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.416061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.417169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.417672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.418180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.419197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.420538 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:26.420538 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:26.420538 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:26.421893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.446925 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3076/6154] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:26 2022] POST /identity/v3/roles => generated 269 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:26.448590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.460703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44f89-1556-4a00-aeea-5fb829d65d91 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.466911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.467754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.468103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.468567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/d64734e59475489f91217bc5107bf590` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.492842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=d64734e59475489f91217bc5107bf590)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.496293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.498739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:26.510004 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3079/6155] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:26 2022] PUT /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/d64734e59475489f91217bc5107bf590 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:26.514343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.524423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19062d1-c452-430b-822c-0ab4d61161da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.584320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.584969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.585255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.585628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.586378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Adding query filter params (scope.project.id=d60a5156f2a14fe196bbfa4eaa948b4e, user.id=0e655e86152b4d719f43b75c4ffd4192) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:26.586686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.589484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.651760 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3077/6156] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:13:26 2022] GET /identity/v3/role_assignments?scope.project.id=d60a5156f2a14fe196bbfa4eaa948b4e&user.id=0e655e86152b4d719f43b75c4ffd4192&effective => generated 228 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:26.653069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.663749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835ef48-1ac5-4fc6-9a36-b8a1fb94c8e5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.669570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.670270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.670557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.670831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/d64734e59475489f91217bc5107bf590` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.673090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=d64734e59475489f91217bc5107bf590)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.675772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.683357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role d64734e59475489f91217bc5107bf590 was removed from user 0e655e86152b4d719f43b75c4ffd4192 on project d60a5156f2a14fe196bbfa4eaa948b4e. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:26.683653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:26.684507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:26.699914 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3080/6157] 10.222.0.33 () {58 vars in 1467 bytes} [Sat Sep 10 21:13:26 2022] DELETE /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/d64734e59475489f91217bc5107bf590 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:26.704125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.706019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:26.719882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48760e4-bf26-4279-b2c9-d460a5fa7487 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.802709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.803461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.803663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.803951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/d64734e59475489f91217bc5107bf590` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.805181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=d64734e59475489f91217bc5107bf590)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.806325 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:26.806325 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:26.806325 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:26.807277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.850094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role d64734e59475489f91217bc5107bf590 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:26.850405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:26.851112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:26.853180 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3078/6158] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:26 2022] DELETE /identity/v3/roles/d64734e59475489f91217bc5107bf590 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:26.858759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.869014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044b3aa-8a31-48b9-9064-1db46742b9b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.938041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:26.938704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:26.938984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:26.939259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/a29c1aaf5ae640c28acbb7270b9d0c9e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:26.941645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=a29c1aaf5ae640c28acbb7270b9d0c9e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:26.944406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:26.959046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3081/6159] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:26 2022] PUT /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/a29c1aaf5ae640c28acbb7270b9d0c9e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:26.963133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:26.965911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:26.982193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494a944-182e-4881-bd67-c99249ac54f1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:26.987151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:27.050545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.051218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.051507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.051781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.053555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.054601 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:27.054601 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:27.054601 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:27.055492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.082105 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3079/6160] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:26 2022] PUT /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 521 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:27.088959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.106088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e910be-6323-430f-9d47-2c751e9a767b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.181020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.181636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.182500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.182789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.183729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Adding query filter params (scope.project.id=d60a5156f2a14fe196bbfa4eaa948b4e, user.id=0e655e86152b4d719f43b75c4ffd4192) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:27.184042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.185966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.240827 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3082/6161] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:13:27 2022] GET /identity/v3/role_assignments?scope.project.id=d60a5156f2a14fe196bbfa4eaa948b4e&user.id=0e655e86152b4d719f43b75c4ffd4192&effective => generated 1061 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:27.246267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.266116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e9670-9650-4813-bf1c-81319e7dc605 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.271045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.271691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.271977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.272255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.273911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.274883 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:27.274883 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:27.274883 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:27.276760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.287129 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3080/6162] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:27 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:27.291211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.305472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c31e2-c6d4-4550-92b1-5f2407610f4f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.326857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:27.365233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.365770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.366088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.366371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.367129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Adding query filter params (scope.project.id=d60a5156f2a14fe196bbfa4eaa948b4e, user.id=0e655e86152b4d719f43b75c4ffd4192) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:27.367438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.369284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.414838 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3083/6163] 10.222.0.33 () {58 vars in 1356 bytes} [Sat Sep 10 21:13:27 2022] GET /identity/v3/role_assignments?scope.project.id=d60a5156f2a14fe196bbfa4eaa948b4e&user.id=0e655e86152b4d719f43b75c4ffd4192&effective => generated 577 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:27.419422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.429714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e970683-279d-4ef7-b58e-9354448b5a00 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.435004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.435640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.435929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.436232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.438145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.439807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197.: keystone.exception.ImpliedRoleNotFound: a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197. Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = query.one() Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._iter().one() Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._only_one_row( Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:27.456555 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 10 21:13:27.458002 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 10 21:13:27.459384 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 10 21:13:27.459384 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 10 21:13:27.459384 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 10 21:13:27.459384 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197. Sep 10 21:13:27.459384 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:27.459384 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3081/6164] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:27 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 137 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:27.461747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.472980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823ed8d-7465-41fe-bee3-c0828a82c7fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.477588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.478282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.478567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.479182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/a29c1aaf5ae640c28acbb7270b9d0c9e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.481649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=a29c1aaf5ae640c28acbb7270b9d0c9e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.484322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.489901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role a29c1aaf5ae640c28acbb7270b9d0c9e was removed from user 0e655e86152b4d719f43b75c4ffd4192 on project d60a5156f2a14fe196bbfa4eaa948b4e. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:27.490193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:27.490881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:27.507205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3084/6165] 10.222.0.33 () {58 vars in 1467 bytes} [Sat Sep 10 21:13:27 2022] DELETE /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/a29c1aaf5ae640c28acbb7270b9d0c9e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:27.513747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.515552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:27.529474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7cc9d-b2b0-4f45-83f0-5d7fda1e828c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.589378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.590030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.590316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.590592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.591253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.592852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.609247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3082/6166] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:27 2022] POST /identity/v3/roles => generated 272 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:27.613519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.624166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b4a12-da28-4f92-ae20-41ad0e87029a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.628750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.629347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.629625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.629930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.631203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Adding query filter params (domain_id=d16dea24deb34f61b91996ba38762e70) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:27.631517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.632533 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:27.632533 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:27.632533 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:27.633424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.647040 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3085/6167] 10.222.0.33 () {58 vars in 1207 bytes} [Sat Sep 10 21:13:27 2022] GET /identity/v3/roles?domain_id=d16dea24deb34f61b91996ba38762e70 => generated 409 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:27.651171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.661855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed1408-0b39-42b3-814c-260d844f57b4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.667306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.667934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.668220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.668498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/347a9c7ec15d40d1b021ec59b9edea41` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.669650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=347a9c7ec15d40d1b021ec59b9edea41)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.671310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.707040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 347a9c7ec15d40d1b021ec59b9edea41 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:27.707330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:27.708005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:27.709766 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3083/6168] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:27 2022] DELETE /identity/v3/roles/347a9c7ec15d40d1b021ec59b9edea41 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:27.714320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.716039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:27.729509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79b53d-4c4d-4a4a-9bd9-5baaedbe2679 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.788184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.788791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.789078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.789407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.790027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Adding query filter params (domain_id=d16dea24deb34f61b91996ba38762e70) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:27.790331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.792002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.797617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3086/6169] 10.222.0.33 () {58 vars in 1207 bytes} [Sat Sep 10 21:13:27 2022] GET /identity/v3/roles?domain_id=d16dea24deb34f61b91996ba38762e70 => generated 148 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:13:27.801629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.811698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a4c7f-e679-4f50-bb41-e9671a5c0055 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.816505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.817122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.817405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.817686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/347a9c7ec15d40d1b021ec59b9edea41` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.829740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Could not find role: 347a9c7ec15d40d1b021ec59b9edea41.: keystone.exception.RoleNotFound: Could not find role: 347a9c7ec15d40d1b021ec59b9edea41. Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Sep 10 21:13:27.834540 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:13:27.835923 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 347a9c7ec15d40d1b021ec59b9edea41. Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:27.837084 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3084/6170] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:27 2022] DELETE /identity/v3/roles/347a9c7ec15d40d1b021ec59b9edea41 => generated 110 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:27.841376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.851439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fac34a-d345-4f93-ba97-9d24a823645a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.856499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.857152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.857439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.857721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.864497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d16dea24deb34f61b91996ba38762e70, group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.867931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:27.883556 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3087/6171] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:27 2022] PUT /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:27.888391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:27.898821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2cda1-fc8a-4f0e-847b-9f05d79b0a9d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:27.905386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:27.976680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:27.977361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:27.977812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:27.978192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:27.981325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=d16dea24deb34f61b91996ba38762e70, group_id=3254e44563e644959f14d1af556eb91a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:27.983922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.001510 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3085/6172] 10.222.0.33 () {58 vars in 1365 bytes} [Sat Sep 10 21:13:27 2022] GET /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles => generated 409 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:13:28.003393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.016020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7234-4aa6-42ea-aade-2c3b606fb2ad tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.022531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.023753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.024208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.024643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.027980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=d16dea24deb34f61b91996ba38762e70, group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.029243 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:28.029243 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:28.029243 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:28.033231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.043848 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3088/6173] 10.222.0.33 () {58 vars in 1465 bytes} [Sat Sep 10 21:13:28 2022] HEAD /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:28.045090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.058682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24eaa-429f-4202-adf6-f2a9ab69b3d1 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.063879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.064899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.065194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.065523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.068060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d16dea24deb34f61b91996ba38762e70, group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.070869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.083174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 6eaa7d92604b403696be366d2a3f9ecd was removed from group 3254e44563e644959f14d1af556eb91a on domain d16dea24deb34f61b91996ba38762e70. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:28.083510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:28.084216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:28.104828 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3086/6174] 10.222.0.33 () {58 vars in 1467 bytes} [Sat Sep 10 21:13:28 2022] DELETE /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:28.110144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.112058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:28.126854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9a5f0-a965-477a-9483-eff9cf3675d5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.159517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:28.211976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.212650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.212956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.213282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.215788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.218728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.233694 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3089/6175] 10.222.0.33 () {60 vars in 1486 bytes} [Sat Sep 10 21:13:28 2022] PUT /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:28.238143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.249625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa659f-9407-4824-910b-ac797621f300 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.324839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.325623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.326206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.326575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.328775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_grants(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, group_id=3254e44563e644959f14d1af556eb91a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.333558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.342329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.344498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:28.363291 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3087/6176] 10.222.0.33 () {58 vars in 1368 bytes} [Sat Sep 10 21:13:28 2022] GET /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles => generated 410 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:28.367492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.404823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33eb23-e7d8-499b-9557-f405e7274258 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.413056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957177a5-8807-4061-93c4-5d32fe2f7c12 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.414141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957177a5-8807-4061-93c4-5d32fe2f7c12 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.414272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957177a5-8807-4061-93c4-5d32fe2f7c12 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.414557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957177a5-8807-4061-93c4-5d32fe2f7c12 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/groups/3254e44563e644959f14d1af556eb91a/users/0e655e86152b4d719f43b75c4ffd4192` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.417567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957177a5-8807-4061-93c4-5d32fe2f7c12 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=3254e44563e644959f14d1af556eb91a, user_id=0e655e86152b4d719f43b75c4ffd4192)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.420547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957177a5-8807-4061-93c4-5d32fe2f7c12 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.465952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3088/6177] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:13:28 2022] PUT /identity/v3/groups/3254e44563e644959f14d1af556eb91a/users/0e655e86152b4d719f43b75c4ffd4192 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:28.467525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867ad42-20fe-43f5-ad68-2bb3be6558d2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.467812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867ad42-20fe-43f5-ad68-2bb3be6558d2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.468088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867ad42-20fe-43f5-ad68-2bb3be6558d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.525001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4867ad42-20fe-43f5-ad68-2bb3be6558d2 None None] MFA Rules not processed for user `0e655e86152b4d719f43b75c4ffd4192`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:28.526552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4867ad42-20fe-43f5-ad68-2bb3be6558d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:28.528467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d70c1-9511-4389-84c6-1327f41ac7d0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.563809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:28.600511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.601121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.601402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.601678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.603503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:28.628825 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3089/6178] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3263 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:28.633435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867ad42-20fe-43f5-ad68-2bb3be6558d2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.652795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:28.670898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.673749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.696231 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3090/6179] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:13:28.705120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867ad42-20fe-43f5-ad68-2bb3be6558d2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.710223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.710900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.711202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.711454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.713942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:check_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.716617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.722884 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3090/6180] 10.222.0.33 () {58 vars in 1468 bytes} [Sat Sep 10 21:13:28 2022] HEAD /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:28.727034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.739190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a57475-483a-4f14-a031-152f56d9d8d8 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.748333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.749265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.749558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.749867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.752181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.755105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.761120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 6eaa7d92604b403696be366d2a3f9ecd was removed from group 3254e44563e644959f14d1af556eb91a on project d60a5156f2a14fe196bbfa4eaa948b4e. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:28.761865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:28.762163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:28.780568 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3091/6181] 10.222.0.33 () {58 vars in 1470 bytes} [Sat Sep 10 21:13:28 2022] DELETE /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:28.785642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.787934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:28.805456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7072c00-6b5a-4e8f-af43-e1ebd464f48e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:28.888858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:28.889550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:28.889780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:28.890082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/groups/3254e44563e644959f14d1af556eb91a/users/0e655e86152b4d719f43b75c4ffd4192` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:28.891753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=3254e44563e644959f14d1af556eb91a, user_id=0e655e86152b4d719f43b75c4ffd4192)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:28.892756 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:28.892756 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:28.892756 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:28.893971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:28.903259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '0e655e86152b4d719f43b75c4ffd4192'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:28.917923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3091/6182] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:13:28 2022] DELETE /identity/v3/groups/3254e44563e644959f14d1af556eb91a/users/0e655e86152b4d719f43b75c4ffd4192 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:28.922935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:28.934300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40826d36-ff79-49d2-8b93-b90be16a1996 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.002729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.003478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.003775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.004073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.006093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.007289 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.007289 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:29.007289 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:29.008289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.016481 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3092/6183] 10.222.0.33 () {60 vars in 1381 bytes} [Sat Sep 10 21:13:28 2022] PUT /identity/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:29.020598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.031398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a115b-da2c-4d36-995b-12842693a143 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.035930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.036557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.036842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.037115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.038274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=3254e44563e644959f14d1af556eb91a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.039281 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.039281 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:29.039281 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:29.040158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.053995 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3092/6184] 10.222.0.33 () {58 vars in 1263 bytes} [Sat Sep 10 21:13:29 2022] GET /identity/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles => generated 375 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:29.056955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.069141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012abab0-b3fc-4dac-9674-1755ccaafdcf tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.074362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.075027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.075292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.075567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.077211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.078270 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.078270 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:29.078270 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:29.079139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.086887 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3093/6185] 10.222.0.33 () {58 vars in 1363 bytes} [Sat Sep 10 21:13:29 2022] HEAD /identity/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.091230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.101512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9a2ad-d50b-47a8-96a4-69985518c59d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.107124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.107757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.108040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.108315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.109981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=3254e44563e644959f14d1af556eb91a, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.110978 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.110978 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:29.110978 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:29.111861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.120685 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3093/6186] 10.222.0.33 () {58 vars in 1365 bytes} [Sat Sep 10 21:13:29 2022] DELETE /identity/v3/system/groups/3254e44563e644959f14d1af556eb91a/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.125579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.135948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec8325-dbe3-45ab-aef4-328e7b9595c0 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.140811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.141525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.141868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.142157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.144476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d16dea24deb34f61b91996ba38762e70, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.147196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.159941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3094/6187] 10.222.0.33 () {60 vars in 1480 bytes} [Sat Sep 10 21:13:29 2022] PUT /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.163982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.175104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb6178-54a1-4a77-a1fa-670f4b835f08 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.179787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:29.236210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.236871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.237160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.237441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.239181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=d16dea24deb34f61b91996ba38762e70, user_id=0e655e86152b4d719f43b75c4ffd4192)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.241712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.257886 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3094/6188] 10.222.0.33 () {58 vars in 1362 bytes} [Sat Sep 10 21:13:29 2022] GET /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles => generated 408 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:29.259196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.270122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9acfd-f8fd-49e3-b43b-96aac5d4015e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.277437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.278156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.278446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.278725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.280925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=d16dea24deb34f61b91996ba38762e70, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.283651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.289762 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3095/6189] 10.222.0.33 () {58 vars in 1462 bytes} [Sat Sep 10 21:13:29 2022] HEAD /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.294545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.304863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7e17-7687-47d1-8647-f907666bcbec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.309965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.310777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.311597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.311963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.314585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d16dea24deb34f61b91996ba38762e70, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.317660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.324417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 6eaa7d92604b403696be366d2a3f9ecd was removed from user 0e655e86152b4d719f43b75c4ffd4192 on domain d16dea24deb34f61b91996ba38762e70. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:29.324793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:29.325565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:29.340045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3095/6190] 10.222.0.33 () {58 vars in 1464 bytes} [Sat Sep 10 21:13:29 2022] DELETE /identity/v3/domains/d16dea24deb34f61b91996ba38762e70/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:29.345369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.347262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:29.360960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737184bb-d459-462a-a748-e96e8cfeb261 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.419075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.419734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.420022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.420304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.422511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.425230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.438835 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3096/6191] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:29 2022] PUT /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.443000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.454908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c63d4-9041-42fe-b7fd-8169d30e63b9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.513739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.514437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.514721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.514999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.517558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_grants(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.519521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.531679 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3096/6192] 10.222.0.33 () {58 vars in 1365 bytes} [Sat Sep 10 21:13:29 2022] GET /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles => generated 409 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:29.535425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.546173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0089-7a5e-4408-a230-48b86db3586a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.550895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.552085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.552085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.552085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.554327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:check_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.557457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.563644 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3097/6193] 10.222.0.33 () {58 vars in 1465 bytes} [Sat Sep 10 21:13:29 2022] HEAD /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 29 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.567769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.581263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7bf5d-2bc6-4264-943a-65c9ce2aee2c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.586278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.586975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.587408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.587698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.590268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=d60a5156f2a14fe196bbfa4eaa948b4e, user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.592941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.598996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 6eaa7d92604b403696be366d2a3f9ecd was removed from user 0e655e86152b4d719f43b75c4ffd4192 on project d60a5156f2a14fe196bbfa4eaa948b4e. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:29.599431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:29.600134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:29.617765 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3097/6194] 10.222.0.33 () {58 vars in 1467 bytes} [Sat Sep 10 21:13:29 2022] DELETE /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.623744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.626172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:29.643157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f6a15-83fa-4f0c-8c62-a499c03bd7ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.657508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:29.712335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.712999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.713869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.714145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.715831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.716854 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.716854 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:29.716854 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:29.717745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.724259 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3098/6195] 10.222.0.33 () {60 vars in 1378 bytes} [Sat Sep 10 21:13:29 2022] PUT /identity/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.728282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.739383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4089ad2-349b-49ed-9089-652ab6424012 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.744094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.744740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.745030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.745305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.746482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=0e655e86152b4d719f43b75c4ffd4192)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.747506 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.747506 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:29.747506 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:29.748390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.761584 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3098/6196] 10.222.0.33 () {58 vars in 1260 bytes} [Sat Sep 10 21:13:29 2022] GET /identity/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles => generated 374 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:29.765518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.775398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00b360-ac0c-4aa6-bd0c-cf4a86da4cf4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.780640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.781299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.781583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.781906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.783541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.784531 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.784531 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:29.784531 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:29.785410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.790215 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3099/6197] 10.222.0.33 () {58 vars in 1360 bytes} [Sat Sep 10 21:13:29 2022] HEAD /identity/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 25 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.794714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.804727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd031-9645-4888-b5a1-f809436ae7da tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.810312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.810990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.811278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.811561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.813305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=0e655e86152b4d719f43b75c4ffd4192, role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.814369 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.814369 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:29.814369 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:29.815328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.823761 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3099/6198] 10.222.0.33 () {58 vars in 1362 bytes} [Sat Sep 10 21:13:29 2022] DELETE /identity/v3/system/users/0e655e86152b4d719f43b75c4ffd4192/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:29.829035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.840640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c6202-3db1-4162-b694-497f6d74b280 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.845688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a704a52-2c3c-4aaf-8609-b30c8d2a5a97 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.846435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704a52-2c3c-4aaf-8609-b30c8d2a5a97 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.847212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704a52-2c3c-4aaf-8609-b30c8d2a5a97 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.847212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704a52-2c3c-4aaf-8609-b30c8d2a5a97 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.848365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a704a52-2c3c-4aaf-8609-b30c8d2a5a97 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.849953 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-4a704a52-2c3c-4aaf-8609-b30c8d2a5a97 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:29.849953 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:29.849953 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:29.851819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a704a52-2c3c-4aaf-8609-b30c8d2a5a97 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.875254 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3100/6199] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:29 2022] POST /identity/v3/roles => generated 270 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:29.878740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.890637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b904854-991a-4450-a0ba-b076c07529f8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.896408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.897112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.897499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.897967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.898800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.900382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.900583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:29.901275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.901545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.933037 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3100/6200] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:29 2022] POST /identity/v3/roles => generated 269 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:29.946787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:29.949101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:29.950203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:29.975621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85df90-4bc0-4e2f-87c0-117e80309a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:29.985033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:29.986680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:29.987220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:29.987680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6633d07bff3948c8bbe5f5845e776bbf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:29.989571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=06fc36a6bf964a00bb4a3ad196f8cc42, implied_role_id=6633d07bff3948c8bbe5f5845e776bbf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:29.992337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.044985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:30.083884 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3101/6201] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:29 2022] PUT /identity/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6633d07bff3948c8bbe5f5845e776bbf => generated 530 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:30.084931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.103572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3101/6202] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:30.122521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.124288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:30.203897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad2d8-b00b-429a-8e35-7329ec96b537 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.210727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.211493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.211831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.212137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.212828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.215235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.242267 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3102/6203] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:13:30 2022] POST /identity/v3/domains => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:30.246993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.283015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19727b7-9a79-4bec-be59-20fd1f5017b6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.297078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:30.346942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa1ed-4937-4cf0-bbb4-7cb83cc12a48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.355572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.356561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.357056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.357382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.358435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.361159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.377143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.377981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.378362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.378734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.383857 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3103/6204] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:30 2022] POST /identity/v3/roles => generated 270 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:30.387973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.399636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f2008-2203-4da8-9d0a-a128e2f7b03c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.415627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.417536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.417854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.418148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/e6afac533cf34f50a8ac369ca70e9f35` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.420177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=06fc36a6bf964a00bb4a3ad196f8cc42, implied_role_id=e6afac533cf34f50a8ac369ca70e9f35)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.422839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.455447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.456968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.467690 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3104/6205] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:30 2022] PUT /identity/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/e6afac533cf34f50a8ac369ca70e9f35 => generated 531 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:30.467942 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3102/6206] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:30.468904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.533136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:30.539763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7143979-23a3-49ae-9dfb-18693d851381 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.547564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.548177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.548464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.548746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.550480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=06fc36a6bf964a00bb4a3ad196f8cc42, implied_role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.552636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.590048 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3105/6207] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:30 2022] PUT /identity/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6eaa7d92604b403696be366d2a3f9ecd => generated 525 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:30.593199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.609186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9e112-a2ee-4d1d-ae91-afdfb9df630b service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.693452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.694176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.694675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.694924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/6eaa7d92604b403696be366d2a3f9ecd/implies/06fc36a6bf964a00bb4a3ad196f8cc42` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.696707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6eaa7d92604b403696be366d2a3f9ecd, implied_role_id=06fc36a6bf964a00bb4a3ad196f8cc42)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.698407 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:30.698407 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:30.698407 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:30.699307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] 06fc36a6bf964a00bb4a3ad196f8cc42 cannot be an implied roles.: keystone.exception.InvalidImpliedRole: 06fc36a6bf964a00bb4a3ad196f8cc42 cannot be an implied roles. Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Sep 10 21:13:30.721807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Sep 10 21:13:30.723086 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:30.723086 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:30.723086 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Sep 10 21:13:30.723086 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Sep 10 21:13:30.723086 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: 06fc36a6bf964a00bb4a3ad196f8cc42 cannot be an implied roles. Sep 10 21:13:30.723086 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:30.723620 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3103/6208] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:30 2022] PUT /identity/v3/roles/6eaa7d92604b403696be366d2a3f9ecd/implies/06fc36a6bf964a00bb4a3ad196f8cc42 => generated 116 bytes in 131 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:13:30.727809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.740701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda2793-8fb9-4b7f-b687-f730f04b5f8f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.748764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.749741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.750231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.750716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.753694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=06fc36a6bf964a00bb4a3ad196f8cc42, implied_role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.755798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.770926 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3106/6209] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:30 2022] DELETE /identity/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:30.775189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.789704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f235d4-11b1-449c-be79-abdc242ebb48 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.838291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:30.863679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.864473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.865029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.865409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/e6afac533cf34f50a8ac369ca70e9f35` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.867605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=06fc36a6bf964a00bb4a3ad196f8cc42, implied_role_id=e6afac533cf34f50a8ac369ca70e9f35)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.869027 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:30.869027 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:30.869027 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:30.870797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:30.883508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3104/6210] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:30 2022] DELETE /identity/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/e6afac533cf34f50a8ac369ca70e9f35 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:30.887969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:30.899279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec47a-cc9f-47e8-8299-dccc1f38e17d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:30.965017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:30.965668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:30.966004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:30.966289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/e6afac533cf34f50a8ac369ca70e9f35` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:30.967453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=e6afac533cf34f50a8ac369ca70e9f35)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:30.969110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.005595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role e6afac533cf34f50a8ac369ca70e9f35 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:31.005915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.006615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.008408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3107/6211] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:30 2022] DELETE /identity/v3/roles/e6afac533cf34f50a8ac369ca70e9f35 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:31.012689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.014417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:31.027807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395e69-6f83-4000-a39e-79d650ee831e tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:31.093226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:31.095176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:31.095176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:31.095176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/41ad256a47604461b3228e65a82d640d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:31.112696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=41ad256a47604461b3228e65a82d640d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.115977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.142194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '41ad256a47604461b3228e65a82d640d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.143000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '41ad256a47604461b3228e65a82d640d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.184704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '41ad256a47604461b3228e65a82d640d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.185443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '41ad256a47604461b3228e65a82d640d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.189861 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3105/6212] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:31 2022] PATCH /identity/v3/domains/41ad256a47604461b3228e65a82d640d => generated 276 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:31.192254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.203477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebbab4-7cc2-4325-b9d8-e22847a91c66 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:31.270456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:31.271097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:31.271383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:31.271659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/41ad256a47604461b3228e65a82d640d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:31.283056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=41ad256a47604461b3228e65a82d640d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.284705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.310267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '41ad256a47604461b3228e65a82d640d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.312158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:31.314068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '41ad256a47604461b3228e65a82d640d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.364296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ad256a47604461b3228e65a82d640d', 'request_id': 'req-74d35955-8dbb-4b0e-904e-f5c722b57b72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.370437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ad256a47604461b3228e65a82d640d', 'request_id': 'req-74d35955-8dbb-4b0e-904e-f5c722b57b72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.377172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ad256a47604461b3228e65a82d640d', 'request_id': 'req-74d35955-8dbb-4b0e-904e-f5c722b57b72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.389622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] The token cache is being invalidate because project 41ad256a47604461b3228e65a82d640d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:31.389942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.390569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.397889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '41ad256a47604461b3228e65a82d640d', 'request_id': 'req-74d35955-8dbb-4b0e-904e-f5c722b57b72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.398557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '41ad256a47604461b3228e65a82d640d', 'request_id': 'req-74d35955-8dbb-4b0e-904e-f5c722b57b72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.402499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '41ad256a47604461b3228e65a82d640d', 'request_id': 'req-74d35955-8dbb-4b0e-904e-f5c722b57b72'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.408048 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3108/6213] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:31 2022] DELETE /identity/v3/domains/41ad256a47604461b3228e65a82d640d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:31.412400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.413815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:31.427354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ada87-f829-4535-b54c-a295963e656d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:31.489108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:31.489683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:31.490067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:31.490355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6633d07bff3948c8bbe5f5845e776bbf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:31.491981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=06fc36a6bf964a00bb4a3ad196f8cc42, implied_role_id=6633d07bff3948c8bbe5f5845e776bbf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.493592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.503456 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3106/6214] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:31 2022] DELETE /identity/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42/implies/6633d07bff3948c8bbe5f5845e776bbf => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:31.507834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.509401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:31.522280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d35955-8dbb-4b0e-904e-f5c722b57b72 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:31.581071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:31.581739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:31.582049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:31.582330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/6633d07bff3948c8bbe5f5845e776bbf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:31.583471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=6633d07bff3948c8bbe5f5845e776bbf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.585062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.621437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 6633d07bff3948c8bbe5f5845e776bbf has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:31.621714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.622534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.624460 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3109/6215] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:31 2022] DELETE /identity/v3/roles/6633d07bff3948c8bbe5f5845e776bbf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:31.628626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.634662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:31.650168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb4e76-4005-4af6-afbf-88bc1ff05a10 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:31.684796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:31.715554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:31.716291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:31.716584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:31.716866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:31.718032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=06fc36a6bf964a00bb4a3ad196f8cc42)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.719052 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:31.719052 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:31.719052 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:31.719941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.759445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 06fc36a6bf964a00bb4a3ad196f8cc42 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:31.759686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.761051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:31.763102 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3107/6216] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:31 2022] DELETE /identity/v3/roles/06fc36a6bf964a00bb4a3ad196f8cc42 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:31.768619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.779003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ebe40-0417-4f4f-80d4-ca6a7a5588d2 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:31.815677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:31.846428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:31.847086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:31.847377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:31.847660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:31.849661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.851303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.875842 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3110/6217] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:31 2022] PUT /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 521 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:31.880198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.881584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:31.896122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9508b-1fba-4943-bfd6-922ace7ef98a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:31.964611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:31.965512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:31.965920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:31.966300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:31.968421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.969547 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:31.969547 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:31.969547 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:31.970483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.972040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:31.972949 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:31.972949 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:31.972949 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:31.973786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:31.981123 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3108/6218] 10.222.0.33 () {58 vars in 1345 bytes} [Sat Sep 10 21:13:31 2022] HEAD /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:31.985273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:31.996247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea5ceb-56f2-4fae-876f-648372c222e8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.003476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.004245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.004555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.004806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.006887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.008021 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:32.008021 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:32.008021 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:32.009303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.020095 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3111/6219] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:13:31 2022] GET /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 521 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:32.021415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.033602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1656a-bce3-4dc0-b3b1-cf6d8c5a23ef tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.038971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.039772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.040065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.040340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.042822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.044433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.054576 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3109/6220] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:32 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:32.059346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.069573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3553b1-613e-4ff4-a0bd-2d942209d2a9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.139224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.139879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.140167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.140442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.142089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.143686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197.: keystone.exception.ImpliedRoleNotFound: a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197. Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = query.one() Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._iter().one() Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._only_one_row( Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:32.152170 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 10 21:13:32.153523 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 10 21:13:32.154709 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 10 21:13:32.154709 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197. Sep 10 21:13:32.154709 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:13:32.154709 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3112/6221] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:13:32 2022] GET /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 137 bytes in 95 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:13:32.157588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.168527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade7cef-03fc-4ba1-86fc-bf55c8faa116 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.173192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.173844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.174141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.174417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.176042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.177637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197.: keystone.exception.ImpliedRoleNotFound: a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197. Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = query.one() Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._iter().one() Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._only_one_row( Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:32.188214 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 10 21:13:32.189429 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 10 21:13:32.190715 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 10 21:13:32.190715 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 10 21:13:32.190715 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 10 21:13:32.190715 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: a29c1aaf5ae640c28acbb7270b9d0c9e does not imply 69a913c8527f45c0aeff76f39f9ed197. Sep 10 21:13:32.190715 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:32.190715 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3110/6222] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:32 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 137 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:32.194415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.205627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43245d7b-dc37-482f-a130-6c38336e90b3 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.210373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.210991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.211273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.211554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.213231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.214837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.240083 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3113/6223] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:32 2022] PUT /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 521 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:32.243976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.256093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a4f3c-9a36-48f6-90fc-7d39a54ce361 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.320391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.321090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.321381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.321663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.323931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=8649a4e29ce849a288dad6d2f32f23c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.325547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.346163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:32.365729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3111/6224] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:32 2022] PUT /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8 => generated 520 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:32.370154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.384222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc53fd-f84d-45c0-86b6-3d8e8689d61b tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.444369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.445024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.445313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.445594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.447487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=8649a4e29ce849a288dad6d2f32f23c8, implied_role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.449664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.470099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:32.476126 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3114/6225] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:32 2022] PUT /identity/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd => generated 519 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:32.480301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.490871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98021e41-0596-4090-820b-f11ccc9762ec tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.553672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.554354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.554797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.555083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.555789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.556680 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:32.556680 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:32.556680 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:32.557614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.580841 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3112/6226] 10.222.0.33 () {58 vars in 1152 bytes} [Sat Sep 10 21:13:32 2022] GET /identity/v3/role_inferences => generated 1605 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:32.581996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.595004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450a6e-802a-4342-92e2-05e91f2715ce tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.601148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.601799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.602122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.602403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.604904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=8649a4e29ce849a288dad6d2f32f23c8, implied_role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.606558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.615341 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3115/6227] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:32 2022] DELETE /identity/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:32.619623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.629863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfbf2b-a998-43e1-8864-fa3916b66ed8 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.690859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.691425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.691711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.691993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.693669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=8649a4e29ce849a288dad6d2f32f23c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.695385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.705497 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3113/6228] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:32 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:32.709780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.720815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7045f-fb5b-40fe-840a-ea71bda06449 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.782305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.782959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.783332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.783511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.785249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.786878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.797449 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3116/6229] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:32 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:32.802356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.813241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e530e-b200-443f-be9e-0e453ce70a73 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.877301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.877835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.878135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.878419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.879051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.880650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.896590 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3114/6230] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:32 2022] GET /identity/v3/roles => generated 2322 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:32.898001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.909555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b80c-a34c-4824-b220-f0e0f3973ffa tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.914300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.914901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.915189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.915469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.916883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.918563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.939324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3117/6231] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:13:32 2022] POST /identity/v3/roles => generated 234 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:32.940841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:32.951672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afcb41-3e5d-49ff-89b5-cdd85e86c662 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:32.956972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:32.957643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:32.958071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:32.958381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/14d4a3b07178448e9d209155690fffda` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:32.959861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:update_role(role_id=14d4a3b07178448e9d209155690fffda)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:32.960872 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:32.960872 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:32.960872 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:32.961764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:32.983382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:33.004911 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3115/6232] 10.222.0.33 () {60 vars in 1243 bytes} [Sat Sep 10 21:13:32 2022] PATCH /identity/v3/roles/14d4a3b07178448e9d209155690fffda => generated 237 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:33.015715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.026259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35255f8-461e-4407-9200-24bc0da3d1c4 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.030958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.031588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.031879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.032234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/14d4a3b07178448e9d209155690fffda` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.041012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:get_role(role_id=14d4a3b07178448e9d209155690fffda)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.042982 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:33.042982 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:33.042982 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:33.043858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.045548 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3118/6233] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:13:33 2022] GET /identity/v3/roles/14d4a3b07178448e9d209155690fffda => generated 237 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:33.049515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.062303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e1aee-eff9-4a60-aee1-527095ece262 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.066977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.067580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.067867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.068147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.068790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.070430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.082952 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3116/6234] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:33 2022] GET /identity/v3/roles => generated 2550 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:33.087122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.098483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dec0d6-633e-48a5-b423-f1032a55d635 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.103342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.104603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.104978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.105322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/14d4a3b07178448e9d209155690fffda` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.106929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_role(role_id=14d4a3b07178448e9d209155690fffda)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.108516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.147205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 14d4a3b07178448e9d209155690fffda has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:33.147500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:33.148399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:33.151707 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3119/6235] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:33 2022] DELETE /identity/v3/roles/14d4a3b07178448e9d209155690fffda => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:33.156625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.158422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:33.172274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c6bc7-c5ee-474c-bc2b-cca6566f0e27 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.236208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.236865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.237152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.237429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.239183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.240801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.271064 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3117/6236] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:33 2022] PUT /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 521 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:33.272226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.283947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a492-865d-43c7-8465-5e5c3a7075b5 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.352775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.353434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.354426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.354714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.357134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=8649a4e29ce849a288dad6d2f32f23c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.358770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.380654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:33.390075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3120/6237] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:33 2022] PUT /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8 => generated 520 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:33.391129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.402894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e728cb-d4a6-4a68-b36a-5580dc647986 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.478049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.478758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.479338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.479592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.481307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=8649a4e29ce849a288dad6d2f32f23c8, implied_role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.483009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.512695 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3118/6238] 10.222.0.33 () {60 vars in 1363 bytes} [Sat Sep 10 21:13:33 2022] PUT /identity/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd => generated 519 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:33.517177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.530179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe2e1-65c5-4a3d-8f4c-38e590a7ef84 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.608358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.609015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.609313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.609599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.610812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=8649a4e29ce849a288dad6d2f32f23c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.611893 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:33.611893 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:13:33.611893 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:13:33.612823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.622212 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3121/6239] 10.222.0.33 () {58 vars in 1245 bytes} [Sat Sep 10 21:13:33 2022] GET /identity/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies => generated 488 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:33.624172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.638280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66ad59-543c-43f4-baf0-0eb736b36279 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.643340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.643917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.644206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.644493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.645614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.646755 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:13:33.646755 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:13:33.646755 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:13:33.647651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.655396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3119/6240] 10.222.0.33 () {58 vars in 1245 bytes} [Sat Sep 10 21:13:33 2022] GET /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies => generated 660 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:33.659550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.669936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb3ebe-df74-4c0a-8e80-acc67efd0f9c tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.674977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.675737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.676022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.676325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.678654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=8649a4e29ce849a288dad6d2f32f23c8, implied_role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.680570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.689733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3122/6241] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:33 2022] DELETE /identity/v3/roles/8649a4e29ce849a288dad6d2f32f23c8/implies/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:33.693775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.704670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079be33-8dbb-49ab-9827-f707da27c46d tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.739998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:33.770671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.771328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.771612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.771900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.773534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=8649a4e29ce849a288dad6d2f32f23c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.775177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.784674 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3120/6242] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:33 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/8649a4e29ce849a288dad6d2f32f23c8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:33.788978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.799793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb1983-ee46-4766-88f9-3b7d52dfa011 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.859200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.859857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.860146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.860429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.862108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=a29c1aaf5ae640c28acbb7270b9d0c9e, implied_role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:33.863707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:33.872255 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3123/6243] 10.222.0.33 () {58 vars in 1347 bytes} [Sat Sep 10 21:13:33 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e/implies/69a913c8527f45c0aeff76f39f9ed197 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:33.877396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.888986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038235db-fd40-4e63-8934-ee21051d0a47 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.955769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.956595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.956890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.957170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/41ad256a47604461b3228e65a82d640d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:33.968230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Could not find domain: 41ad256a47604461b3228e65a82d640d.: keystone.exception.DomainNotFound: Could not find domain: 41ad256a47604461b3228e65a82d640d. Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 41ad256a47604461b3228e65a82d640d. Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:13:33.974240 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:13:33.976610 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 41ad256a47604461b3228e65a82d640d. Sep 10 21:13:33.979753 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:13:33.985053 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3121/6244] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:33 2022] PATCH /identity/v3/domains/41ad256a47604461b3228e65a82d640d => generated 112 bytes in 99 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:13:33.985229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:33.991381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d49d0-1aad-422d-994d-247039f69892 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:33.997457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:33.998247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:33.998546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:33.998949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/users/0e655e86152b4d719f43b75c4ffd4192` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:34.000256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0e655e86152b4d719f43b75c4ffd4192)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:34.002400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:34.093384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e655e86152b4d719f43b75c4ffd4192'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.115371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e655e86152b4d719f43b75c4ffd4192'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.130539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e655e86152b4d719f43b75c4ffd4192'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.135335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:34.142084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e655e86152b4d719f43b75c4ffd4192'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.143199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e655e86152b4d719f43b75c4ffd4192'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.153879 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3124/6245] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:33 2022] DELETE /identity/v3/users/0e655e86152b4d719f43b75c4ffd4192 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:34.156445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:34.158297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:34.172796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ac65b-c50d-4b49-afe3-e48ea5be9b81 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:34.237157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:34.237811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:34.238143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:34.238426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/6eaa7d92604b403696be366d2a3f9ecd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:34.239866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6eaa7d92604b403696be366d2a3f9ecd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:34.241517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:34.280711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 6eaa7d92604b403696be366d2a3f9ecd has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:34.281007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.281695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.283723 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3122/6246] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:34 2022] DELETE /identity/v3/roles/6eaa7d92604b403696be366d2a3f9ecd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:34.291051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:34.303409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe058d-417f-49d4-ac62-3458d947dd32 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:34.366785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:34.367436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:34.367726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:34.368011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/groups/3254e44563e644959f14d1af556eb91a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:34.370069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_group(group_id=3254e44563e644959f14d1af556eb91a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:34.372289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:34.420669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3125/6247] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:13:34 2022] DELETE /identity/v3/groups/3254e44563e644959f14d1af556eb91a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:34.423424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:34.425019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:34.439949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a41afa-b5b0-4c61-bb0f-6a35692fa6d6 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:34.480386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:34.507413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:34.509135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:34.509476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:34.510453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:34.512216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d60a5156f2a14fe196bbfa4eaa948b4e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:34.514358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:34.566662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60a5156f2a14fe196bbfa4eaa948b4e', 'request_id': 'req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.573293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60a5156f2a14fe196bbfa4eaa948b4e', 'request_id': 'req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.587289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60a5156f2a14fe196bbfa4eaa948b4e', 'request_id': 'req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.600069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] The token cache is being invalidate because project d60a5156f2a14fe196bbfa4eaa948b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:34.600518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.601358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.606714 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3123/6248] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:34 2022] DELETE /identity/v3/projects/d60a5156f2a14fe196bbfa4eaa948b4e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:34.607852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:34.610269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:34.632717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649b20c-68a5-4033-8490-5b09456fe714 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:34.710864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:34.711520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:34.711805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:34.712114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:34.713953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d16dea24deb34f61b91996ba38762e70)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:34.715903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:34.740706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd16dea24deb34f61b91996ba38762e70'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.746776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd16dea24deb34f61b91996ba38762e70'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.784621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd16dea24deb34f61b91996ba38762e70'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.785273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd16dea24deb34f61b91996ba38762e70'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.788154 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3126/6249] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:13:34 2022] PATCH /identity/v3/domains/d16dea24deb34f61b91996ba38762e70 => generated 277 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:34.791872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:34.804002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcedfe5-00ab-4773-a4a2-f775bbcdae99 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:34.874110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:34.874770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:34.875059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:34.875339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/domains/d16dea24deb34f61b91996ba38762e70` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:34.887365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d16dea24deb34f61b91996ba38762e70)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:34.888793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:34.910952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd16dea24deb34f61b91996ba38762e70'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.914355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd16dea24deb34f61b91996ba38762e70'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.955247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:34.976885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16dea24deb34f61b91996ba38762e70', 'request_id': 'req-bbde8ae3-50d8-4185-88e6-f78757ff945f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.984952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16dea24deb34f61b91996ba38762e70', 'request_id': 'req-bbde8ae3-50d8-4185-88e6-f78757ff945f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:34.993424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16dea24deb34f61b91996ba38762e70', 'request_id': 'req-bbde8ae3-50d8-4185-88e6-f78757ff945f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.007542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] The token cache is being invalidate because project d16dea24deb34f61b91996ba38762e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:35.008059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.008977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.018127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd16dea24deb34f61b91996ba38762e70', 'request_id': 'req-bbde8ae3-50d8-4185-88e6-f78757ff945f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.018795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd16dea24deb34f61b91996ba38762e70', 'request_id': 'req-bbde8ae3-50d8-4185-88e6-f78757ff945f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.019435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd16dea24deb34f61b91996ba38762e70', 'request_id': 'req-bbde8ae3-50d8-4185-88e6-f78757ff945f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.028078 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3124/6250] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:13:34 2022] DELETE /identity/v3/domains/d16dea24deb34f61b91996ba38762e70 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:35.032023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:35.033460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:35.046978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9dad7-c5b0-48d6-b7f9-b7d34093e198 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:35.060552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:35.112489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:35.113189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:35.113290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:35.113554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/8649a4e29ce849a288dad6d2f32f23c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:35.114983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8649a4e29ce849a288dad6d2f32f23c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:35.116588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:35.162801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 8649a4e29ce849a288dad6d2f32f23c8 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:35.163640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.164357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.167322 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3127/6251] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:35 2022] DELETE /identity/v3/roles/8649a4e29ce849a288dad6d2f32f23c8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:35.171282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:35.172786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:35.192641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde8ae3-50d8-4185-88e6-f78757ff945f tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:35.296362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:35.297117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:35.297407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:35.297686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/69a913c8527f45c0aeff76f39f9ed197` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:35.299153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_role(role_id=69a913c8527f45c0aeff76f39f9ed197)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:35.300779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:35.346171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role 69a913c8527f45c0aeff76f39f9ed197 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:35.346481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.349516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.351724 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3125/6252] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:35 2022] DELETE /identity/v3/roles/69a913c8527f45c0aeff76f39f9ed197 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:35.356390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:35.358003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:35.379653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62630-7fec-4724-bd90-acbd54808ca9 tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:35.458635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '542101eb7a3e42be99c101a4d5230701', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a87748e68c42489e49214189197a0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:35.461698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:35.463706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:35.466585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] PATH_INFO: `/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:35.469545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a29c1aaf5ae640c28acbb7270b9d0c9e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:35.473651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:35.552708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invalidating the token cache because role a29c1aaf5ae640c28acbb7270b9d0c9e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:35.553413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.554501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c0a299d-8628-47dd-9216-4fa4317411fd tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.558850 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3128/6253] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:35 2022] DELETE /identity/v3/roles/a29c1aaf5ae640c28acbb7270b9d0c9e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:35.562944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:35.566487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:35.664154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf105355-359f-423e-bc0c-acbdde4b9c7a tempest-RolesV3TestJSON-490128636 tempest-RolesV3TestJSON-490128636-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:35.685591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:35.739083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:35.740429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:35.740779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:35.741153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] PATH_INFO: `/v3/users/093bfd9ca8a9453fb05e332ca8c38729` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:35.742514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=093bfd9ca8a9453fb05e332ca8c38729)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:35.745206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:35.794342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '093bfd9ca8a9453fb05e332ca8c38729'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.802411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '093bfd9ca8a9453fb05e332ca8c38729'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.803256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '093bfd9ca8a9453fb05e332ca8c38729'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.815254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '093bfd9ca8a9453fb05e332ca8c38729'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.815901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '093bfd9ca8a9453fb05e332ca8c38729'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:35.841780 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3126/6254] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:35 2022] DELETE /identity/v3/users/093bfd9ca8a9453fb05e332ca8c38729 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:35.843471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:35.843759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:35.844057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:35.864640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:35.865909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:35.917179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:35.933782 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3129/6255] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:35.943478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:35.944918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:36.014254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb21972-3ea2-4f71-95e2-e20431f3afb8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:36.019089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:36.019804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:36.020566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:36.020566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:36.030953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:36.032417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:36.043833 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3127/6256] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:13:36.365541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:36.371699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:36.389791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdbc70-78d5-40e3-bd8c-7922cdca9386 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:36.394981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:36.395546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:36.395834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:36.396106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] PATH_INFO: `/v3/projects/5caea13722034559ac5318ed37296e84` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:36.397332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5caea13722034559ac5318ed37296e84)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:36.399329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:36.456616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5caea13722034559ac5318ed37296e84', 'request_id': 'req-34a6b168-4407-4275-803a-c472313be732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.463940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5caea13722034559ac5318ed37296e84', 'request_id': 'req-34a6b168-4407-4275-803a-c472313be732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.471745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5caea13722034559ac5318ed37296e84', 'request_id': 'req-34a6b168-4407-4275-803a-c472313be732'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.493703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] The token cache is being invalidate because project 5caea13722034559ac5318ed37296e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:36.494052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.495277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.498274 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3130/6257] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:36 2022] DELETE /identity/v3/projects/5caea13722034559ac5318ed37296e84 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:36.502228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:36.514453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f5ce-8077-4cfa-9a3a-61496920d035 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:36.563498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:36.592101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:36.592849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:36.593207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:36.593502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] PATH_INFO: `/v3/users/542101eb7a3e42be99c101a4d5230701` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:36.595223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542101eb7a3e42be99c101a4d5230701)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:36.597444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:36.698600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542101eb7a3e42be99c101a4d5230701'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.737172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542101eb7a3e42be99c101a4d5230701'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.738693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542101eb7a3e42be99c101a4d5230701'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.752754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542101eb7a3e42be99c101a4d5230701'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.753485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-10e9edbd-4f5e-4e29-8d6b-dd789cb44387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542101eb7a3e42be99c101a4d5230701'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:36.783517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3128/6258] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:36 2022] DELETE /identity/v3/users/542101eb7a3e42be99c101a4d5230701 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:37.218267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:37.219950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:37.269726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6b168-4407-4275-803a-c472313be732 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:37.358432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:37.359149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:37.359347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:37.359578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] PATH_INFO: `/v3/projects/06a87748e68c42489e49214189197a0b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:37.361969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a87748e68c42489e49214189197a0b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:37.363926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:37.450042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a87748e68c42489e49214189197a0b', 'request_id': 'req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:37.477238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a87748e68c42489e49214189197a0b', 'request_id': 'req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:37.485733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a87748e68c42489e49214189197a0b', 'request_id': 'req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:37.530778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] The token cache is being invalidate because project 06a87748e68c42489e49214189197a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:37.531374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:37.532619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-39ab6fb6-ed01-4db3-ba84-ca293464f6a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:37.534816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3131/6259] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:37 2022] DELETE /identity/v3/projects/06a87748e68c42489e49214189197a0b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:37.541582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac4532-b4f0-44d2-9b64-c9b444364ff4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:37.541906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac4532-b4f0-44d2-9b64-c9b444364ff4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:37.542185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac4532-b4f0-44d2-9b64-c9b444364ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:37.576457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0dac4532-b4f0-44d2-9b64-c9b444364ff4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:37.577990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0dac4532-b4f0-44d2-9b64-c9b444364ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:37.685495 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3129/6260] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:37.691962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5a682-dcdd-492d-ab0e-9355aea48cb4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:37.692265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5a682-dcdd-492d-ab0e-9355aea48cb4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:37.692572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5a682-dcdd-492d-ab0e-9355aea48cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:37.707857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:37.745776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c1a5a682-dcdd-492d-ab0e-9355aea48cb4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:37.747132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c1a5a682-dcdd-492d-ab0e-9355aea48cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:37.769108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3132/6261] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:37.774250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac4532-b4f0-44d2-9b64-c9b444364ff4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:37.814933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac4532-b4f0-44d2-9b64-c9b444364ff4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:37.824951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:37.826305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:37.826888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:37.827438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:37.828356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:37.829315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:37.833464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:37.883616 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3130/6262] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:37.887226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5a682-dcdd-492d-ab0e-9355aea48cb4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:37.897055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5a682-dcdd-492d-ab0e-9355aea48cb4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:37.913138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:37.913777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:37.914092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:37.914371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:37.915026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:37.917488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:38.011873 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3133/6263] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:37 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:38.014439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:38.054847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242764-dfb5-4661-bd81-b032edbdc4ca admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:38.179999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:38.199578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:38.200784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.201072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.201351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.202106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:38.204367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:38.233813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3131/6264] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:38 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:38.237777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:38.250997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5828fce-8eb3-491a-9aff-5032b6d6c656 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:38.256784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:38.257400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.257685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.258297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.258882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:38.260303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:38.271368 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3134/6265] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:38.275532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:38.286628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df892c1-63bd-4142-8b32-3b28f7c2eae5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:38.293703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7157b86-9544-45a6-9f8a-e3b6e61b2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:38.294511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7157b86-9544-45a6-9f8a-e3b6e61b2758 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.295252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7157b86-9544-45a6-9f8a-e3b6e61b2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.295252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7157b86-9544-45a6-9f8a-e3b6e61b2758 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.296229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7157b86-9544-45a6-9f8a-e3b6e61b2758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:38.298468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7157b86-9544-45a6-9f8a-e3b6e61b2758 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:38.313862 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3132/6266] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:13:38.317917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:38.328336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac96aa-66fc-419b-9211-dab9b8c687fa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:38.332953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:38.333605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.334270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.334270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] PATH_INFO: `/v3/projects/cb3b9e7f118a41b0acca80e157a43961/users/342b525bfc67431ca994ba125e2516ff/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.351196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3b9e7f118a41b0acca80e157a43961, user_id=342b525bfc67431ca994ba125e2516ff, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:38.355748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:38.378480 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3135/6267] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:38 2022] PUT /identity/v3/projects/cb3b9e7f118a41b0acca80e157a43961/users/342b525bfc67431ca994ba125e2516ff/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:38.390306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdccf70-3e39-4409-a26c-9b9bc35ea501 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.390695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdccf70-3e39-4409-a26c-9b9bc35ea501 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.391038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdccf70-3e39-4409-a26c-9b9bc35ea501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.421188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3fdccf70-3e39-4409-a26c-9b9bc35ea501 None None] MFA Rules not processed for user `342b525bfc67431ca994ba125e2516ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:38.422481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3fdccf70-3e39-4409-a26c-9b9bc35ea501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:38.500841 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3133/6268] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:38.503603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:38.533609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757add-0041-4702-b3ae-211eeea63161 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:38.547017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cce839-3563-4b17-87ae-2f6ff4b5dd03 tempest-DefaultDomainTestJSON-1955590291 tempest-DefaultDomainTestJSON-1955590291-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '342b525bfc67431ca994ba125e2516ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb3b9e7f118a41b0acca80e157a43961', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:38.548243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce839-3563-4b17-87ae-2f6ff4b5dd03 tempest-DefaultDomainTestJSON-1955590291 tempest-DefaultDomainTestJSON-1955590291-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.549159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce839-3563-4b17-87ae-2f6ff4b5dd03 tempest-DefaultDomainTestJSON-1955590291 tempest-DefaultDomainTestJSON-1955590291-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.550210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce839-3563-4b17-87ae-2f6ff4b5dd03 tempest-DefaultDomainTestJSON-1955590291 tempest-DefaultDomainTestJSON-1955590291-project] PATH_INFO: `/v3/domains/default` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.552291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cce839-3563-4b17-87ae-2f6ff4b5dd03 tempest-DefaultDomainTestJSON-1955590291 tempest-DefaultDomainTestJSON-1955590291-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:38.558369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cce839-3563-4b17-87ae-2f6ff4b5dd03 tempest-DefaultDomainTestJSON-1955590291 tempest-DefaultDomainTestJSON-1955590291-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:38.561081 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3136/6269] 10.222.0.33 () {58 vars in 1152 bytes} [Sat Sep 10 21:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:38.568429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdccf70-3e39-4409-a26c-9b9bc35ea501 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:38.691252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdccf70-3e39-4409-a26c-9b9bc35ea501 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:38.704696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:38.705357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.705660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.705991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] PATH_INFO: `/v3/users/342b525bfc67431ca994ba125e2516ff` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.707876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] RBAC: Authorizing `identity:delete_user(user_id=342b525bfc67431ca994ba125e2516ff)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:38.709866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:38.790126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342b525bfc67431ca994ba125e2516ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:38.801675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342b525bfc67431ca994ba125e2516ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:38.802501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342b525bfc67431ca994ba125e2516ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:38.804630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:38.812267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342b525bfc67431ca994ba125e2516ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:38.813065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342b525bfc67431ca994ba125e2516ff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:38.836054 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3134/6270] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:38 2022] DELETE /identity/v3/users/342b525bfc67431ca994ba125e2516ff => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:38.841338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:38.841625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:38.841933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:38.889627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:38.891090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:38.967205 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3137/6271] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:38.985189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:38.986678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:39.056264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498613c3-39e4-4a90-8bc0-082e829b055f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:39.059993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c219fcb-89ca-480a-a084-43196051181d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:39.060603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c219fcb-89ca-480a-a084-43196051181d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.060901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c219fcb-89ca-480a-a084-43196051181d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.061160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c219fcb-89ca-480a-a084-43196051181d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.070911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c219fcb-89ca-480a-a084-43196051181d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:39.072891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c219fcb-89ca-480a-a084-43196051181d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:39.083679 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3135/6272] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:39.435557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:39.436978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:39.463762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec505c4-63e5-44ef-9dad-cc351b57194b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:39.469622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:39.470411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.470815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.471156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] PATH_INFO: `/v3/projects/cb3b9e7f118a41b0acca80e157a43961` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.472349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3b9e7f118a41b0acca80e157a43961)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:39.474811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:39.526317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3b9e7f118a41b0acca80e157a43961', 'request_id': 'req-e1303486-2bac-4a5e-80c0-919638b37c29'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:39.533620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3b9e7f118a41b0acca80e157a43961', 'request_id': 'req-e1303486-2bac-4a5e-80c0-919638b37c29'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:39.539855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3b9e7f118a41b0acca80e157a43961', 'request_id': 'req-e1303486-2bac-4a5e-80c0-919638b37c29'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:39.554299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] The token cache is being invalidate because project cb3b9e7f118a41b0acca80e157a43961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:39.554598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:39.555240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e1303486-2bac-4a5e-80c0-919638b37c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:39.556808 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3138/6273] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:39 2022] DELETE /identity/v3/projects/cb3b9e7f118a41b0acca80e157a43961 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:39.561481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d68f-8451-4669-9d1b-5a5bfb899d31 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.561800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d68f-8451-4669-9d1b-5a5bfb899d31 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.562405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d68f-8451-4669-9d1b-5a5bfb899d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.582968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a444d68f-8451-4669-9d1b-5a5bfb899d31 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:39.584183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a444d68f-8451-4669-9d1b-5a5bfb899d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:39.645373 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3136/6274] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:39.654152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae2a39-b195-4534-9ab8-1149eae9f6d5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.654468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae2a39-b195-4534-9ab8-1149eae9f6d5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.654741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae2a39-b195-4534-9ab8-1149eae9f6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.674781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d5ae2a39-b195-4534-9ab8-1149eae9f6d5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:39.675978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d5ae2a39-b195-4534-9ab8-1149eae9f6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:39.684288 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3139/6275] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:39.688390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d68f-8451-4669-9d1b-5a5bfb899d31 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:39.698769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d68f-8451-4669-9d1b-5a5bfb899d31 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:39.703333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:39.703926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.704212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.704493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.705171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:39.705475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:39.707111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:39.719613 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3137/6276] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:39.723818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae2a39-b195-4534-9ab8-1149eae9f6d5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:39.733893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae2a39-b195-4534-9ab8-1149eae9f6d5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:39.738706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:39.739313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.739595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.739869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.740497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:39.742428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:39.753348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:39.786019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3140/6277] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:39 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:39.788013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:39.800245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb7516-f558-45cd-a24e-5be4b8e8ab7e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:39.863737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:39.864356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.864641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.864918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.865548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:39.867513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:39.879496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:39.894341 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3138/6278] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:39 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:39.898449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:39.909379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34365084-6251-42b6-ac7e-5bb58ad6d537 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:39.914310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:39.915160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.915160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.915422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.915975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:39.917636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:39.928326 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3141/6279] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:39.932895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:39.943983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992be57-c3fd-49f7-a181-73037421eae8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:39.949433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70a34c-8db1-4864-8c90-6f911b3a5038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:39.950147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70a34c-8db1-4864-8c90-6f911b3a5038 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:39.950433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70a34c-8db1-4864-8c90-6f911b3a5038 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:39.950710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70a34c-8db1-4864-8c90-6f911b3a5038 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:39.951494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70a34c-8db1-4864-8c90-6f911b3a5038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:39.953095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70a34c-8db1-4864-8c90-6f911b3a5038 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:39.965546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3139/6280] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:39.971072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:39.996588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5e9ee-49c2-4da3-9913-bc5eaad36c90 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:40.001577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:40.002282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:40.002573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:40.003160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] PATH_INFO: `/v3/projects/d314dd68ea38450992dc90cdddf02979/users/15e38aa94c8c4315a0f846d48611af9d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:40.023783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d314dd68ea38450992dc90cdddf02979, user_id=15e38aa94c8c4315a0f846d48611af9d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:40.026975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:40.041515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3142/6281] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:39 2022] PUT /identity/v3/projects/d314dd68ea38450992dc90cdddf02979/users/15e38aa94c8c4315a0f846d48611af9d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:40.052967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945174e-9d01-488a-9517-9d2faf0ed68a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:40.053186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945174e-9d01-488a-9517-9d2faf0ed68a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:40.053457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945174e-9d01-488a-9517-9d2faf0ed68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:40.086854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7945174e-9d01-488a-9517-9d2faf0ed68a None None] MFA Rules not processed for user `15e38aa94c8c4315a0f846d48611af9d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:40.088881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7945174e-9d01-488a-9517-9d2faf0ed68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:40.156637 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3140/6282] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:40.168818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:40.170714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:40.242554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbff30-510d-448f-a386-813f766e9608 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:40.246691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:40.247300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:40.247582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:40.247859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:40.256542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:40.258047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:40.267432 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3143/6283] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:40.409268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383a621-6447-4d37-9eb3-94a3adf472c7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:40.409555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383a621-6447-4d37-9eb3-94a3adf472c7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:40.409838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383a621-6447-4d37-9eb3-94a3adf472c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:40.435297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7383a621-6447-4d37-9eb3-94a3adf472c7 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:40.436759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7383a621-6447-4d37-9eb3-94a3adf472c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:40.498145 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3141/6284] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:40.519824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:40.521337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:40.607877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5a7a-0100-45c7-8e31-5d7c32ca245c service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:40.613925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:40.614738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:40.615125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:40.615500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:40.625460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:40.627393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:40.638282 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3144/6285] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:41.033221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03377d-3486-46fa-bfbf-000efa467b92 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:41.033710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03377d-3486-46fa-bfbf-000efa467b92 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:41.034476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03377d-3486-46fa-bfbf-000efa467b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:41.056386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fd03377d-3486-46fa-bfbf-000efa467b92 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:41.057654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fd03377d-3486-46fa-bfbf-000efa467b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:41.066766 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3142/6286] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:41.080006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:41.094888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66191ad9-f2e5-4ecb-90ff-324e09d1af6d service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:41.101386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:41.102033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:41.102567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:41.102851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:41.112739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:41.114646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:41.123840 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3145/6287] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:41.657694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de035f6-c4d6-4b11-837c-29a6302e31fb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:41.658111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de035f6-c4d6-4b11-837c-29a6302e31fb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:41.658818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de035f6-c4d6-4b11-837c-29a6302e31fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:41.690851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7de035f6-c4d6-4b11-837c-29a6302e31fb None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:41.692063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7de035f6-c4d6-4b11-837c-29a6302e31fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:41.701366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3143/6288] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:41.715001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:41.734184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f284b2a-1490-4f00-ab62-88b3d3ab627a service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:41.739422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:41.740040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:41.740310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:41.740578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:41.750408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:41.752001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:41.772934 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3146/6289] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:42.003157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ba78-9ba6-43e0-b4ba-7a5e4d4796e4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:42.003797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ba78-9ba6-43e0-b4ba-7a5e4d4796e4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:42.004325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ba78-9ba6-43e0-b4ba-7a5e4d4796e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:42.030583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2344ba78-9ba6-43e0-b4ba-7a5e4d4796e4 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:42.031853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2344ba78-9ba6-43e0-b4ba-7a5e4d4796e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:42.042227 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3144/6290] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:42.054304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:42.083049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433f044-b826-481f-928f-8611b5031c1c service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:42.093503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:42.094676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:42.095187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:42.095535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:42.108339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:42.110822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:42.119024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:42.128997 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3147/6291] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:42.578138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30517618-6b30-4c7b-be40-3f452ed199dc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:42.578426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30517618-6b30-4c7b-be40-3f452ed199dc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:42.579278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30517618-6b30-4c7b-be40-3f452ed199dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:42.610665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-30517618-6b30-4c7b-be40-3f452ed199dc None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:42.611991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30517618-6b30-4c7b-be40-3f452ed199dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:42.634205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3145/6292] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:42.643364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:42.670235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663416-2db8-414f-84fa-226e383690c9 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:42.700435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:42.701366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:42.702128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:42.702593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:42.717726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:42.721365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:42.736274 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3148/6293] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:42.963378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b409d-7a8b-4e19-942f-496bbad33269 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:42.964112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b409d-7a8b-4e19-942f-496bbad33269 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:42.964843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b409d-7a8b-4e19-942f-496bbad33269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:42.987144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d69b409d-7a8b-4e19-942f-496bbad33269 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:42.988346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d69b409d-7a8b-4e19-942f-496bbad33269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:42.996538 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3146/6294] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:43.015756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:43.051966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e35558-7d98-40cf-b7c6-948a9f333410 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:43.066691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d463de8-910f-4d2d-899a-f00d3cc849c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:43.067602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d463de8-910f-4d2d-899a-f00d3cc849c0 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:43.068113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d463de8-910f-4d2d-899a-f00d3cc849c0 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:43.068710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d463de8-910f-4d2d-899a-f00d3cc849c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:43.078537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d463de8-910f-4d2d-899a-f00d3cc849c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:43.080428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d463de8-910f-4d2d-899a-f00d3cc849c0 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:43.090485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3149/6295] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:13:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:13:43.235601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b409d-7a8b-4e19-942f-496bbad33269 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:43.290417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:43.313811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b409d-7a8b-4e19-942f-496bbad33269 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:43.319596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:43.320561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:43.320918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:43.321259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] PATH_INFO: `/v3/users/15e38aa94c8c4315a0f846d48611af9d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:43.323273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e38aa94c8c4315a0f846d48611af9d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:43.325237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:43.390993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e38aa94c8c4315a0f846d48611af9d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:43.399174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e38aa94c8c4315a0f846d48611af9d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:43.399941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e38aa94c8c4315a0f846d48611af9d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:43.408402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e38aa94c8c4315a0f846d48611af9d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:43.409323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e38aa94c8c4315a0f846d48611af9d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:43.430729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3147/6296] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:43 2022] DELETE /identity/v3/users/15e38aa94c8c4315a0f846d48611af9d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:43.435843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:43.436130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:43.436616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:43.457915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:43.459079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:43.607175 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3150/6297] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:43.618444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:43.623837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:43.712531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b695bbb-39da-4eed-9ce3-fae42d0cf6f6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:43.717963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65157-e4b4-4420-90ac-087b35ef138c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:43.718800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65157-e4b4-4420-90ac-087b35ef138c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:43.719944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65157-e4b4-4420-90ac-087b35ef138c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:43.720428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65157-e4b4-4420-90ac-087b35ef138c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:43.744917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65157-e4b4-4420-90ac-087b35ef138c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:43.746427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65157-e4b4-4420-90ac-087b35ef138c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:43.759081 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3148/6298] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:44.108556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.110148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:44.126872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01bbcd-a1fa-4536-b9ac-4a64fd8c349b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:44.135343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:44.138774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.140194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.141117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] PATH_INFO: `/v3/projects/d314dd68ea38450992dc90cdddf02979` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.142956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d314dd68ea38450992dc90cdddf02979)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:44.145355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:44.209323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd314dd68ea38450992dc90cdddf02979', 'request_id': 'req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:44.215862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd314dd68ea38450992dc90cdddf02979', 'request_id': 'req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:44.223624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd314dd68ea38450992dc90cdddf02979', 'request_id': 'req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:44.238954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] The token cache is being invalidate because project d314dd68ea38450992dc90cdddf02979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:44.239251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:44.239913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2e1d198d-0f3e-40ef-84eb-f3febba4c26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:44.241409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3151/6299] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:44 2022] DELETE /identity/v3/projects/d314dd68ea38450992dc90cdddf02979 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:44.248059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e68f6b-7661-4c35-ab81-bc8ade9dc167 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.248915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e68f6b-7661-4c35-ab81-bc8ade9dc167 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.249195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e68f6b-7661-4c35-ab81-bc8ade9dc167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.271543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a5e68f6b-7661-4c35-ab81-bc8ade9dc167 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:44.273663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a5e68f6b-7661-4c35-ab81-bc8ade9dc167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:44.355834 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3149/6300] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:44.363644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52341cb7-0582-4b8a-b007-5b1cf88a077f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.363933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52341cb7-0582-4b8a-b007-5b1cf88a077f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.364218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52341cb7-0582-4b8a-b007-5b1cf88a077f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.388827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-52341cb7-0582-4b8a-b007-5b1cf88a077f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:44.390523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-52341cb7-0582-4b8a-b007-5b1cf88a077f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:44.399435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3152/6301] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:44.404203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e68f6b-7661-4c35-ab81-bc8ade9dc167 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.414062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e68f6b-7661-4c35-ab81-bc8ade9dc167 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:44.419034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:44.419632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.419912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.420184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.420822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:44.421124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:44.422752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:44.424268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:44.436397 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3150/6302] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:44.440756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52341cb7-0582-4b8a-b007-5b1cf88a077f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.451000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52341cb7-0582-4b8a-b007-5b1cf88a077f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:44.456455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:44.456975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.457292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.457569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.458253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:44.460167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:44.490173 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3153/6303] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:44 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:44.495299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.506610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ea564-f7d9-468d-a1bb-3062ae8ef0ea admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:44.581602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:44.582638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.582638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.582868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.583430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:44.585633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:44.617575 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3151/6304] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:44 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:44.618715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.637988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ae0a1-07ec-464e-a6ca-249cfe366316 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:44.649553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:44.650425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.650720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.651000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.651654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:44.653258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:44.664802 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3154/6305] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:13:44.668984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.680339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d8a96-3a11-4c6a-8a4c-3ef3ec07a299 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:44.686875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5718d9-efc3-4c3e-8830-6a1fd4237e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:44.687637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5718d9-efc3-4c3e-8830-6a1fd4237e1a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.688022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5718d9-efc3-4c3e-8830-6a1fd4237e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.688507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5718d9-efc3-4c3e-8830-6a1fd4237e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.689257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5718d9-efc3-4c3e-8830-6a1fd4237e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:44.691041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5718d9-efc3-4c3e-8830-6a1fd4237e1a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:44.706541 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3152/6306] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:44.710473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.722502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4033-6963-417f-8012-3c8eb80ef1cb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:44.727935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:44.728800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.729436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.729812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] PATH_INFO: `/v3/projects/5d2c5b035da34388aa1605d3a756e334/users/981e57aa469c4b12ab94547a7d8788ef/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.743163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:44.748973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2c5b035da34388aa1605d3a756e334, user_id=981e57aa469c4b12ab94547a7d8788ef, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:44.752515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:44.768341 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3155/6307] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:44 2022] PUT /identity/v3/projects/5d2c5b035da34388aa1605d3a756e334/users/981e57aa469c4b12ab94547a7d8788ef/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:44.779302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38c8fb-66c2-42c2-90b8-736c042a60a5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:44.779594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38c8fb-66c2-42c2-90b8-736c042a60a5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:44.779885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38c8fb-66c2-42c2-90b8-736c042a60a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:44.812252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4c38c8fb-66c2-42c2-90b8-736c042a60a5 None None] MFA Rules not processed for user `981e57aa469c4b12ab94547a7d8788ef`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:44.814012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4c38c8fb-66c2-42c2-90b8-736c042a60a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:44.886998 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3153/6308] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:44.901655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:44.904149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:44.992365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06473df7-509c-4a7d-a00a-e247fa3c6ed5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:45.001941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c04c97-9b22-49ce-9273-308d0ff085b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:45.001941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c04c97-9b22-49ce-9273-308d0ff085b2 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:45.001941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c04c97-9b22-49ce-9273-308d0ff085b2 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:45.001941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c04c97-9b22-49ce-9273-308d0ff085b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:45.014347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c04c97-9b22-49ce-9273-308d0ff085b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:45.016269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c04c97-9b22-49ce-9273-308d0ff085b2 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:45.026824 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3156/6309] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:45.152361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38c8fb-66c2-42c2-90b8-736c042a60a5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:45.253344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38c8fb-66c2-42c2-90b8-736c042a60a5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:45.262584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:45.263257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:45.263548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:45.263832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] PATH_INFO: `/v3/users/981e57aa469c4b12ab94547a7d8788ef` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:45.265510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=981e57aa469c4b12ab94547a7d8788ef)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:45.267524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:45.271424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:45.338980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '981e57aa469c4b12ab94547a7d8788ef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:45.348499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '981e57aa469c4b12ab94547a7d8788ef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:45.349242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '981e57aa469c4b12ab94547a7d8788ef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:45.360282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '981e57aa469c4b12ab94547a7d8788ef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:45.361052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '981e57aa469c4b12ab94547a7d8788ef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:45.385350 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3154/6310] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:45 2022] DELETE /identity/v3/users/981e57aa469c4b12ab94547a7d8788ef => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:45.390499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:45.391056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:45.391056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:45.412509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:45.414278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:45.503306 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3157/6311] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:45.512578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:45.514243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:45.616851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf24da6-a512-4cbe-9c14-994a1b938471 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:45.622975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fc65c-d8a4-48e5-a6f1-595d99da25e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:45.623750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fc65c-d8a4-48e5-a6f1-595d99da25e5 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:45.624157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fc65c-d8a4-48e5-a6f1-595d99da25e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:45.624548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fc65c-d8a4-48e5-a6f1-595d99da25e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:45.636887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fc65c-d8a4-48e5-a6f1-595d99da25e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:45.641263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fc65c-d8a4-48e5-a6f1-595d99da25e5 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:45.660490 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3155/6312] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:46.004487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:46.006500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:46.031889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06136ce-79eb-4fbe-93c9-d625f61e544a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:46.055625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:46.058151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.058509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.059365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] PATH_INFO: `/v3/projects/5d2c5b035da34388aa1605d3a756e334` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.061656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2c5b035da34388aa1605d3a756e334)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:46.065658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:46.126603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2c5b035da34388aa1605d3a756e334', 'request_id': 'req-5e28863f-95c7-44c0-ac43-805889d7fa77'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:46.135134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2c5b035da34388aa1605d3a756e334', 'request_id': 'req-5e28863f-95c7-44c0-ac43-805889d7fa77'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:46.144847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2c5b035da34388aa1605d3a756e334', 'request_id': 'req-5e28863f-95c7-44c0-ac43-805889d7fa77'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:46.162651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] The token cache is being invalidate because project 5d2c5b035da34388aa1605d3a756e334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:46.164889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:46.166517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5e28863f-95c7-44c0-ac43-805889d7fa77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:46.168514 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3158/6313] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:46 2022] DELETE /identity/v3/projects/5d2c5b035da34388aa1605d3a756e334 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:46.173140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bfbf-12ba-420c-8457-0ac84b9f56ff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.173428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bfbf-12ba-420c-8457-0ac84b9f56ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.173759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92bfbf-12ba-420c-8457-0ac84b9f56ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.196693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8f92bfbf-12ba-420c-8457-0ac84b9f56ff None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:46.198327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8f92bfbf-12ba-420c-8457-0ac84b9f56ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:46.291091 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3156/6314] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:46.299810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317d2d8-08cb-4294-8f38-299001f66c35 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.300284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317d2d8-08cb-4294-8f38-299001f66c35 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.300564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f317d2d8-08cb-4294-8f38-299001f66c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.323759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f317d2d8-08cb-4294-8f38-299001f66c35 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:46.325023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f317d2d8-08cb-4294-8f38-299001f66c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:46.344920 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3159/6315] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:46.349844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bfbf-12ba-420c-8457-0ac84b9f56ff None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:46.366866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92bfbf-12ba-420c-8457-0ac84b9f56ff None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:46.373882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:46.374705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.375092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.375379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.376080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:46.376326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:46.382637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:46.396877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3157/6316] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:46.401234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317d2d8-08cb-4294-8f38-299001f66c35 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:46.412762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f317d2d8-08cb-4294-8f38-299001f66c35 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:46.419339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:46.420053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.420354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.420611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.421312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:46.423596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:46.435877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:46.462978 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3160/6317] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:46 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:46.464654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:46.476544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0298bc-19aa-406c-8b3f-5f635669fc8b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:46.488909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:46.555235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:46.555829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.556108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.556525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.557169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:46.559546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:46.597141 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3158/6318] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:46 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:46.598842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:46.615298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad3cef-b567-4e05-b95a-8642b5c9af3f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:46.622004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:46.622628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.622916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.623181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.623845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:46.626153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:46.642096 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3161/6319] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:46.643750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:46.656407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853f8ac-4c6d-43b6-9c92-d0e7b40a88c3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:46.661979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c6441-4f6f-4798-b636-8896781020dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:46.662588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c6441-4f6f-4798-b636-8896781020dc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.662872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c6441-4f6f-4798-b636-8896781020dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.663515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c6441-4f6f-4798-b636-8896781020dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.663820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c6441-4f6f-4798-b636-8896781020dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:46.665861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c6441-4f6f-4798-b636-8896781020dc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:46.678333 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3159/6320] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:46.683487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:46.696082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cd78f-2298-4cd0-b289-a330acc5e9c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:46.706845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:46.707456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.707784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.708100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] PATH_INFO: `/v3/projects/0b4b4457bd2647218bd8cd055cdc3cd6/users/a7f143035240427ab58fb02866b531dc/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.729200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4b4457bd2647218bd8cd055cdc3cd6, user_id=a7f143035240427ab58fb02866b531dc, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:46.732822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:46.758099 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3162/6321] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:46 2022] PUT /identity/v3/projects/0b4b4457bd2647218bd8cd055cdc3cd6/users/a7f143035240427ab58fb02866b531dc/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:46.844362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:46.844629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:46.844915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:46.891864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] MFA Rules not processed for user `a7f143035240427ab58fb02866b531dc`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:46.893375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:46.989645 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3160/6322] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:47.001674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:47.003448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:47.081387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3c4f-f03b-455d-bbd3-509ab004d965 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:47.086865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:47.088155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:47.088704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:47.091305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:47.101362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:47.103134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:47.115966 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3163/6323] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:47.315884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:47.317329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:47.430075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce038c-ab67-41f6-9244-ca0d1d42c3d2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:47.437967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08990d28-1d2a-4171-8ed5-51ccc75bae8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:47.438928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08990d28-1d2a-4171-8ed5-51ccc75bae8c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:47.439429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08990d28-1d2a-4171-8ed5-51ccc75bae8c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:47.439733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08990d28-1d2a-4171-8ed5-51ccc75bae8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:47.442333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08990d28-1d2a-4171-8ed5-51ccc75bae8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:47.520246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:47.568493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08990d28-1d2a-4171-8ed5-51ccc75bae8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:47.570019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08990d28-1d2a-4171-8ed5-51ccc75bae8c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:47.581903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3161/6324] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:47.692755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:47.709540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c090-59d7-41b0-a8f9-ba76dd8d3989 service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:47.716437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:47.717307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:47.717595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:47.717912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] PATH_INFO: `/v3/users/a7f143035240427ab58fb02866b531dc` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:47.719068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f143035240427ab58fb02866b531dc)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:47.720989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:47.797805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f143035240427ab58fb02866b531dc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:47.814811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f143035240427ab58fb02866b531dc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:47.824972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f143035240427ab58fb02866b531dc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:47.834083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f143035240427ab58fb02866b531dc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:47.834745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f143035240427ab58fb02866b531dc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:47.841236 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3164/6325] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:47 2022] DELETE /identity/v3/users/a7f143035240427ab58fb02866b531dc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:47.846000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:47.846229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:47.846504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:47.866769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:47.868005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:47.934076 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3162/6326] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:47.943854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:47.945291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:47.986709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:48.019126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63746e-6db6-4f28-aec7-2d03d7807b63 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:48.022812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd2765-40a1-4c74-a19b-753c5dc0a9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:48.023420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2765-40a1-4c74-a19b-753c5dc0a9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:48.023703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2765-40a1-4c74-a19b-753c5dc0a9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:48.023978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2765-40a1-4c74-a19b-753c5dc0a9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:48.033388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd2765-40a1-4c74-a19b-753c5dc0a9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:48.034868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd2765-40a1-4c74-a19b-753c5dc0a9e6 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:48.045569 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3165/6327] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:48.694861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:48.697739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:48.711245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e0f66-e10b-48c7-871c-1684cc41ecb5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:48.716869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:48.717593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:48.717982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:48.718462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] PATH_INFO: `/v3/projects/0b4b4457bd2647218bd8cd055cdc3cd6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:48.719859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4b4457bd2647218bd8cd055cdc3cd6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:48.721793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:48.785780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4b4457bd2647218bd8cd055cdc3cd6', 'request_id': 'req-889b69cd-f7ef-496b-9749-f20b0bb251d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:48.796449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4b4457bd2647218bd8cd055cdc3cd6', 'request_id': 'req-889b69cd-f7ef-496b-9749-f20b0bb251d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:48.807653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4b4457bd2647218bd8cd055cdc3cd6', 'request_id': 'req-889b69cd-f7ef-496b-9749-f20b0bb251d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:48.821325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] The token cache is being invalidate because project 0b4b4457bd2647218bd8cd055cdc3cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:48.823082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:48.823082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-889b69cd-f7ef-496b-9749-f20b0bb251d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:48.824832 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3163/6328] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:48 2022] DELETE /identity/v3/projects/0b4b4457bd2647218bd8cd055cdc3cd6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:48.829650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee8fc-c0d0-479d-83d1-a8de486db44a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:48.830297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee8fc-c0d0-479d-83d1-a8de486db44a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:48.830297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ee8fc-c0d0-479d-83d1-a8de486db44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:48.856988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-282ee8fc-c0d0-479d-83d1-a8de486db44a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:48.858242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-282ee8fc-c0d0-479d-83d1-a8de486db44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:48.924618 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3166/6329] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:48.932868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57633b-9175-4a8f-9e90-c8cbdf426cf0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:48.933472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57633b-9175-4a8f-9e90-c8cbdf426cf0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:48.933818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57633b-9175-4a8f-9e90-c8cbdf426cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:48.961425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2a57633b-9175-4a8f-9e90-c8cbdf426cf0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:48.963915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a57633b-9175-4a8f-9e90-c8cbdf426cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:48.975489 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3164/6330] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:48.980307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee8fc-c0d0-479d-83d1-a8de486db44a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:48.992479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ee8fc-c0d0-479d-83d1-a8de486db44a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:48.998302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:48.999303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:48.999955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.000404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.001266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:49.001734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.004266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:49.026299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3167/6331] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:49.027252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57633b-9175-4a8f-9e90-c8cbdf426cf0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:49.039905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57633b-9175-4a8f-9e90-c8cbdf426cf0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:49.050385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:49.051304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.052103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.052632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.053499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.056659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:49.093591 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3165/6332] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:49 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:49.097962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:49.114518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b80311-6ae6-4e87-ad19-569b37557715 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:49.192753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:49.193413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.193700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.194008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.194679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.196724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:49.224127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3168/6333] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:49 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:49.229341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:49.240858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3507590-b50c-4501-90ff-830673dab1c9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:49.248517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:49.249208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.249561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.249954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.250674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.252347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:49.266783 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3166/6334] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:49.268225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:49.279939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fa55b-9eb7-467a-a165-b6da4a7a9e0c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:49.284669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1898f-7ad7-4d97-b516-dc2df49d306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:49.285282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1898f-7ad7-4d97-b516-dc2df49d306a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.285559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1898f-7ad7-4d97-b516-dc2df49d306a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.286079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1898f-7ad7-4d97-b516-dc2df49d306a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.286726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1898f-7ad7-4d97-b516-dc2df49d306a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.288329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1898f-7ad7-4d97-b516-dc2df49d306a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:49.314829 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3169/6335] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:49.325851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:49.337354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef2e0-9411-4cca-957f-7f24214944b0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:49.343122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:49.344045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.344328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.344602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] PATH_INFO: `/v3/projects/57f9046375fa4d0f93073253ada99152/users/9fe510eb66024366b08f76430dbabf9a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.358782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:49.363918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f9046375fa4d0f93073253ada99152, user_id=9fe510eb66024366b08f76430dbabf9a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.367942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:49.383079 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3167/6336] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:49 2022] PUT /identity/v3/projects/57f9046375fa4d0f93073253ada99152/users/9fe510eb66024366b08f76430dbabf9a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:49.396625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6636-3be8-4e49-8dbd-f11947e69ed2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.396932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6636-3be8-4e49-8dbd-f11947e69ed2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.397165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6636-3be8-4e49-8dbd-f11947e69ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.427982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-095c6636-3be8-4e49-8dbd-f11947e69ed2 None None] MFA Rules not processed for user `9fe510eb66024366b08f76430dbabf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:49.429109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-095c6636-3be8-4e49-8dbd-f11947e69ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:49.457510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:49.503917 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3170/6337] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:49.516256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:49.517975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:49.578511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8684689-89a6-499d-8258-9db66e974fec admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:49.582208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbf6bd-8a7b-4c42-b911-205049093502 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:49.582823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbf6bd-8a7b-4c42-b911-205049093502 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.583107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbf6bd-8a7b-4c42-b911-205049093502 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.583386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbf6bd-8a7b-4c42-b911-205049093502 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.601312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbf6bd-8a7b-4c42-b911-205049093502 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.602570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbf6bd-8a7b-4c42-b911-205049093502 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:49.612645 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3168/6338] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:49.886439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6636-3be8-4e49-8dbd-f11947e69ed2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:49.975405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6636-3be8-4e49-8dbd-f11947e69ed2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:49.983199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:49.984095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:49.984567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:49.985217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] PATH_INFO: `/v3/users/9fe510eb66024366b08f76430dbabf9a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:49.986839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe510eb66024366b08f76430dbabf9a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:49.989372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:50.053206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe510eb66024366b08f76430dbabf9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:50.068749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe510eb66024366b08f76430dbabf9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:50.078201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe510eb66024366b08f76430dbabf9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:50.090237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe510eb66024366b08f76430dbabf9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:50.091428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe510eb66024366b08f76430dbabf9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:50.099724 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3171/6339] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:49 2022] DELETE /identity/v3/users/9fe510eb66024366b08f76430dbabf9a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:50.105516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:50.106101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:50.106346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:50.129531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:50.130795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:50.205534 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3169/6340] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:50.215282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:50.216781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:50.281583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:50.286368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991224c-9334-4b63-bd45-f54087273cab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:50.292574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe949-cf0d-4857-9b43-fb4d35d307c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:50.293415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe949-cf0d-4857-9b43-fb4d35d307c2 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:50.293888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe949-cf0d-4857-9b43-fb4d35d307c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:50.294395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe949-cf0d-4857-9b43-fb4d35d307c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:50.305741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe949-cf0d-4857-9b43-fb4d35d307c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:50.308214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe949-cf0d-4857-9b43-fb4d35d307c2 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:50.318867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3172/6341] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:50.956298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:50.957777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:50.971960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a0cbe-466d-487a-913a-e5ae32dcbc90 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:50.979583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:50.980245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:50.980550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:50.980794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] PATH_INFO: `/v3/projects/57f9046375fa4d0f93073253ada99152` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:50.982105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f9046375fa4d0f93073253ada99152)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:50.986401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:51.049221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f9046375fa4d0f93073253ada99152', 'request_id': 'req-b537f71f-f464-44e0-93f7-9eaa53ef4537'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:51.058145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f9046375fa4d0f93073253ada99152', 'request_id': 'req-b537f71f-f464-44e0-93f7-9eaa53ef4537'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:51.069321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f9046375fa4d0f93073253ada99152', 'request_id': 'req-b537f71f-f464-44e0-93f7-9eaa53ef4537'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:51.092411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] The token cache is being invalidate because project 57f9046375fa4d0f93073253ada99152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:51.092833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:51.095843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b537f71f-f464-44e0-93f7-9eaa53ef4537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:51.106579 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3170/6342] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:50 2022] DELETE /identity/v3/projects/57f9046375fa4d0f93073253ada99152 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:51.110042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6cd7-2bcd-4088-b1b3-de8d6ae1d35c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.110904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6cd7-2bcd-4088-b1b3-de8d6ae1d35c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.111500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6cd7-2bcd-4088-b1b3-de8d6ae1d35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.149229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-185a6cd7-2bcd-4088-b1b3-de8d6ae1d35c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:51.151683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-185a6cd7-2bcd-4088-b1b3-de8d6ae1d35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:51.252751 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3173/6343] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:51.261154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2c29b-616c-4ae2-8e4e-a9f671651f76 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.261389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2c29b-616c-4ae2-8e4e-a9f671651f76 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.261592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2c29b-616c-4ae2-8e4e-a9f671651f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.284424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-98e2c29b-616c-4ae2-8e4e-a9f671651f76 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:51.285728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98e2c29b-616c-4ae2-8e4e-a9f671651f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:51.299210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6cd7-2bcd-4088-b1b3-de8d6ae1d35c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:51.299607 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3171/6344] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:51.313140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6cd7-2bcd-4088-b1b3-de8d6ae1d35c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:51.318925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:51.320167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.320988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.321273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.322139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:51.322451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:51.325070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:51.340762 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3174/6345] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:51.346042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2c29b-616c-4ae2-8e4e-a9f671651f76 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:51.357509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2c29b-616c-4ae2-8e4e-a9f671651f76 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:51.363552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:51.364165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.365004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.365156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.365705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:51.368125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:51.374478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:51.409927 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3172/6346] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:51 2022] POST /identity/v3/projects => generated 353 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:51.414254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:51.428825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9610a5-ac38-4b46-99f8-f0f2b0b3b83a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:51.579564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:51.580252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.580547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.580969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.581613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:51.584101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:51.629063 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3175/6347] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:51 2022] POST /identity/v3/users => generated 277 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:51.633180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:51.645476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87469d4b-84e3-4dc0-b512-f99f48bfae7f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:51.651028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:51.651744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.652025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.652299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.652942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:51.654613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:51.670710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:51.684304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07070b15-223e-4ac2-a9f3-45d68b767e56 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:51.685264 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3173/6348] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:51.694614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ef58d-c4d6-4394-8b0e-b4f9e8d8f66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:51.695415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ef58d-c4d6-4394-8b0e-b4f9e8d8f66d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.695558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ef58d-c4d6-4394-8b0e-b4f9e8d8f66d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.695822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ef58d-c4d6-4394-8b0e-b4f9e8d8f66d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.696756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ef58d-c4d6-4394-8b0e-b4f9e8d8f66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:51.698858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ef58d-c4d6-4394-8b0e-b4f9e8d8f66d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:51.715638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:51.723343 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3176/6349] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:13:51.727516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c0e7a-1299-4e5a-8941-3c5df91221b3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:51.734259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:51.735477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.735852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.736554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] PATH_INFO: `/v3/projects/844d3804d1d14a1eb8c79a6eb123df58/users/b611574cdbea4acaa81e87aed3247b74/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.754522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=844d3804d1d14a1eb8c79a6eb123df58, user_id=b611574cdbea4acaa81e87aed3247b74, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:51.758273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:51.784166 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3174/6350] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:51 2022] PUT /identity/v3/projects/844d3804d1d14a1eb8c79a6eb123df58/users/b611574cdbea4acaa81e87aed3247b74/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:51.795501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:51.795817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:51.796142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:51.833766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] MFA Rules not processed for user `b611574cdbea4acaa81e87aed3247b74`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:51.835135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:51.887595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:51.915403 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3177/6351] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:51.927207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:51.934319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:52.013949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ed81-1a35-4ba8-bcf4-7e88796f2faf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:52.018973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:52.019587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:52.019869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:52.020254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:52.029546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:52.031015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:52.043765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3175/6352] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:54.519486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:54.521220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:54.702551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9789b-e82f-43f1-8a87-fb1c37303285 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:54.709779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:54.710714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:54.711219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:54.712238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] PATH_INFO: `/v3/users/5346063245704d50b918a54c29bc297a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:54.714324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5346063245704d50b918a54c29bc297a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:54.716880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:54.791229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5346063245704d50b918a54c29bc297a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:54.815482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5346063245704d50b918a54c29bc297a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:54.831823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5346063245704d50b918a54c29bc297a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:54.844296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5346063245704d50b918a54c29bc297a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:54.845306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5346063245704d50b918a54c29bc297a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:54.852059 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3178/6353] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:54 2022] DELETE /identity/v3/users/5346063245704d50b918a54c29bc297a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:55.112401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:55.124246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f124d-b70f-40f0-afa8-fae4f94f8962 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:55.208915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:55.209996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:55.210289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:55.210607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] PATH_INFO: `/v3/projects/84a8662ae28642d1bd1f404c391ff23e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:55.211754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a8662ae28642d1bd1f404c391ff23e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:55.214587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:55.269391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a8662ae28642d1bd1f404c391ff23e', 'request_id': 'req-b7061b3c-4d4b-475a-85c6-55d97d9ede43'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.280433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a8662ae28642d1bd1f404c391ff23e', 'request_id': 'req-b7061b3c-4d4b-475a-85c6-55d97d9ede43'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.284565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:55.293148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a8662ae28642d1bd1f404c391ff23e', 'request_id': 'req-b7061b3c-4d4b-475a-85c6-55d97d9ede43'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.312333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] The token cache is being invalidate because project 84a8662ae28642d1bd1f404c391ff23e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:55.312630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.313309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.315621 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3176/6354] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:55 2022] DELETE /identity/v3/projects/84a8662ae28642d1bd1f404c391ff23e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:55.320805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:55.323033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:55.350829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae640a0c-9dc1-4ac2-adc4-9d21daf0a421 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:55.423620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:55.424260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:55.424543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:55.424818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] PATH_INFO: `/v3/users/a52844a1f8e142849762f7394dd0010a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:55.426005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52844a1f8e142849762f7394dd0010a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:55.427966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:55.433187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:55.447645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:55.449736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:55.467291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7061b3c-4d4b-475a-85c6-55d97d9ede43 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:55.473690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:55.474590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:55.474896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:55.475260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] PATH_INFO: `/v3/users/b611574cdbea4acaa81e87aed3247b74` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:55.476530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b611574cdbea4acaa81e87aed3247b74)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:55.480991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:55.492807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52844a1f8e142849762f7394dd0010a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.513701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52844a1f8e142849762f7394dd0010a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.524555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52844a1f8e142849762f7394dd0010a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.535118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52844a1f8e142849762f7394dd0010a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.535934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-886aa3f1-2148-42fb-a996-e3285772ca5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52844a1f8e142849762f7394dd0010a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.548438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3179/6355] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:55 2022] DELETE /identity/v3/users/a52844a1f8e142849762f7394dd0010a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:55.559820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b611574cdbea4acaa81e87aed3247b74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.570750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b611574cdbea4acaa81e87aed3247b74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.572876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b611574cdbea4acaa81e87aed3247b74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.583884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b611574cdbea4acaa81e87aed3247b74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.584836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b611574cdbea4acaa81e87aed3247b74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:55.628246 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3177/6356] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:55 2022] DELETE /identity/v3/users/b611574cdbea4acaa81e87aed3247b74 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:55.632723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:55.632961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:55.633224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:55.654682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:55.655989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:55.721418 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3180/6357] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:55.731630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:55.733029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:55.794783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e358-775a-4487-bf8f-1f98dec6062b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:55.798848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:55.799569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:55.799897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:55.800213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:55.810430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:55.812024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:55.823028 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3178/6358] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:55.927702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:55.929275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:55.942271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf06850-38f0-43ed-8e38-55dbc017ce27 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:55.955318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:55.955900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:55.956697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:55.956697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] PATH_INFO: `/v3/projects/284406fb5bca447e99e96da850338927` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:55.959755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=284406fb5bca447e99e96da850338927)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:55.962260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.027942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:56.029971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:56.031000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284406fb5bca447e99e96da850338927', 'request_id': 'req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.041969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284406fb5bca447e99e96da850338927', 'request_id': 'req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.051252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c2013-871e-4c1a-ba7e-db24cb7bacf7 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:56.053495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284406fb5bca447e99e96da850338927', 'request_id': 'req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.081572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] The token cache is being invalidate because project 284406fb5bca447e99e96da850338927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:56.082016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.083753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-50a88bcb-90b9-4644-918f-01aa6f2ad2eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.098128 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3181/6359] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:55 2022] DELETE /identity/v3/projects/284406fb5bca447e99e96da850338927 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:56.175359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:56.175949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.176226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.176517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] PATH_INFO: `/v3/projects/844d3804d1d14a1eb8c79a6eb123df58` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.177949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=844d3804d1d14a1eb8c79a6eb123df58)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:56.180013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.182293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:56.246737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844d3804d1d14a1eb8c79a6eb123df58', 'request_id': 'req-7de4d6d1-71e7-4217-9ad2-efb067b92283'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.256132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844d3804d1d14a1eb8c79a6eb123df58', 'request_id': 'req-7de4d6d1-71e7-4217-9ad2-efb067b92283'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.268787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844d3804d1d14a1eb8c79a6eb123df58', 'request_id': 'req-7de4d6d1-71e7-4217-9ad2-efb067b92283'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.285305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] The token cache is being invalidate because project 844d3804d1d14a1eb8c79a6eb123df58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:56.286017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.286952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7de4d6d1-71e7-4217-9ad2-efb067b92283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:56.288744 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3179/6360] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:56 2022] DELETE /identity/v3/projects/844d3804d1d14a1eb8c79a6eb123df58 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:56.294615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34705c14-a5ae-48dc-b5e4-f7791bbda1f4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.294910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34705c14-a5ae-48dc-b5e4-f7791bbda1f4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.295206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34705c14-a5ae-48dc-b5e4-f7791bbda1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.324282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:56.339439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-34705c14-a5ae-48dc-b5e4-f7791bbda1f4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:56.343587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34705c14-a5ae-48dc-b5e4-f7791bbda1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:56.480213 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3182/6361] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:13:56.480496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199cf77-b09e-42da-acfa-1d49fe6405a1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.480781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199cf77-b09e-42da-acfa-1d49fe6405a1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.481062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199cf77-b09e-42da-acfa-1d49fe6405a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.508903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7199cf77-b09e-42da-acfa-1d49fe6405a1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:56.510817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7199cf77-b09e-42da-acfa-1d49fe6405a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:56.520519 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3180/6362] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:56.524574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34705c14-a5ae-48dc-b5e4-f7791bbda1f4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:56.536789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34705c14-a5ae-48dc-b5e4-f7791bbda1f4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:56.542242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:56.543030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.543318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.543595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.544258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:13:56.544566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:56.546349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.563207 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3183/6363] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:13:56.564580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199cf77-b09e-42da-acfa-1d49fe6405a1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:56.600266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199cf77-b09e-42da-acfa-1d49fe6405a1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:56.609293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:56.609942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.610224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.610501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.611131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:56.613122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.647926 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3181/6364] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:56 2022] POST /identity/v3/projects => generated 353 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:13:56.652982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:56.664879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb64fdb-3f85-4756-8ace-8955a1a2cbc1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:56.733100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:56.733622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.733954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.734237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.734875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:56.738751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.767809 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3184/6365] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:56 2022] POST /identity/v3/users => generated 277 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:56.769249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:56.784829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f516f83-a2ae-44d7-93ec-db9332c29b69 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:56.791000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:56.791784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.792098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.792479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.793139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:56.794961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.811403 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3182/6366] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:56.811749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:56.824870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea553a2f-f9fe-4f08-a234-dfdc72115484 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:56.830495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015371a-03b2-47de-9036-b15c3cf3a055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:56.831234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015371a-03b2-47de-9036-b15c3cf3a055 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.831557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015371a-03b2-47de-9036-b15c3cf3a055 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.831836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015371a-03b2-47de-9036-b15c3cf3a055 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.832521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015371a-03b2-47de-9036-b15c3cf3a055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:56.834265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015371a-03b2-47de-9036-b15c3cf3a055 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.846343 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3185/6367] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:56.850476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:56.863968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef620-0ce6-41db-b312-b322cae72878 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:56.869043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:56.869898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.870279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.870554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] PATH_INFO: `/v3/projects/80ecf957cccc4dabac747ecd3984275a/users/890a8afdb00f4ada90900f0dce2f8b06/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.886506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ecf957cccc4dabac747ecd3984275a, user_id=890a8afdb00f4ada90900f0dce2f8b06, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:56.890878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:56.913369 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3183/6368] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:56 2022] PUT /identity/v3/projects/80ecf957cccc4dabac747ecd3984275a/users/890a8afdb00f4ada90900f0dce2f8b06/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:56.924197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f195f3-df94-46c7-9f1c-0bca419748f3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:56.924482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f195f3-df94-46c7-9f1c-0bca419748f3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:56.924760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f195f3-df94-46c7-9f1c-0bca419748f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:56.955296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-26f195f3-df94-46c7-9f1c-0bca419748f3 None None] MFA Rules not processed for user `890a8afdb00f4ada90900f0dce2f8b06`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:56.956442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-26f195f3-df94-46c7-9f1c-0bca419748f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:57.047195 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3186/6369] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:57.051614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:57.133110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f58b7a-201e-4937-84af-f466992332d5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:57.140657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:57.141357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:57.141702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:57.142138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:57.142900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:57.148370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:57.187337 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3184/6370] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:13:57 2022] POST /identity/v3/projects => generated 355 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:57.189057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f195f3-df94-46c7-9f1c-0bca419748f3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:57.194994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:57.203403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f195f3-df94-46c7-9f1c-0bca419748f3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:57.287291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:57.288025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:57.288438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:57.288863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:57.289595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:57.292803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:57.320782 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3187/6371] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:13:57 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:13:57.325519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:57.338063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecae7b-819b-40dd-860c-f315b34ea740 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:57.343921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:57.344691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:57.345090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:57.345367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:57.346057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:57.347655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:57.355366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:57.365679 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3185/6372] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:57.370473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:57.383464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccf59b-5167-4317-a23e-ef512e2745c3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:57.389159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:57.389841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:57.390217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:57.390705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] PATH_INFO: `/v3/projects/3bae822e47d44d16b804ad6ed0f4f78d/users/34aeebfafa12436d97b782e504a4b07d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:57.438906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bae822e47d44d16b804ad6ed0f4f78d, user_id=34aeebfafa12436d97b782e504a4b07d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:57.442112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:57.460221 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3188/6373] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:57 2022] PUT /identity/v3/projects/3bae822e47d44d16b804ad6ed0f4f78d/users/34aeebfafa12436d97b782e504a4b07d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:57.467321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:57.500766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1d31c-606c-40a4-83e8-9b1a0cfe76e6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:57.627937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d829ca4-d235-4d17-9d0a-4ff3c76781c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:57.629318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d829ca4-d235-4d17-9d0a-4ff3c76781c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:57.629673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d829ca4-d235-4d17-9d0a-4ff3c76781c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:57.630079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d829ca4-d235-4d17-9d0a-4ff3c76781c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:57.630843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d829ca4-d235-4d17-9d0a-4ff3c76781c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:57.632751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d829ca4-d235-4d17-9d0a-4ff3c76781c4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:57.666820 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3186/6374] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:13:57.667802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:57.681411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57536dd1-b1b2-4dd7-90e2-a5c7f0e75f1c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:57.707488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:57.708398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:57.708551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:57.708824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] PATH_INFO: `/v3/projects/3bae822e47d44d16b804ad6ed0f4f78d/users/34aeebfafa12436d97b782e504a4b07d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:57.713297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bae822e47d44d16b804ad6ed0f4f78d, user_id=34aeebfafa12436d97b782e504a4b07d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:57.716199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:57.740638 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3189/6375] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:13:57 2022] PUT /identity/v3/projects/3bae822e47d44d16b804ad6ed0f4f78d/users/34aeebfafa12436d97b782e504a4b07d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:57.753640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085b58-1a77-45d2-952f-ed9e3a810e58 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:57.753955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085b58-1a77-45d2-952f-ed9e3a810e58 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:57.754238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff085b58-1a77-45d2-952f-ed9e3a810e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:57.806406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ff085b58-1a77-45d2-952f-ed9e3a810e58 None None] MFA Rules not processed for user `34aeebfafa12436d97b782e504a4b07d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:57.808067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ff085b58-1a77-45d2-952f-ed9e3a810e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:57.925549 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3187/6376] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:13:57.938915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:57.941077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:58.020765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ad388-4a70-4360-8ac1-2262f7413935 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:58.024482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:58.025837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:58.026536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:58.026853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:58.041552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:58.043237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:58.053940 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3190/6377] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:13:58.157568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085b58-1a77-45d2-952f-ed9e3a810e58 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:58.182371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff085b58-1a77-45d2-952f-ed9e3a810e58 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:58.189144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:58.190042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:58.190423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:58.190702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:58.265268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:58.266771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:58.277143 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3188/6378] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:13:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:13:58.816554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:58.818289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:58.881899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:58.899163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e5fcc-be8d-426b-bfef-fb5c42db3a83 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:58.904734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:58.905372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:58.905651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:58.905957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] PATH_INFO: `/v3/users/86fa572828244998b44eb39d9330da7e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:58.907079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fa572828244998b44eb39d9330da7e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:58.909174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:58.973803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fa572828244998b44eb39d9330da7e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:58.994445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fa572828244998b44eb39d9330da7e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.002630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fa572828244998b44eb39d9330da7e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.013108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fa572828244998b44eb39d9330da7e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.014037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fa572828244998b44eb39d9330da7e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.022131 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3191/6379] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:58 2022] DELETE /identity/v3/users/86fa572828244998b44eb39d9330da7e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:59.217215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:59.230071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c4de8-1c59-4d43-b758-fa7ee1e51c1a service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:59.253249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:59.298511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:59.299316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:59.300263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:59.300520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] PATH_INFO: `/v3/projects/861e58c5bee84a44be5045e10b4594a1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:59.301976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] RBAC: Authorizing `identity:delete_project(project_id=861e58c5bee84a44be5045e10b4594a1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:59.303964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:59.362802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861e58c5bee84a44be5045e10b4594a1', 'request_id': 'req-951d3849-bd38-4cb6-90c9-5695c770e37b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.369940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861e58c5bee84a44be5045e10b4594a1', 'request_id': 'req-951d3849-bd38-4cb6-90c9-5695c770e37b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.381820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861e58c5bee84a44be5045e10b4594a1', 'request_id': 'req-951d3849-bd38-4cb6-90c9-5695c770e37b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.395044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] The token cache is being invalidate because project 861e58c5bee84a44be5045e10b4594a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:59.395394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.396236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.399618 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3189/6380] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:59 2022] DELETE /identity/v3/projects/861e58c5bee84a44be5045e10b4594a1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:59.403382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:59.404103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:59.418978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562d554-05b7-4d2b-8e13-2663102c0027 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:59.476607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:59.477242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:59.477525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:59.477802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] PATH_INFO: `/v3/users/d14a68a648cf45e1b929d0abbf039b8a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:59.480178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14a68a648cf45e1b929d0abbf039b8a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:59.482199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:59.538438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14a68a648cf45e1b929d0abbf039b8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.559424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14a68a648cf45e1b929d0abbf039b8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.564591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:59.574951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14a68a648cf45e1b929d0abbf039b8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.590413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14a68a648cf45e1b929d0abbf039b8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.591207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-76bee9ce-f008-4424-9435-3c3961383d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14a68a648cf45e1b929d0abbf039b8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.622455 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3192/6381] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:13:59 2022] DELETE /identity/v3/users/d14a68a648cf45e1b929d0abbf039b8a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:13:59.809002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:13:59.810412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:13:59.826284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d3849-bd38-4cb6-90c9-5695c770e37b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:13:59.885848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:13:59.886489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:59.886766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:59.887035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] PATH_INFO: `/v3/projects/ca87d496227d4cf4bccf1bdd2b7aeeae` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:59.888142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca87d496227d4cf4bccf1bdd2b7aeeae)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:13:59.890061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:13:59.935623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca87d496227d4cf4bccf1bdd2b7aeeae', 'request_id': 'req-96b2d076-01ba-4cdf-ad57-ebf83056e162'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.937493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:13:59.942734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca87d496227d4cf4bccf1bdd2b7aeeae', 'request_id': 'req-96b2d076-01ba-4cdf-ad57-ebf83056e162'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.950710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca87d496227d4cf4bccf1bdd2b7aeeae', 'request_id': 'req-96b2d076-01ba-4cdf-ad57-ebf83056e162'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.963858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] The token cache is being invalidate because project ca87d496227d4cf4bccf1bdd2b7aeeae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:13:59.964384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.965063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-96b2d076-01ba-4cdf-ad57-ebf83056e162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:13:59.966605 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3190/6382] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:13:59 2022] DELETE /identity/v3/projects/ca87d496227d4cf4bccf1bdd2b7aeeae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:13:59.972379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c6b0b-ea9b-4d2c-8938-7ada637dd5a9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:13:59.972830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c6b0b-ea9b-4d2c-8938-7ada637dd5a9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:13:59.972935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c6b0b-ea9b-4d2c-8938-7ada637dd5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:13:59.994091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ca4c6b0b-ea9b-4d2c-8938-7ada637dd5a9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:13:59.995438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ca4c6b0b-ea9b-4d2c-8938-7ada637dd5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:00.055224 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3193/6383] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:00.062939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0d14-d880-43fd-adaf-09b04abb271e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.063175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0d14-d880-43fd-adaf-09b04abb271e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.063458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0d14-d880-43fd-adaf-09b04abb271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.085079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d62e0d14-d880-43fd-adaf-09b04abb271e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:00.086815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d62e0d14-d880-43fd-adaf-09b04abb271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:00.096241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3191/6384] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:00.100585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c6b0b-ea9b-4d2c-8938-7ada637dd5a9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.111612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c6b0b-ea9b-4d2c-8938-7ada637dd5a9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.116206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.116786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.117068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.117339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.118020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:14:00.118322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.120178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.136422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3194/6385] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:14:00.137604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0d14-d880-43fd-adaf-09b04abb271e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.147894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0d14-d880-43fd-adaf-09b04abb271e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.153384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.154018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.154329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.154604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.155238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.157266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.189878 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3192/6386] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:00 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:14:00.192698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.203445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395c9fd-07eb-4e60-b46b-c0afcad3f2c7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.270251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.270889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.271172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.271446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.272079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.274078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.295738 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3195/6387] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:00 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:00.299806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.311877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6e5d-f7b8-4a1c-893e-85745b2d1d27 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.316773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.317326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.317609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.317917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.318557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.320151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.331681 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3193/6388] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:00.335869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.346455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8847b6-2fce-44ea-9b79-bc67676b7384 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.351280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658b0e0-7af3-4e43-beab-1b79b66183ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.351887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b0e0-7af3-4e43-beab-1b79b66183ec admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.352171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b0e0-7af3-4e43-beab-1b79b66183ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.352447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658b0e0-7af3-4e43-beab-1b79b66183ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.353229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658b0e0-7af3-4e43-beab-1b79b66183ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.355146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658b0e0-7af3-4e43-beab-1b79b66183ec admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.369631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3196/6389] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:00.370988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.383944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e3ae1-8294-4a43-955e-8deeda989214 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.388934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.389581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.389887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.390158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] PATH_INFO: `/v3/projects/d2931f7a57c6444db783288bda1a74b8/users/9bbe85c6ecdb49a8b186a8cb6e1f400e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.406203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2931f7a57c6444db783288bda1a74b8, user_id=9bbe85c6ecdb49a8b186a8cb6e1f400e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.408841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.423911 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3194/6390] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:00 2022] PUT /identity/v3/projects/d2931f7a57c6444db783288bda1a74b8/users/9bbe85c6ecdb49a8b186a8cb6e1f400e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:00.429178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.429498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.429965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.450018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:00.451353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:00.507262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:00.525359 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3197/6391] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:00.538841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.541482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:00.618248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8068f6-2789-4ea1-a93a-e3efabc33f80 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.623975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e069d-d4cc-4871-a99c-3e88f89fcc5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.623975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e069d-d4cc-4871-a99c-3e88f89fcc5f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.623975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e069d-d4cc-4871-a99c-3e88f89fcc5f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.623975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e069d-d4cc-4871-a99c-3e88f89fcc5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.636922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e069d-d4cc-4871-a99c-3e88f89fcc5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.638491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e069d-d4cc-4871-a99c-3e88f89fcc5f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.658208 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3195/6392] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:00.783211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:00.785179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:00.800939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924765f-f604-4d9a-9cde-3863231ed5cb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:00.806919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57c290-b7b9-4956-a100-b5d1184c23c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:00.807583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57c290-b7b9-4956-a100-b5d1184c23c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:00.807864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57c290-b7b9-4956-a100-b5d1184c23c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:00.808141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57c290-b7b9-4956-a100-b5d1184c23c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:00.810358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b57c290-b7b9-4956-a100-b5d1184c23c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:00.822241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57c290-b7b9-4956-a100-b5d1184c23c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:00.823901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57c290-b7b9-4956-a100-b5d1184c23c4 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:00.847644 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3198/6393] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:07.534764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:07.535268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:07.535268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:07.598820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] MFA Rules not processed for user `9bbe85c6ecdb49a8b186a8cb6e1f400e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:07.601000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:07.837904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3196/6394] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:07.840100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee585c-62eb-463e-b119-938fca9f3320 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:07.840591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee585c-62eb-463e-b119-938fca9f3320 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:07.840668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee585c-62eb-463e-b119-938fca9f3320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:07.884862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-eeee585c-62eb-463e-b119-938fca9f3320 None None] MFA Rules not processed for user `9bbe85c6ecdb49a8b186a8cb6e1f400e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:07.886572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eeee585c-62eb-463e-b119-938fca9f3320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:07.914155 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3199/6395] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:14:07.921954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:07.923399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:07.937415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:07.953032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:07.953791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:07.954330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:08.002034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] MFA Rules not processed for user `54dcc78ee39c4119b08023d3cc692d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:08.006751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:08.097234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fe930-49b5-44c5-ae88-e646ad766726 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:08.105790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3200/6396] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:14:08.109005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:08.122802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:08.122802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:08.122802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:08.129069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:08.131854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:08.163489 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3197/6397] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:08.820897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:08.823895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:08.853961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:08.856252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:08.956078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718ed85-6c33-4466-87ca-df6d7f83791e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:08.961491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:08.962765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:08.963062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:08.963306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:08.965013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:08.978788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:08.980661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:08.985730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb75453-74e6-40ae-904d-5829fb572c5e service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:08.991976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:08.992715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:08.993072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:08.993922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:08.994086 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3201/6398] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:09.004529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:09.006289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:09.020748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3198/6399] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:12.805496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:12.810808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:12.897498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5eac4-e7eb-4c56-9101-366350b94dbc service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:12.911837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:12.913080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:12.913701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:12.914636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:12.918792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:12.959435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:12.964392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:13.005031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3202/6400] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:13.386928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:13.400597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcad69-b914-4329-b12a-5455c6bdaca8 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:13.406890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c14e06-0e91-4097-a2b7-ae8d685e7309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:13.407510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c14e06-0e91-4097-a2b7-ae8d685e7309 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:13.407794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c14e06-0e91-4097-a2b7-ae8d685e7309 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:13.408072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c14e06-0e91-4097-a2b7-ae8d685e7309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:13.409973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-01c14e06-0e91-4097-a2b7-ae8d685e7309 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:13.425259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c14e06-0e91-4097-a2b7-ae8d685e7309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:13.427470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c14e06-0e91-4097-a2b7-ae8d685e7309 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:13.445032 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3199/6401] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:14:15.782410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:15.783795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:15.797393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c646316-3902-46ae-8cc9-474aa7db0b7a service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:15.801829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:15.802466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:15.802743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:15.803015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] PATH_INFO: `/v3/users/890a8afdb00f4ada90900f0dce2f8b06` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:15.804110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=890a8afdb00f4ada90900f0dce2f8b06)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:15.806852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:15.841163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:15.855096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '890a8afdb00f4ada90900f0dce2f8b06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:15.867050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '890a8afdb00f4ada90900f0dce2f8b06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:15.874627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '890a8afdb00f4ada90900f0dce2f8b06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:15.881962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '890a8afdb00f4ada90900f0dce2f8b06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:15.882590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '890a8afdb00f4ada90900f0dce2f8b06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:15.888566 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3203/6402] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:15 2022] DELETE /identity/v3/users/890a8afdb00f4ada90900f0dce2f8b06 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:15.893153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:15.893606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:15.893739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:15.914098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:15.916159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:15.974174 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3200/6403] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:15.984878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:15.986319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:16.048970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332e64-0e2c-4e7e-ba6b-b326d38c3f11 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:16.052617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:16.053220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:16.053501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:16.053776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:16.062901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:16.064342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:16.074320 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3204/6404] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:16.496843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:16.498308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:16.511656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ebb6-0073-429b-ae28-d77515da1f41 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:16.517748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:16.518445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:16.518728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:16.519003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] PATH_INFO: `/v3/projects/80ecf957cccc4dabac747ecd3984275a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:16.520546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ecf957cccc4dabac747ecd3984275a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:16.522523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:16.530976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:16.596509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ecf957cccc4dabac747ecd3984275a', 'request_id': 'req-31c7337d-f31d-4d8e-b052-7c2b06279f93'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.604867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ecf957cccc4dabac747ecd3984275a', 'request_id': 'req-31c7337d-f31d-4d8e-b052-7c2b06279f93'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.615357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ecf957cccc4dabac747ecd3984275a', 'request_id': 'req-31c7337d-f31d-4d8e-b052-7c2b06279f93'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.628676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] The token cache is being invalidate because project 80ecf957cccc4dabac747ecd3984275a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:16.628992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.629644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.650062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3201/6405] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:16 2022] DELETE /identity/v3/projects/80ecf957cccc4dabac747ecd3984275a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:16.652303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:16.674318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea39e-3aba-4322-a027-8780c0f8a976 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:16.744654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:16.745280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:16.746256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:16.746256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] PATH_INFO: `/v3/users/34aeebfafa12436d97b782e504a4b07d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:16.747341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34aeebfafa12436d97b782e504a4b07d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:16.749856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:16.812942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34aeebfafa12436d97b782e504a4b07d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.823015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:16.831807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34aeebfafa12436d97b782e504a4b07d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.839475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34aeebfafa12436d97b782e504a4b07d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.846680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34aeebfafa12436d97b782e504a4b07d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.847330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ec8039d2-f9f7-4b20-a189-999556358436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34aeebfafa12436d97b782e504a4b07d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:16.854018 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3205/6406] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:16 2022] DELETE /identity/v3/users/34aeebfafa12436d97b782e504a4b07d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:17.149578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.151371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:17.169000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7337d-f31d-4d8e-b052-7c2b06279f93 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.228074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.228766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.229063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.229633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] PATH_INFO: `/v3/projects/3bae822e47d44d16b804ad6ed0f4f78d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.230866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bae822e47d44d16b804ad6ed0f4f78d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.232775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:17.279246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bae822e47d44d16b804ad6ed0f4f78d', 'request_id': 'req-1e02afb4-0aa0-4a06-9046-14eee425acc1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:17.285353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bae822e47d44d16b804ad6ed0f4f78d', 'request_id': 'req-1e02afb4-0aa0-4a06-9046-14eee425acc1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:17.294780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bae822e47d44d16b804ad6ed0f4f78d', 'request_id': 'req-1e02afb4-0aa0-4a06-9046-14eee425acc1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:17.305454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] The token cache is being invalidate because project 3bae822e47d44d16b804ad6ed0f4f78d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:17.305986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:17.306535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e02afb4-0aa0-4a06-9046-14eee425acc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:17.308019 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3202/6407] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:17 2022] DELETE /identity/v3/projects/3bae822e47d44d16b804ad6ed0f4f78d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:17.313766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91519d7-2ea8-4eb9-88fe-2562021ba81f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.314089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91519d7-2ea8-4eb9-88fe-2562021ba81f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.314370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91519d7-2ea8-4eb9-88fe-2562021ba81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.334621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c91519d7-2ea8-4eb9-88fe-2562021ba81f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:17.335813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c91519d7-2ea8-4eb9-88fe-2562021ba81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:17.395322 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3206/6408] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:17.403692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d285b14-4157-4085-96ac-0aabc5bd321b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.403979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d285b14-4157-4085-96ac-0aabc5bd321b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.404256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d285b14-4157-4085-96ac-0aabc5bd321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.424331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9d285b14-4157-4085-96ac-0aabc5bd321b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:17.425530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9d285b14-4157-4085-96ac-0aabc5bd321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:17.434065 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3203/6409] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:17.442214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91519d7-2ea8-4eb9-88fe-2562021ba81f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.452814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91519d7-2ea8-4eb9-88fe-2562021ba81f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.457570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.458201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.458488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.458770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.459416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:14:17.459722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.461342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:17.473328 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3207/6410] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:14:17.478681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d285b14-4157-4085-96ac-0aabc5bd321b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.488167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d285b14-4157-4085-96ac-0aabc5bd321b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.492900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.494327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.494608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.494880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.495608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.497509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:17.514434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:17.527432 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3204/6411] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:17 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:17.531827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.542077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44105cd-4abc-4205-bb37-f69ad12568fd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.599025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.599623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.599904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.600180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.600815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.603039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:17.624710 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3208/6412] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:17 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:14:17.628950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.640782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9d1c-9cf5-4e59-ac71-4636fa6c8e51 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.645542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.646277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.646563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.646843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.647598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.649231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:17.662563 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3205/6413] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:14:17.663820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.676520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e23751-af7e-4ca6-be91-49876afeaac8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.682407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b988f-4f65-4476-94c8-ac559353c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.683026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b988f-4f65-4476-94c8-ac559353c968 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.683307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b988f-4f65-4476-94c8-ac559353c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.683577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b988f-4f65-4476-94c8-ac559353c968 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.684273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b988f-4f65-4476-94c8-ac559353c968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.686069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b988f-4f65-4476-94c8-ac559353c968 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:17.701518 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3209/6414] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:17.702976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.714128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a3827-412f-4878-ba77-9175fc381dcb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.721280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.721996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.722403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.722690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] PATH_INFO: `/v3/projects/5bb3d1fd19be49578edffd7252647724/users/b920e117d949441c94be4c9296978165/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.738581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb3d1fd19be49578edffd7252647724, user_id=b920e117d949441c94be4c9296978165, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.741209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:17.757413 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3206/6415] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:17 2022] PUT /identity/v3/projects/5bb3d1fd19be49578edffd7252647724/users/b920e117d949441c94be4c9296978165/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:17.770054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a09d9-8a19-4c99-80ee-79a6c9a6892c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.770339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a09d9-8a19-4c99-80ee-79a6c9a6892c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.770610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a09d9-8a19-4c99-80ee-79a6c9a6892c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.803730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cf6a09d9-8a19-4c99-80ee-79a6c9a6892c None None] MFA Rules not processed for user `b920e117d949441c94be4c9296978165`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:17.804979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cf6a09d9-8a19-4c99-80ee-79a6c9a6892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:17.840007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:17.889244 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3210/6416] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:14:17.893648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:17.969681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145f0c4-2491-43e7-b13e-beb92a2a364d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:17.975727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:17.976642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:17.976933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:17.977211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:17.978414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:17.981321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:18.020617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3207/6417] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:17 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:18.026096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a09d9-8a19-4c99-80ee-79a6c9a6892c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:18.037366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a09d9-8a19-4c99-80ee-79a6c9a6892c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:18.108323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:18.108943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.109276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.109549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.110216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:18.112124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:18.138537 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3211/6418] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:18 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:18.143078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:18.157461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d1447-b2f0-4d74-beac-618e063e02ab admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:18.165940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:18.166870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.167322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.167757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.168793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:18.171513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:18.185151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3208/6419] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:14:18.189707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:18.200247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754558f-7d90-473c-825a-99b9fe0b7134 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:18.205341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:18.206041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.206326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.206609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] PATH_INFO: `/v3/projects/392bd5c088c84a20819c0e97df33f215/users/4490a0307c5b48a0b4caf9d29d7ea726/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.223531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392bd5c088c84a20819c0e97df33f215, user_id=4490a0307c5b48a0b4caf9d29d7ea726, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:18.226331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:18.242625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3212/6420] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:18 2022] PUT /identity/v3/projects/392bd5c088c84a20819c0e97df33f215/users/4490a0307c5b48a0b4caf9d29d7ea726/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:18.247740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:18.263120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66e93b-b1f6-4b9c-aaa2-941ccbf73440 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:18.328490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d1a75-d712-43fe-9092-1f83bcd02559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:18.329099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d1a75-d712-43fe-9092-1f83bcd02559 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.329380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d1a75-d712-43fe-9092-1f83bcd02559 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.329656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d1a75-d712-43fe-9092-1f83bcd02559 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.330329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d1a75-d712-43fe-9092-1f83bcd02559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:18.332959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d1a75-d712-43fe-9092-1f83bcd02559 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:18.346705 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3209/6421] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:18.349949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:18.363800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c61440-e6f9-4913-8065-3eda069cf0d4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:18.371911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:18.372916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.373403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.373688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] PATH_INFO: `/v3/projects/392bd5c088c84a20819c0e97df33f215/users/4490a0307c5b48a0b4caf9d29d7ea726/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.375774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392bd5c088c84a20819c0e97df33f215, user_id=4490a0307c5b48a0b4caf9d29d7ea726, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:18.378710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:18.389920 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3213/6422] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:18 2022] PUT /identity/v3/projects/392bd5c088c84a20819c0e97df33f215/users/4490a0307c5b48a0b4caf9d29d7ea726/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:18.402212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.402502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.402781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.433783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] MFA Rules not processed for user `4490a0307c5b48a0b4caf9d29d7ea726`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:18.435216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:18.457784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:18.528472 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3210/6423] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:18.541760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:18.543970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:18.608778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1fce7-bb28-4b7d-b6f9-448df8bfd813 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:18.612647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:18.613301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.613594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.613940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.625182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:18.626741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:18.637602 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3214/6424] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:18.840746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:18.842314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:18.982951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c22b4d-fe18-4ce7-85ee-5ecc0cac10d8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:18.989102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498e7c2-b34d-4e96-b31f-17fa1fc4b740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:18.989678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e7c2-b34d-4e96-b31f-17fa1fc4b740 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:18.989993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e7c2-b34d-4e96-b31f-17fa1fc4b740 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:18.990273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e7c2-b34d-4e96-b31f-17fa1fc4b740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:18.994570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c498e7c2-b34d-4e96-b31f-17fa1fc4b740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:19.014089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498e7c2-b34d-4e96-b31f-17fa1fc4b740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:19.019174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498e7c2-b34d-4e96-b31f-17fa1fc4b740 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:19.034003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3211/6425] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:20.167368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:20.204517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:20.252766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9818c-1bd7-40e4-83cd-17f18737e495 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:20.257992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:20.259481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:20.259768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:20.260046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] PATH_INFO: `/v3/users/b920e117d949441c94be4c9296978165` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:20.261241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b920e117d949441c94be4c9296978165)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:20.263223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:20.308573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b920e117d949441c94be4c9296978165'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.320391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b920e117d949441c94be4c9296978165'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.327984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b920e117d949441c94be4c9296978165'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.334684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b920e117d949441c94be4c9296978165'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.335731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b920e117d949441c94be4c9296978165'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.342751 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3215/6426] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:20 2022] DELETE /identity/v3/users/b920e117d949441c94be4c9296978165 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:20.347716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:20.348163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:20.348271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:20.368304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:20.369614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:20.436849 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3212/6427] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:20.449152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:20.450627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:20.514785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646822f-bc81-4354-a5f6-7fbe9a03a427 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:20.518602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:20.519246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:20.519545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:20.519816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:20.530670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:20.532115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:20.551162 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3216/6428] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:20.852981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:20.855079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:20.871308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fbd51-d67c-4008-972e-01606da5e960 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:20.876589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:20.877232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:20.877516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:20.877814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] PATH_INFO: `/v3/projects/5bb3d1fd19be49578edffd7252647724` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:20.878979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb3d1fd19be49578edffd7252647724)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:20.880907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:20.920344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:20.929998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb3d1fd19be49578edffd7252647724', 'request_id': 'req-60c77d4f-81b8-4384-a98e-9019610948e7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.937193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb3d1fd19be49578edffd7252647724', 'request_id': 'req-60c77d4f-81b8-4384-a98e-9019610948e7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.944669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb3d1fd19be49578edffd7252647724', 'request_id': 'req-60c77d4f-81b8-4384-a98e-9019610948e7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.955517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] The token cache is being invalidate because project 5bb3d1fd19be49578edffd7252647724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:20.955812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.956436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:20.957949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3213/6429] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:20 2022] DELETE /identity/v3/projects/5bb3d1fd19be49578edffd7252647724 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:20.962065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:20.972029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316516c-308a-476d-8da6-de535b11b2ad service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:21.017859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:21.034444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:21.035087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:21.035373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:21.035678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] PATH_INFO: `/v3/users/4490a0307c5b48a0b4caf9d29d7ea726` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:21.036828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4490a0307c5b48a0b4caf9d29d7ea726)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:21.038831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:21.086386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4490a0307c5b48a0b4caf9d29d7ea726'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.099208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4490a0307c5b48a0b4caf9d29d7ea726'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.106839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4490a0307c5b48a0b4caf9d29d7ea726'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.114849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4490a0307c5b48a0b4caf9d29d7ea726'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.115958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b1a98ec7-3186-496e-a5be-9a085b67b70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4490a0307c5b48a0b4caf9d29d7ea726'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.121932 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3217/6430] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:20 2022] DELETE /identity/v3/users/4490a0307c5b48a0b4caf9d29d7ea726 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:21.513342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:21.514898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:21.530797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c77d4f-81b8-4384-a98e-9019610948e7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:21.598756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:21.600141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:21.600141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:21.600141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] PATH_INFO: `/v3/projects/392bd5c088c84a20819c0e97df33f215` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:21.602296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392bd5c088c84a20819c0e97df33f215)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:21.605169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:21.674138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392bd5c088c84a20819c0e97df33f215', 'request_id': 'req-71e7ea10-be81-4bfe-a87b-718812aa1c25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.682824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392bd5c088c84a20819c0e97df33f215', 'request_id': 'req-71e7ea10-be81-4bfe-a87b-718812aa1c25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.696791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392bd5c088c84a20819c0e97df33f215', 'request_id': 'req-71e7ea10-be81-4bfe-a87b-718812aa1c25'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.710199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] The token cache is being invalidate because project 392bd5c088c84a20819c0e97df33f215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:21.710590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.711323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71e7ea10-be81-4bfe-a87b-718812aa1c25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:21.713455 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3214/6431] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:21 2022] DELETE /identity/v3/projects/392bd5c088c84a20819c0e97df33f215 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:21.719288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59572dbf-2111-4341-bc17-7168b9428265 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:21.719811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59572dbf-2111-4341-bc17-7168b9428265 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:21.719912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59572dbf-2111-4341-bc17-7168b9428265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:21.757477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-59572dbf-2111-4341-bc17-7168b9428265 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:21.759270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59572dbf-2111-4341-bc17-7168b9428265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:21.825518 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3218/6432] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:14:21.833329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15999c-167e-41cd-bec3-1450b279286b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:21.833627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15999c-167e-41cd-bec3-1450b279286b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:21.833964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15999c-167e-41cd-bec3-1450b279286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:21.855155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ae15999c-167e-41cd-bec3-1450b279286b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:21.856412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ae15999c-167e-41cd-bec3-1450b279286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:21.864750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3215/6433] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:21.868797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59572dbf-2111-4341-bc17-7168b9428265 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:21.878880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59572dbf-2111-4341-bc17-7168b9428265 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:21.884927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:21.885533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:21.885817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:21.886208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:21.886863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:14:21.887177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:21.888842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:21.901461 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3219/6434] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:14:21.905728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15999c-167e-41cd-bec3-1450b279286b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:21.915490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15999c-167e-41cd-bec3-1450b279286b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:21.920306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:21.920902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:21.921185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:21.921460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:21.922159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:21.924062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:21.959248 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3216/6435] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:21 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:14:21.960364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:21.978458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220c92-30cf-43ff-ba97-f888b6d378c3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.029788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:22.041641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.042291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.042576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.042854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.043494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.045435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.069227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3220/6436] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:21 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:14:22.075125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.087605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ca8ff-1edb-4d2c-960b-ee81f59170dd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.092263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.092873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.093157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.093433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.094098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.095710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.106584 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3217/6437] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:22.110540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.121245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ebe8d-b309-4069-9af8-f9bef8223bda admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.128055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4165c04-b8c5-40af-a419-69d2992dd688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.128659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165c04-b8c5-40af-a419-69d2992dd688 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.129375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165c04-b8c5-40af-a419-69d2992dd688 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.129375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165c04-b8c5-40af-a419-69d2992dd688 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.129876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4165c04-b8c5-40af-a419-69d2992dd688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.131475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4165c04-b8c5-40af-a419-69d2992dd688 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.145867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3221/6438] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:22.147558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.158145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58e93f-fa78-410b-ac04-e0aa71ae08d5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.163029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.163682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.163959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.164231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] PATH_INFO: `/v3/projects/38cbd58e2e324b7087b5866685583c1e/users/e18e2a84d5864a0e86023af0f72e13c8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.177271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:22.181218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cbd58e2e324b7087b5866685583c1e, user_id=e18e2a84d5864a0e86023af0f72e13c8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.184534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.200781 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3218/6439] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:22 2022] PUT /identity/v3/projects/38cbd58e2e324b7087b5866685583c1e/users/e18e2a84d5864a0e86023af0f72e13c8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:22.212331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65840b9b-a40d-4376-9e28-a8ecbbbd302b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.212617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65840b9b-a40d-4376-9e28-a8ecbbbd302b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.212914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65840b9b-a40d-4376-9e28-a8ecbbbd302b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.248019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-65840b9b-a40d-4376-9e28-a8ecbbbd302b None None] MFA Rules not processed for user `e18e2a84d5864a0e86023af0f72e13c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:22.249324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65840b9b-a40d-4376-9e28-a8ecbbbd302b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:22.319586 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3222/6440] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:22.324666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.393909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599b991-b6ab-42c2-8500-5ff02d55299c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.398766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.399994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.400298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.400577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.401228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.403261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.432847 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3219/6441] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:22 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:22.437306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65840b9b-a40d-4376-9e28-a8ecbbbd302b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.450691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65840b9b-a40d-4376-9e28-a8ecbbbd302b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.515182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.515815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.516101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.516378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.517161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.519196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.548464 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3223/6442] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:22 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:22.549671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.562644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912a46b-5c1c-4e82-806d-40243a7cb33d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.568594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.569206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.569523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.569868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.570664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.572821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.583986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3220/6443] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:22.588548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.605930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478f5ed-c49c-44ad-bf2d-c3ec7fb969a8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.611376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.612075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.612361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.612643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] PATH_INFO: `/v3/projects/8c2a6c910129406a9fc540c0db62cd40/users/66d33119c79d420f8761faf794a84b84/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.629743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2a6c910129406a9fc540c0db62cd40, user_id=66d33119c79d420f8761faf794a84b84, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.632464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.646816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3224/6444] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:22 2022] PUT /identity/v3/projects/8c2a6c910129406a9fc540c0db62cd40/users/66d33119c79d420f8761faf794a84b84/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:14:22.651817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.662928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e27a0-d659-4a17-b49c-e858c8b23a5b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.721718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589425d9-5a72-48dd-a88d-9e8ce7947e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.722367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589425d9-5a72-48dd-a88d-9e8ce7947e89 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.722652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589425d9-5a72-48dd-a88d-9e8ce7947e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.722929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589425d9-5a72-48dd-a88d-9e8ce7947e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.723563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589425d9-5a72-48dd-a88d-9e8ce7947e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.725157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589425d9-5a72-48dd-a88d-9e8ce7947e89 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.736251 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3221/6445] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:22.740332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.752015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0e98d-ba5f-4af0-a8d7-683779c839e3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:22.757011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:22.757715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.758027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.758299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] PATH_INFO: `/v3/projects/8c2a6c910129406a9fc540c0db62cd40/users/66d33119c79d420f8761faf794a84b84/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.760798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2a6c910129406a9fc540c0db62cd40, user_id=66d33119c79d420f8761faf794a84b84, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:22.763461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:22.776894 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3225/6446] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:22 2022] PUT /identity/v3/projects/8c2a6c910129406a9fc540c0db62cd40/users/66d33119c79d420f8761faf794a84b84/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:22.788036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:22.788411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:22.788758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:22.825191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] MFA Rules not processed for user `66d33119c79d420f8761faf794a84b84`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:22.826487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:22.878813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:22.901984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3222/6447] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:22.914092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:22.916964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:22.934025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:23.026135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae3e58-78d3-437a-a743-ea4eb5b40225 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:23.030929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:23.031822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:23.032330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:23.032917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:23.044300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:23.046330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:23.082121 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3226/6448] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:14:23.138587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:23.140111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:23.157381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909a374-ba44-4577-bc4c-11248ec23838 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:23.166734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:23.167919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:23.168304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:23.168710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:23.302071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:23.303800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:23.328045 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3223/6449] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:24.529383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:24.530844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:24.592007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceff1bb-ad6d-4d8b-8b52-2244e4bd5964 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:24.595589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fdb4f-ac11-4d64-a89f-5ec0e633e201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:24.596192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fdb4f-ac11-4d64-a89f-5ec0e633e201 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:24.596468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fdb4f-ac11-4d64-a89f-5ec0e633e201 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:24.597000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fdb4f-ac11-4d64-a89f-5ec0e633e201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:24.598459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-323fdb4f-ac11-4d64-a89f-5ec0e633e201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:24.610518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fdb4f-ac11-4d64-a89f-5ec0e633e201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:24.613663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323fdb4f-ac11-4d64-a89f-5ec0e633e201 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:24.641916 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3227/6450] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:26.770410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:26.834467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326071a-93d6-49cd-886a-3f53c333da41 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:26.839419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:26.840066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:26.840346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:26.840620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] PATH_INFO: `/v3/users/e18e2a84d5864a0e86023af0f72e13c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:26.841722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18e2a84d5864a0e86023af0f72e13c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:26.843664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:26.889098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18e2a84d5864a0e86023af0f72e13c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:26.895854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18e2a84d5864a0e86023af0f72e13c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:26.896513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18e2a84d5864a0e86023af0f72e13c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:26.904241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18e2a84d5864a0e86023af0f72e13c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:26.904845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18e2a84d5864a0e86023af0f72e13c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:26.922526 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3224/6451] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:26 2022] DELETE /identity/v3/users/e18e2a84d5864a0e86023af0f72e13c8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:26.927390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:26.927871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:26.927969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:26.946972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:26.948285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:27.010241 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3228/6452] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:27.022066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:27.023475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:27.030701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:27.087992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9738f6-77fd-4120-a1b8-d70a2335d783 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:27.092374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:27.093001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:27.093281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:27.093551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:27.102837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:27.104593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:27.115285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3225/6453] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:27.549169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:27.550726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:27.563966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343f3ef-5a84-4ca0-bbac-c476c70278ef None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:27.568687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:27.569343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:27.569629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:27.569939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] PATH_INFO: `/v3/projects/38cbd58e2e324b7087b5866685583c1e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:27.571068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=38cbd58e2e324b7087b5866685583c1e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:27.573145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:27.631324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38cbd58e2e324b7087b5866685583c1e', 'request_id': 'req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.633596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:27.638957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38cbd58e2e324b7087b5866685583c1e', 'request_id': 'req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.645994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38cbd58e2e324b7087b5866685583c1e', 'request_id': 'req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.659947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] The token cache is being invalidate because project 38cbd58e2e324b7087b5866685583c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:27.660267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.660914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.662483 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3229/6454] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:27 2022] DELETE /identity/v3/projects/38cbd58e2e324b7087b5866685583c1e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:27.666491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:27.677722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e07b2b-2b69-4f7a-bc89-4670f89d1119 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:27.737552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:27.738221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:27.738504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:27.738781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] PATH_INFO: `/v3/users/66d33119c79d420f8761faf794a84b84` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:27.739911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d33119c79d420f8761faf794a84b84)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:27.741858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:27.791116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d33119c79d420f8761faf794a84b84'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.799616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d33119c79d420f8761faf794a84b84'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.800363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d33119c79d420f8761faf794a84b84'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.810432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d33119c79d420f8761faf794a84b84'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.810432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a3560302-36b9-4183-baaf-3228788df69d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d33119c79d420f8761faf794a84b84'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:27.828252 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3226/6455] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:27 2022] DELETE /identity/v3/users/66d33119c79d420f8761faf794a84b84 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:28.290079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:28.292111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:28.305201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bc1ea-6c79-4584-9a2e-bb7799aec4ab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:28.376836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:28.377479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.378116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.378116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] PATH_INFO: `/v3/projects/8c2a6c910129406a9fc540c0db62cd40` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.379198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2a6c910129406a9fc540c0db62cd40)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:28.381161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:28.436405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2a6c910129406a9fc540c0db62cd40', 'request_id': 'req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:28.445120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2a6c910129406a9fc540c0db62cd40', 'request_id': 'req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:28.452469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2a6c910129406a9fc540c0db62cd40', 'request_id': 'req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:28.469912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] The token cache is being invalidate because project 8c2a6c910129406a9fc540c0db62cd40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:28.470263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:28.470957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f37e821-ebb8-47ec-988b-d7f389b4e0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:28.472506 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3230/6456] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:28 2022] DELETE /identity/v3/projects/8c2a6c910129406a9fc540c0db62cd40 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:14:28.477358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e489b5-c22b-4f4b-8198-efcc4ed65cd4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.477727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e489b5-c22b-4f4b-8198-efcc4ed65cd4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.478136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e489b5-c22b-4f4b-8198-efcc4ed65cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.498451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-48e489b5-c22b-4f4b-8198-efcc4ed65cd4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:28.499666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-48e489b5-c22b-4f4b-8198-efcc4ed65cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:28.521732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:28.566242 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3227/6457] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:28.574484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c76cac-a33a-459d-a082-701d6e7f13b3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.574773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c76cac-a33a-459d-a082-701d6e7f13b3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.575046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c76cac-a33a-459d-a082-701d6e7f13b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.598802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-29c76cac-a33a-459d-a082-701d6e7f13b3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:28.601343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-29c76cac-a33a-459d-a082-701d6e7f13b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:28.615698 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3231/6458] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:28.620472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e489b5-c22b-4f4b-8198-efcc4ed65cd4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:28.632628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e489b5-c22b-4f4b-8198-efcc4ed65cd4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:28.638116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:28.638735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.639021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.639299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.639954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:14:28.640266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:28.642060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:28.655831 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3228/6459] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:14:28.660062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c76cac-a33a-459d-a082-701d6e7f13b3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:28.670603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c76cac-a33a-459d-a082-701d6e7f13b3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:28.678541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:28.678737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.679019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.679261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.679843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:28.681933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:28.715901 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3232/6460] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:28 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:28.720007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:28.731888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4a6d1-2225-4fdb-bcf3-6312e9c593c9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:28.804017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:28.804789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.805075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.805441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.806136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:28.808282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:28.838528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3229/6461] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:28 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:14:28.839941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:28.851602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f5239-300b-4943-b3ed-0eec6dd9e8a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:28.858644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:28.859410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.859854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.860137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.861378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:28.862812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:28.879807 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3233/6462] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:28.881365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:28.893666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44662b95-264b-4bae-87d1-c2c9447ea4ea admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:28.898926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d36cda-92be-48c8-b09b-f452735493ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:28.899641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36cda-92be-48c8-b09b-f452735493ba admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.900366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36cda-92be-48c8-b09b-f452735493ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.900366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d36cda-92be-48c8-b09b-f452735493ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.901020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d36cda-92be-48c8-b09b-f452735493ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:28.902960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d36cda-92be-48c8-b09b-f452735493ba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:28.915485 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3230/6463] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:28.920051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:28.930195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0c225-b406-493e-9c15-7188a14222c6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:28.935517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:28.936187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.936470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.936752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] PATH_INFO: `/v3/projects/dc8be867a018492fa241d488d560c359/users/97dcd111b68d4ee0a300488a8d9fa6cf/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:28.954104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8be867a018492fa241d488d560c359, user_id=97dcd111b68d4ee0a300488a8d9fa6cf, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:28.956760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:28.979985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3234/6464] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:28 2022] PUT /identity/v3/projects/dc8be867a018492fa241d488d560c359/users/97dcd111b68d4ee0a300488a8d9fa6cf/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:28.991671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:28.992503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:28.992503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:29.026511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] MFA Rules not processed for user `97dcd111b68d4ee0a300488a8d9fa6cf`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:29.027963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:29.108827 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3231/6465] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:29.125759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:29.127629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:29.140292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:29.213177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761ca9-23d3-4467-81dc-7e9cf7ecac15 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:29.217688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:29.218497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:29.218778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:29.219049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:29.233368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:29.234901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:29.245609 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3235/6466] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:30.337629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:30.339417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:30.368870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:30.413372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e623f79-a39f-4ab6-bdb8-d91f13b3af2b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:30.418210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:30.418847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:30.419129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:30.419406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] PATH_INFO: `/v3/users/9bbe85c6ecdb49a8b186a8cb6e1f400e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:30.420552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bbe85c6ecdb49a8b186a8cb6e1f400e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:30.422648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:30.471944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bbe85c6ecdb49a8b186a8cb6e1f400e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.478701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bbe85c6ecdb49a8b186a8cb6e1f400e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.479658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bbe85c6ecdb49a8b186a8cb6e1f400e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.487422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bbe85c6ecdb49a8b186a8cb6e1f400e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.488072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c9a9014-3f57-4c94-9757-0029c236f39b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bbe85c6ecdb49a8b186a8cb6e1f400e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.506977 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3232/6467] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:30 2022] DELETE /identity/v3/users/9bbe85c6ecdb49a8b186a8cb6e1f400e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:30.686237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:30.699554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c24c4a-7748-4d36-b778-5f255cf063c4 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:30.768419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:30.769183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:30.769603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:30.769918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] PATH_INFO: `/v3/projects/d2931f7a57c6444db783288bda1a74b8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:30.771042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2931f7a57c6444db783288bda1a74b8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:30.772914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:30.829813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2931f7a57c6444db783288bda1a74b8', 'request_id': 'req-4bf9cc16-aa71-4c36-9b3f-be3495c988df'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.836035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2931f7a57c6444db783288bda1a74b8', 'request_id': 'req-4bf9cc16-aa71-4c36-9b3f-be3495c988df'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.843683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2931f7a57c6444db783288bda1a74b8', 'request_id': 'req-4bf9cc16-aa71-4c36-9b3f-be3495c988df'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.865713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] The token cache is being invalidate because project d2931f7a57c6444db783288bda1a74b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:30.865946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.866568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bf9cc16-aa71-4c36-9b3f-be3495c988df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:30.869970 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3236/6468] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:30 2022] DELETE /identity/v3/projects/d2931f7a57c6444db783288bda1a74b8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:30.875138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20085c4-80d6-4ed1-9573-95b76ab2a0a5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:30.875431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20085c4-80d6-4ed1-9573-95b76ab2a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:30.875759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20085c4-80d6-4ed1-9573-95b76ab2a0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:30.896730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d20085c4-80d6-4ed1-9573-95b76ab2a0a5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:30.897872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d20085c4-80d6-4ed1-9573-95b76ab2a0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:30.962500 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3233/6469] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:30.971733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa0a96-3dff-4885-a69c-5ab87aa713dd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:30.972020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa0a96-3dff-4885-a69c-5ab87aa713dd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:30.972415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa0a96-3dff-4885-a69c-5ab87aa713dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:30.992818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5daa0a96-3dff-4885-a69c-5ab87aa713dd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:30.994059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5daa0a96-3dff-4885-a69c-5ab87aa713dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:31.002195 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3237/6470] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:31.006427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20085c4-80d6-4ed1-9573-95b76ab2a0a5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:31.018480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20085c4-80d6-4ed1-9573-95b76ab2a0a5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:31.023253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:31.023854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.024137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.024409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.025056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:14:31.025367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:31.027580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:31.039000 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3234/6471] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:14:31.043515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa0a96-3dff-4885-a69c-5ab87aa713dd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:31.052856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa0a96-3dff-4885-a69c-5ab87aa713dd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:31.058296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:31.058893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.059176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.059450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.060075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:31.061999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:31.089271 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3238/6472] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:31 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:31.093564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:31.103748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9e33c-46a3-4bd0-8da7-08c40412d32f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:31.130072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:31.162722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:31.163261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.163547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.163825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.164464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:31.166608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:31.188817 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3235/6473] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:31 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:31.192960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:31.204099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b17685-ba9e-46b9-9d47-1dc291c2b0c2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:31.208641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:31.209227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.209506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.209775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.210437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:31.212045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:31.218197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:31.223978 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3239/6474] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:31.227944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:31.238676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155fa73-4b54-40c5-a5bd-869f58558a79 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:31.243332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99411bd2-f464-4429-a793-4d3c64c21a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:31.243918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99411bd2-f464-4429-a793-4d3c64c21a43 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.244197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99411bd2-f464-4429-a793-4d3c64c21a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.244506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99411bd2-f464-4429-a793-4d3c64c21a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.245446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99411bd2-f464-4429-a793-4d3c64c21a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:31.247093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99411bd2-f464-4429-a793-4d3c64c21a43 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:31.257797 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3236/6475] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:31.262588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:31.272471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68f99e-26f6-4910-ab99-5fceddbaae51 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:31.278152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:31.278795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.279100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.279376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] PATH_INFO: `/v3/projects/f5c9ea913d214c569e6d6c3c1327f43a/users/9b7e48fd0dab42d3a904f7a07b939011/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.294073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c9ea913d214c569e6d6c3c1327f43a, user_id=9b7e48fd0dab42d3a904f7a07b939011, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:31.296835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:31.309800 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3240/6476] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:31 2022] PUT /identity/v3/projects/f5c9ea913d214c569e6d6c3c1327f43a/users/9b7e48fd0dab42d3a904f7a07b939011/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:31.315078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.315391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.315665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.334830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:31.336009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:31.394590 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3237/6477] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:31.404405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:31.406155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:31.464419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ade58-028e-4c64-9667-7cb4d604768d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:31.468293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb51d71-6ff4-49c2-86d5-ba4111aa7639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:31.468910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb51d71-6ff4-49c2-86d5-ba4111aa7639 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:31.469203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb51d71-6ff4-49c2-86d5-ba4111aa7639 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:31.469474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb51d71-6ff4-49c2-86d5-ba4111aa7639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:31.480445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb51d71-6ff4-49c2-86d5-ba4111aa7639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:31.481941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb51d71-6ff4-49c2-86d5-ba4111aa7639 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:31.491659 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3241/6478] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:35.901944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:35.903678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:35.923126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763f189-466f-4e17-843a-2abe1e2e086d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:35.928499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:35.930124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:35.930917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:35.932729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] PATH_INFO: `/v3/users/97dcd111b68d4ee0a300488a8d9fa6cf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:35.934403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=97dcd111b68d4ee0a300488a8d9fa6cf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:35.936361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:35.995206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6ad6e-ceea-423c-8adf-82af902dbb7b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:35.996268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6ad6e-ceea-423c-8adf-82af902dbb7b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:35.996553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6ad6e-ceea-423c-8adf-82af902dbb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.008169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97dcd111b68d4ee0a300488a8d9fa6cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.018653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97dcd111b68d4ee0a300488a8d9fa6cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.019476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97dcd111b68d4ee0a300488a8d9fa6cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.029274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97dcd111b68d4ee0a300488a8d9fa6cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.030131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-74a1cf11-ada3-4197-9eae-01492d2e74ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97dcd111b68d4ee0a300488a8d9fa6cf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.031158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-36d6ad6e-ceea-423c-8adf-82af902dbb7b None None] MFA Rules not processed for user `9b7e48fd0dab42d3a904f7a07b939011`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:36.032403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-36d6ad6e-ceea-423c-8adf-82af902dbb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.053873 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3238/6479] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:35 2022] DELETE /identity/v3/users/97dcd111b68d4ee0a300488a8d9fa6cf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:36.058830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.059379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.060059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.086128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:36.087362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.101446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:36.105014 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3242/6480] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:14:36.120362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.120936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.121281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.143238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] MFA Rules not processed for user `9b7e48fd0dab42d3a904f7a07b939011`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:36.144507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.174726 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3239/6481] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:36.180619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:36.183672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.230523 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3243/6482] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:36.249679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:36.251819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.271205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:36.275452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13b419-d16f-4132-a32d-6157d606dfa0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:36.284662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:36.285288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.285569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.285858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.298101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:36.299634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:36.311457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3240/6483] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:14:36.342013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c557d9b-57d4-49b6-839f-f0527d702e9d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:36.356583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:36.357404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.357787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.358196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.367633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:36.369573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:36.379820 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3244/6484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:14:36.631193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:36.632799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.654073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1832-4f64-474c-a87b-c1e3b09410d8 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:36.660169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:36.661362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.661880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.662357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] PATH_INFO: `/v3/projects/dc8be867a018492fa241d488d560c359` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.663838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8be867a018492fa241d488d560c359)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:36.666423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:36.715892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:36.730166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642a836-ac4f-4a42-ae81-5cc59c2f11a8 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:36.733918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8be867a018492fa241d488d560c359', 'request_id': 'req-38617bf3-ede2-432c-becb-2c048b052f58'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.747705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8be867a018492fa241d488d560c359', 'request_id': 'req-38617bf3-ede2-432c-becb-2c048b052f58'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.757197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8be867a018492fa241d488d560c359', 'request_id': 'req-38617bf3-ede2-432c-becb-2c048b052f58'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.771280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] The token cache is being invalidate because project dc8be867a018492fa241d488d560c359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:36.771877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.772747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-38617bf3-ede2-432c-becb-2c048b052f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:36.774549 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3241/6485] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:36 2022] DELETE /identity/v3/projects/dc8be867a018492fa241d488d560c359 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:36.779678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723d265-9521-4d1d-a71e-83c0fe96f8e1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.779971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723d265-9521-4d1d-a71e-83c0fe96f8e1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.780433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723d265-9521-4d1d-a71e-83c0fe96f8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.799460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c307e-b8d0-4e84-b586-8eb6698af46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:36.800079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c307e-b8d0-4e84-b586-8eb6698af46c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.800360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c307e-b8d0-4e84-b586-8eb6698af46c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.800633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c307e-b8d0-4e84-b586-8eb6698af46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.802308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b723d265-9521-4d1d-a71e-83c0fe96f8e1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:36.802631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-111c307e-b8d0-4e84-b586-8eb6698af46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.803510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b723d265-9521-4d1d-a71e-83c0fe96f8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.814270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c307e-b8d0-4e84-b586-8eb6698af46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:36.815810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c307e-b8d0-4e84-b586-8eb6698af46c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:36.826365 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3245/6486] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:36.851698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:36.873401 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3242/6487] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:14:36.880957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4be77f-d49e-4003-8bc4-ed19ef426f88 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.881316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4be77f-d49e-4003-8bc4-ed19ef426f88 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.881661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4be77f-d49e-4003-8bc4-ed19ef426f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.904165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2c4be77f-d49e-4003-8bc4-ed19ef426f88 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:36.905389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c4be77f-d49e-4003-8bc4-ed19ef426f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:36.914019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3246/6488] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:36.919608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723d265-9521-4d1d-a71e-83c0fe96f8e1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:36.929981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723d265-9521-4d1d-a71e-83c0fe96f8e1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:36.934761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:36.935377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.935660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.935931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.936585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:14:36.942516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:36.944288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:36.957152 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3243/6489] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:14:36.961635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4be77f-d49e-4003-8bc4-ed19ef426f88 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:36.975387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4be77f-d49e-4003-8bc4-ed19ef426f88 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:36.983785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:36.984517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:36.984870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:36.985217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:36.986069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:36.988047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:37.023455 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3247/6490] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:36 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:14:37.028451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:37.039306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0daf2-ca20-449d-973c-0fc2966d54c5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:37.101778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:37.102538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:37.102824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:37.103095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:37.103795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:37.105995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:37.130481 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3244/6491] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:37 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:37.131697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:37.143114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d18e03-1a04-4422-b64a-09bf8f98b419 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:37.147821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:37.148415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:37.148692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:37.148968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:37.149596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:37.151822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:37.165342 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3248/6492] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:37.167159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:37.177666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1a697-9d58-430c-b0f7-fb2521631602 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:37.183665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b5e0c-46b3-40f8-83df-05688ea34f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:37.184375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b5e0c-46b3-40f8-83df-05688ea34f47 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:37.184687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b5e0c-46b3-40f8-83df-05688ea34f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:37.184959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b5e0c-46b3-40f8-83df-05688ea34f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:37.185599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b5e0c-46b3-40f8-83df-05688ea34f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:37.187235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b5e0c-46b3-40f8-83df-05688ea34f47 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:37.198586 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3245/6493] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:14:37.202946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:37.213340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d55f9a-cd6b-4164-96c4-935cfcb511d7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:37.218386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:37.219019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:37.219306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:37.219628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] PATH_INFO: `/v3/projects/a33ae4d34d6c48a8b734b9f7c36c15fa/users/1c5a961863e64fb88901c1d9a0f8b307/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:37.238948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33ae4d34d6c48a8b734b9f7c36c15fa, user_id=1c5a961863e64fb88901c1d9a0f8b307, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:37.241543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:37.258609 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3249/6494] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:37 2022] PUT /identity/v3/projects/a33ae4d34d6c48a8b734b9f7c36c15fa/users/1c5a961863e64fb88901c1d9a0f8b307/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:37.269677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a176b89-b78b-4061-9ed1-591c36f0a9a9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:37.270248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a176b89-b78b-4061-9ed1-591c36f0a9a9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:37.270532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a176b89-b78b-4061-9ed1-591c36f0a9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:37.301693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3a176b89-b78b-4061-9ed1-591c36f0a9a9 None None] MFA Rules not processed for user `1c5a961863e64fb88901c1d9a0f8b307`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:37.302933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3a176b89-b78b-4061-9ed1-591c36f0a9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:37.383167 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3246/6495] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:37.394989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:37.396735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:37.458486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:37.491611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c532461-76a7-4116-bffd-31dc384cb988 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:37.496050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debbb23-be6f-44eb-a198-2eb2a5345518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:37.496661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debbb23-be6f-44eb-a198-2eb2a5345518 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:37.496944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debbb23-be6f-44eb-a198-2eb2a5345518 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:37.497217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debbb23-be6f-44eb-a198-2eb2a5345518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:37.523452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debbb23-be6f-44eb-a198-2eb2a5345518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:37.525420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debbb23-be6f-44eb-a198-2eb2a5345518 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:37.540690 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3250/6496] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:47.379835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:47.534911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:48.057305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a176b89-b78b-4061-9ed1-591c36f0a9a9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:48.127280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a176b89-b78b-4061-9ed1-591c36f0a9a9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:48.133200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:48.133873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:48.134170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:48.134452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] PATH_INFO: `/v3/users/1c5a961863e64fb88901c1d9a0f8b307` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:48.136463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5a961863e64fb88901c1d9a0f8b307)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:48.138553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:48.193019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5a961863e64fb88901c1d9a0f8b307'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.201235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5a961863e64fb88901c1d9a0f8b307'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.202134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5a961863e64fb88901c1d9a0f8b307'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.210226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5a961863e64fb88901c1d9a0f8b307'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.210986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5a961863e64fb88901c1d9a0f8b307'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.240503 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3247/6497] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:14:48 2022] DELETE /identity/v3/users/1c5a961863e64fb88901c1d9a0f8b307 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:48.242087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffcd8-17d4-4774-9245-8dc7ac0ac850 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:48.242383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffcd8-17d4-4774-9245-8dc7ac0ac850 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:48.242664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ffcd8-17d4-4774-9245-8dc7ac0ac850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:48.266393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-122ffcd8-17d4-4774-9245-8dc7ac0ac850 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:48.267618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-122ffcd8-17d4-4774-9245-8dc7ac0ac850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:48.340087 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3251/6498] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:48.347131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:48.352428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:48.419379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956a2ba-2470-493d-92ba-1df7015501bd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:48.423691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:48.424419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:48.424774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:48.425158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:48.436459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:48.437947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:48.440343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5fab6-52f4-46b6-b21b-05b4429408c9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:48.440641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5fab6-52f4-46b6-b21b-05b4429408c9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:48.440935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5fab6-52f4-46b6-b21b-05b4429408c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:48.448724 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3248/6499] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:48.460705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-94d5fab6-52f4-46b6-b21b-05b4429408c9 None None] MFA Rules not processed for user `3f03d688d0ee4d2b951828a3f496c914`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:48.461948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-94d5fab6-52f4-46b6-b21b-05b4429408c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:48.534790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3252/6500] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:14:48.788604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:48.790103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:48.804283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6226-cd13-48fd-badc-e5847c9a86dc service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:48.812414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:48.813198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:48.813614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:48.814177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] PATH_INFO: `/v3/projects/a33ae4d34d6c48a8b734b9f7c36c15fa` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:48.816119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33ae4d34d6c48a8b734b9f7c36c15fa)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:48.827238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:48.859961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:48.911275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33ae4d34d6c48a8b734b9f7c36c15fa', 'request_id': 'req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.920874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33ae4d34d6c48a8b734b9f7c36c15fa', 'request_id': 'req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.940715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33ae4d34d6c48a8b734b9f7c36c15fa', 'request_id': 'req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.959602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] The token cache is being invalidate because project a33ae4d34d6c48a8b734b9f7c36c15fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:14:48.960316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.961177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4b51b0f-ed5e-403b-8d53-27c9a8227ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:14:48.963489 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3249/6501] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:14:48 2022] DELETE /identity/v3/projects/a33ae4d34d6c48a8b734b9f7c36c15fa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:48.968513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b531bf-f089-4c94-8127-a16721aef49c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:48.969314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b531bf-f089-4c94-8127-a16721aef49c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:48.969595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b531bf-f089-4c94-8127-a16721aef49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:48.996042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-41b531bf-f089-4c94-8127-a16721aef49c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:48.997307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-41b531bf-f089-4c94-8127-a16721aef49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:49.074367 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3253/6502] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:49.082042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba5c73-faa7-41db-9415-b93af1be68f3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.082511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba5c73-faa7-41db-9415-b93af1be68f3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.082789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba5c73-faa7-41db-9415-b93af1be68f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.104281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7fba5c73-faa7-41db-9415-b93af1be68f3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:49.105700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7fba5c73-faa7-41db-9415-b93af1be68f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:49.116784 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3250/6503] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:49.121256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b531bf-f089-4c94-8127-a16721aef49c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:49.131565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b531bf-f089-4c94-8127-a16721aef49c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:49.136998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:49.137632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.137954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.138234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.138882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:14:49.139189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:49.140805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:49.157643 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3254/6504] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:14:49.158939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba5c73-faa7-41db-9415-b93af1be68f3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:49.169244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba5c73-faa7-41db-9415-b93af1be68f3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:49.174726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:49.175340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.175632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.175898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.177144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:49.179134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:49.208365 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3251/6505] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:14:49 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:14:49.212582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:49.223724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe9b6d-d171-4c81-907c-596645bff90d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:49.246064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:49.286833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:49.287445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.287729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.288140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.288766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:49.290982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:49.318463 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3255/6506] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:14:49 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:14:49.319643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:49.331220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3755ff-7f7d-45ed-b8f1-862b2dfd0f9c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:49.336045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:49.336741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.337022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.337293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.338084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:49.339996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:49.353456 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3252/6507] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:49.355102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:49.365312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3a318-2b5e-4b75-8314-2cc2df925c5c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:49.371186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e91bf-1eb0-44a6-8f04-048b030dabd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:49.371851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e91bf-1eb0-44a6-8f04-048b030dabd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.372256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e91bf-1eb0-44a6-8f04-048b030dabd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.372529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e91bf-1eb0-44a6-8f04-048b030dabd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.373517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e91bf-1eb0-44a6-8f04-048b030dabd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:49.374894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e91bf-1eb0-44a6-8f04-048b030dabd8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:49.386369 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3256/6508] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:14:49.390437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:49.403265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66c5a-f9e1-4696-98f0-2d70a25e681e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:49.417344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:49.418087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.418386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.418653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] PATH_INFO: `/v3/projects/ff8f2031a94548e5bb3eea8bce391e8e/users/8db29080e8c047b88b33e61bf5270c35/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.442157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8f2031a94548e5bb3eea8bce391e8e, user_id=8db29080e8c047b88b33e61bf5270c35, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:49.445502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:49.468357 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3253/6509] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:14:49 2022] PUT /identity/v3/projects/ff8f2031a94548e5bb3eea8bce391e8e/users/8db29080e8c047b88b33e61bf5270c35/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:14:49.484323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04592cdf-f81b-428c-be78-28e37eb6017a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.484612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04592cdf-f81b-428c-be78-28e37eb6017a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.484890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04592cdf-f81b-428c-be78-28e37eb6017a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.525241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-04592cdf-f81b-428c-be78-28e37eb6017a None None] MFA Rules not processed for user `8db29080e8c047b88b33e61bf5270c35`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:49.528359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-04592cdf-f81b-428c-be78-28e37eb6017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:49.622559 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3257/6510] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:49.635316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:49.643823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:49.709167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fda3c-f3a4-47ed-8155-2488fa793b95 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:49.713544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:49.714264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:49.714548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:49.714827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:49.723570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:49.725004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:49.734808 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3254/6511] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:14:51.123819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfbbbf-163d-4f16-8ec3-7fb09b009012 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:51.124353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfbbbf-163d-4f16-8ec3-7fb09b009012 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:51.124593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfbbbf-163d-4f16-8ec3-7fb09b009012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:51.143593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bcbfbbbf-163d-4f16-8ec3-7fb09b009012 None None] MFA Rules not processed for user `8db29080e8c047b88b33e61bf5270c35`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:14:51.144882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bcbfbbbf-163d-4f16-8ec3-7fb09b009012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:51.154261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3258/6512] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:14:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:14:59.231066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:14:59.232660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:59.287032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:14:59.316967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74196f-23d2-4849-a80b-c1077b210ec1 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:14:59.322692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:14:59.323290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:14:59.323572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:14:59.323848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:14:59.325584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:14:59.384175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:14:59.386059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:14:59.395240 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3255/6513] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:01.138662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:09.390887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=36 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:12.205401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87aed0-6f5a-4554-95d1-37794d65c5a5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:12.206057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87aed0-6f5a-4554-95d1-37794d65c5a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:12.206057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87aed0-6f5a-4554-95d1-37794d65c5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:12.226890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7a87aed0-6f5a-4554-95d1-37794d65c5a5 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:12.228364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a87aed0-6f5a-4554-95d1-37794d65c5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:12.296630 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3259/6514] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:12.349000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:12.350551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:12.442294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93584465-523c-485d-9ec9-95170b01bd10 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:12.450581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:12.451647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:12.452083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:12.452669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:12.464625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:12.466490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:12.479947 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3256/6515] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:15:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:15:13.110513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae27c6-004e-47ed-b895-ebb26bbdc364 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:13.110806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae27c6-004e-47ed-b895-ebb26bbdc364 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:13.111083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae27c6-004e-47ed-b895-ebb26bbdc364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:13.140200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-56ae27c6-004e-47ed-b895-ebb26bbdc364 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:13.141431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-56ae27c6-004e-47ed-b895-ebb26bbdc364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:13.152873 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3260/6516] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:13.170505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:13.185084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2042f-d950-4200-a1c6-06f455d36d59 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:13.190810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:13.191649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:13.192108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:13.192561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:13.203031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:13.204847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:13.214573 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3257/6517] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:15:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:15:13.472064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b964-88c7-4703-b099-7a1bf7c5ac51 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:13.472972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b964-88c7-4703-b099-7a1bf7c5ac51 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:13.473489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b964-88c7-4703-b099-7a1bf7c5ac51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:13.495743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2f03b964-88c7-4703-b099-7a1bf7c5ac51 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:13.497699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2f03b964-88c7-4703-b099-7a1bf7c5ac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:13.516881 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3261/6518] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:13.534878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:13.547584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d684c-2fd9-4422-a022-c6c9c706698e service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:13.559559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:13.560161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:13.560758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:13.561561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:13.578316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:13.580135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:13.601765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3258/6519] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:15:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:15:14.080080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:14.080456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:14.080848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:14.116043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:14.117368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:14.143463 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3262/6520] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:15:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:14.160623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:14.180413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb08cf-2edb-4f61-b44e-2df49a76c441 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:14.190739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:14.191053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:14.191373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:14.191708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:14.202624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:14.204412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:14.217449 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3259/6521] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:15:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:15:14.599793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:14.603770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:14.706473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1c4e1-0406-41b9-a8f0-3286af801f61 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:14.714373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:14.715131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:14.715415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:14.715693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] PATH_INFO: `/v3/users/9b7e48fd0dab42d3a904f7a07b939011` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:14.716966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b7e48fd0dab42d3a904f7a07b939011)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:14.719094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:14.804370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b7e48fd0dab42d3a904f7a07b939011'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:14.820898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b7e48fd0dab42d3a904f7a07b939011'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:14.830043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b7e48fd0dab42d3a904f7a07b939011'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:14.839257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b7e48fd0dab42d3a904f7a07b939011'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:14.840890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62591e9e-194a-483a-922f-bdb8fcaa4c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b7e48fd0dab42d3a904f7a07b939011'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:14.849222 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3263/6522] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:14 2022] DELETE /identity/v3/users/9b7e48fd0dab42d3a904f7a07b939011 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:15.074347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.089196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c487f73-f663-4af3-866f-2877830e606e service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.172345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.174055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.174354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.174639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] PATH_INFO: `/v3/projects/f5c9ea913d214c569e6d6c3c1327f43a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.176128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c9ea913d214c569e6d6c3c1327f43a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:15.178600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:15.258717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c9ea913d214c569e6d6c3c1327f43a', 'request_id': 'req-c15b079f-5a57-4226-a962-066d5649d100'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:15.262181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:15.267690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c9ea913d214c569e6d6c3c1327f43a', 'request_id': 'req-c15b079f-5a57-4226-a962-066d5649d100'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:15.287839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c9ea913d214c569e6d6c3c1327f43a', 'request_id': 'req-c15b079f-5a57-4226-a962-066d5649d100'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:15.301741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] The token cache is being invalidate because project f5c9ea913d214c569e6d6c3c1327f43a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:15.302318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:15.303064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c15b079f-5a57-4226-a962-066d5649d100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:15.304679 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3260/6523] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:15 2022] DELETE /identity/v3/projects/f5c9ea913d214c569e6d6c3c1327f43a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:15.311006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75de066-a7ee-497c-bac4-a4ea9ec2d6ac None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.311265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75de066-a7ee-497c-bac4-a4ea9ec2d6ac None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.311605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75de066-a7ee-497c-bac4-a4ea9ec2d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.321544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:15.335728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a75de066-a7ee-497c-bac4-a4ea9ec2d6ac None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:15.337114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a75de066-a7ee-497c-bac4-a4ea9ec2d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:15.407836 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3264/6524] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:15.417534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701793-37f1-478b-990b-7c1c22ea8517 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.418218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701793-37f1-478b-990b-7c1c22ea8517 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.418682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9701793-37f1-478b-990b-7c1c22ea8517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.449305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d9701793-37f1-478b-990b-7c1c22ea8517 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:15.450823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d9701793-37f1-478b-990b-7c1c22ea8517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:15.460481 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3261/6525] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:15.465108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75de066-a7ee-497c-bac4-a4ea9ec2d6ac None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.476623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75de066-a7ee-497c-bac4-a4ea9ec2d6ac None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.482565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.483176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.483459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.483734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.484384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:15.484690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:15.486347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:15.501685 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3265/6526] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:15:15.506133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701793-37f1-478b-990b-7c1c22ea8517 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.517340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9701793-37f1-478b-990b-7c1c22ea8517 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.534428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.535229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.535511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.535803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.536428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:15.540059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:15.578284 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3262/6527] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:15 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:15.579910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.591106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcedc76-efca-4b5c-91b3-b7a843cc4c6b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.661135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.661819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.662059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.662405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.663066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:15.665062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:15.688769 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3266/6528] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:15 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:15.692806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.705731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f0af5-2327-43cf-a49d-deea9a9e373c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.711355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.711995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.712276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.712551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.713241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:15.715008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:15.726743 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3263/6529] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:15.731169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.743098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7971d-84dd-400e-913d-23927266ed30 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.749037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb95310-2629-4d38-bf4c-6fba8961b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.749674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb95310-2629-4d38-bf4c-6fba8961b6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.750001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb95310-2629-4d38-bf4c-6fba8961b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.750290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb95310-2629-4d38-bf4c-6fba8961b6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.750919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb95310-2629-4d38-bf4c-6fba8961b6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:15.752534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb95310-2629-4d38-bf4c-6fba8961b6c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:15.764888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3267/6530] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:15:15.769146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.780289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001057cc-edf0-495c-9de8-489f528c3694 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:15.785519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a2091-b914-42ae-84cd-59535ef866ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:15.786290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2091-b914-42ae-84cd-59535ef866ae admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.786574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2091-b914-42ae-84cd-59535ef866ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.787021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2091-b914-42ae-84cd-59535ef866ae admin admin] PATH_INFO: `/v3/projects/3ae9868b7610448b90be6b692fd087d8/users/a16673f69a1445f982ad5c5c800ca3af/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.802516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a2091-b914-42ae-84cd-59535ef866ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae9868b7610448b90be6b692fd087d8, user_id=a16673f69a1445f982ad5c5c800ca3af, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:15.805177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a2091-b914-42ae-84cd-59535ef866ae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:15.820417 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3264/6531] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:15 2022] PUT /identity/v3/projects/3ae9868b7610448b90be6b692fd087d8/users/a16673f69a1445f982ad5c5c800ca3af/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:15.831587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.831876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.832154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.864027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] MFA Rules not processed for user `a16673f69a1445f982ad5c5c800ca3af`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:15.865406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:15.931286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:15.935215 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3268/6532] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:15.945910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:15.946199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:15.946470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:15.967578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] MFA Rules not processed for user `a16673f69a1445f982ad5c5c800ca3af`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:15.969039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:15.983280 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3265/6533] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:15.997888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:15.999302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:16.082151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30a08e-1e9d-4902-ae21-73d3b566c2b4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:16.087957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef32cc6-dd3b-45c5-a9ac-2630f2d7123a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:16.089149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef32cc6-dd3b-45c5-a9ac-2630f2d7123a service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:16.089493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef32cc6-dd3b-45c5-a9ac-2630f2d7123a service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:16.089913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef32cc6-dd3b-45c5-a9ac-2630f2d7123a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:16.098394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef32cc6-dd3b-45c5-a9ac-2630f2d7123a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:16.099969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef32cc6-dd3b-45c5-a9ac-2630f2d7123a service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:16.109559 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3269/6534] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:16.556031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:16.558625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:16.653677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddfb73-05c9-4aaf-8baa-514a04567ca8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:16.662072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:16.662758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:16.663048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:16.663353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] PATH_INFO: `/v3/users/8db29080e8c047b88b33e61bf5270c35` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:16.664552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db29080e8c047b88b33e61bf5270c35)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:16.667267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:16.735062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db29080e8c047b88b33e61bf5270c35'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:16.742330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db29080e8c047b88b33e61bf5270c35'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:16.743018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db29080e8c047b88b33e61bf5270c35'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:16.750724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db29080e8c047b88b33e61bf5270c35'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:16.751358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db29080e8c047b88b33e61bf5270c35'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:16.759638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:16.774202 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3266/6535] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:16 2022] DELETE /identity/v3/users/8db29080e8c047b88b33e61bf5270c35 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:16.779395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:16.779862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:16.779954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:16.799231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:16.800460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:16.867303 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3270/6536] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:16.874739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:16.876250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:16.938237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10ee70-4c44-487c-89a5-79f1a9c38401 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:16.941939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8748-07de-4eba-88d5-9a62a560a394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:16.942552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8748-07de-4eba-88d5-9a62a560a394 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:16.942836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8748-07de-4eba-88d5-9a62a560a394 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:16.943264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8748-07de-4eba-88d5-9a62a560a394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:16.952938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8748-07de-4eba-88d5-9a62a560a394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:16.954548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8748-07de-4eba-88d5-9a62a560a394 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:16.964935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3267/6537] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:17.158963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.160627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:17.172985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072bcb1-b931-4968-8dcc-1f4085e8286f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.178416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.179053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.179336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.179611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] PATH_INFO: `/v3/projects/ff8f2031a94548e5bb3eea8bce391e8e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.180739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8f2031a94548e5bb3eea8bce391e8e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.183226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.231381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8f2031a94548e5bb3eea8bce391e8e', 'request_id': 'req-83d3d23e-25ea-4125-a0f2-19bd96ddb342'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:17.237781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8f2031a94548e5bb3eea8bce391e8e', 'request_id': 'req-83d3d23e-25ea-4125-a0f2-19bd96ddb342'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:17.246379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8f2031a94548e5bb3eea8bce391e8e', 'request_id': 'req-83d3d23e-25ea-4125-a0f2-19bd96ddb342'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:17.265711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] The token cache is being invalidate because project ff8f2031a94548e5bb3eea8bce391e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:17.266314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:17.266971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:17.268454 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3271/6538] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:17 2022] DELETE /identity/v3/projects/ff8f2031a94548e5bb3eea8bce391e8e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:17.273665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb05a0-4fcd-4f5f-9b81-f1a9cdc3cafc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.273996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb05a0-4fcd-4f5f-9b81-f1a9cdc3cafc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.274281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb05a0-4fcd-4f5f-9b81-f1a9cdc3cafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.301713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a2fb05a0-4fcd-4f5f-9b81-f1a9cdc3cafc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:17.303264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2fb05a0-4fcd-4f5f-9b81-f1a9cdc3cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:17.357127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.358606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:17.370922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:17.381671 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3268/6539] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:17.391145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ea9c6-a7fb-4ad1-bbfa-268a6c3a5025 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.391434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ea9c6-a7fb-4ad1-bbfa-268a6c3a5025 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.392016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ea9c6-a7fb-4ad1-bbfa-268a6c3a5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.415718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c40ea9c6-a7fb-4ad1-bbfa-268a6c3a5025 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:17.416980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c40ea9c6-a7fb-4ad1-bbfa-268a6c3a5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:17.427706 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3269/6540] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:17.431745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ea9c6-a7fb-4ad1-bbfa-268a6c3a5025 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.441814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ea9c6-a7fb-4ad1-bbfa-268a6c3a5025 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.447012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.448726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.448726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.449003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.449666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:17.450147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.450936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3d23e-25ea-4125-a0f2-19bd96ddb342 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.451872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.456890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.457532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.457904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.458383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.460443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:17.465864 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3270/6541] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:15:17.469638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.481782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.483533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b7de6-e967-4769-a5de-9fa1a6e1ea15 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.483834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.494319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.495202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.495753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3272/6542] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:15:17.498153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.498534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.499268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.501392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.537252 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3271/6543] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:17 2022] POST /identity/v3/projects => generated 397 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:17.539344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.550531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006ccf2-2d05-4c91-9b20-7cb825ce0545 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.628011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.628643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.628930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.629599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.629913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.631873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.663638 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3273/6544] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:17 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:17.664971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.677657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d980-5c22-43ea-a78c-d40df6fadab2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.683085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.683992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.684274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.684550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.685249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.686966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.700730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3272/6545] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:17.701994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.713026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c33772-f3a9-4a18-ac91-ad675d571c9a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.718558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497a366-a8c8-4aa3-9076-c7c71b1396aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.719168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a366-a8c8-4aa3-9076-c7c71b1396aa admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.719452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a366-a8c8-4aa3-9076-c7c71b1396aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.719726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a366-a8c8-4aa3-9076-c7c71b1396aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.720362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497a366-a8c8-4aa3-9076-c7c71b1396aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.722016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497a366-a8c8-4aa3-9076-c7c71b1396aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.736305 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3274/6546] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:17.737496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.749795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fcdcc-d81c-4e05-9733-945a7e89178c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:17.754736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:17.755471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.755762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.756043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] PATH_INFO: `/v3/projects/b32d92ed96cb4b289d4cf1b80ab78136/users/3d54e02a4fe049d89acf354ed6ef62ad/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.771758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32d92ed96cb4b289d4cf1b80ab78136, user_id=3d54e02a4fe049d89acf354ed6ef62ad, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:17.775220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:17.795156 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3273/6547] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:17 2022] PUT /identity/v3/projects/b32d92ed96cb4b289d4cf1b80ab78136/users/3d54e02a4fe049d89acf354ed6ef62ad/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:17.806992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:17.807703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:17.807703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:17.840457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] MFA Rules not processed for user `3d54e02a4fe049d89acf354ed6ef62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:17.841796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:17.922584 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3275/6548] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:17.934476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:17.936611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:17.957654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:18.011317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae3fff-c483-4e1c-8340-56c76848148a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:18.015477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d281ee8-5627-48b8-b87d-4a638133f92c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:18.016505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d281ee8-5627-48b8-b87d-4a638133f92c service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:18.016823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d281ee8-5627-48b8-b87d-4a638133f92c service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:18.017309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d281ee8-5627-48b8-b87d-4a638133f92c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:18.030621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d281ee8-5627-48b8-b87d-4a638133f92c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:18.032074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d281ee8-5627-48b8-b87d-4a638133f92c service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:18.044953 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3274/6549] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:18.523605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:18.526111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:18.559371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:18.675921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72e79-c49d-4635-a0da-09d9b905499f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:18.683662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:18.684479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:18.685006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:18.685381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:18.687654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:18.812899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:18.815059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:18.840376 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3276/6550] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:19.842866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ad633-fdbc-48cc-98d8-a95ab2b1c2d2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:19.843481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ad633-fdbc-48cc-98d8-a95ab2b1c2d2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:19.843750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ad633-fdbc-48cc-98d8-a95ab2b1c2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:19.871557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c86ad633-fdbc-48cc-98d8-a95ab2b1c2d2 None None] MFA Rules not processed for user `d83b6ceb599746b394e379d52683b70c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:19.872791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c86ad633-fdbc-48cc-98d8-a95ab2b1c2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:19.889313 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3275/6551] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:21.590941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:21.593124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:21.685809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9bbb6-c4bb-4f80-9390-dbc142423db7 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:21.692163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:21.692941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:21.693298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:21.693653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] PATH_INFO: `/v3/users/54dcc78ee39c4119b08023d3cc692d6b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:21.695028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54dcc78ee39c4119b08023d3cc692d6b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:21.697505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:21.778110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54dcc78ee39c4119b08023d3cc692d6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:21.780444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:21.794764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54dcc78ee39c4119b08023d3cc692d6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:21.810237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54dcc78ee39c4119b08023d3cc692d6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:21.820014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54dcc78ee39c4119b08023d3cc692d6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:21.822450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-37236ced-25a3-4811-b162-db20e1a86502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54dcc78ee39c4119b08023d3cc692d6b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:21.831588 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3277/6552] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:21 2022] DELETE /identity/v3/users/54dcc78ee39c4119b08023d3cc692d6b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:22.056068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ad633-fdbc-48cc-98d8-a95ab2b1c2d2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.069963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ad633-fdbc-48cc-98d8-a95ab2b1c2d2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.137039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.137895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.138400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.138872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] PATH_INFO: `/v3/projects/c8d7ded29d1345b4b3111a69f4efec6f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.140158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d7ded29d1345b4b3111a69f4efec6f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.142472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.214108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d7ded29d1345b4b3111a69f4efec6f', 'request_id': 'req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.222200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d7ded29d1345b4b3111a69f4efec6f', 'request_id': 'req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.233702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d7ded29d1345b4b3111a69f4efec6f', 'request_id': 'req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.247702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] The token cache is being invalidate because project c8d7ded29d1345b4b3111a69f4efec6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:22.248051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.248895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c4876ad6-1c4e-4551-93aa-02a38d40f9fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:22.258284 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3276/6553] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:22 2022] DELETE /identity/v3/projects/c8d7ded29d1345b4b3111a69f4efec6f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:22.259624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43c9c3-47e7-42ce-aca1-3a79b2c44636 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.259946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43c9c3-47e7-42ce-aca1-3a79b2c44636 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.260811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43c9c3-47e7-42ce-aca1-3a79b2c44636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.290803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ab43c9c3-47e7-42ce-aca1-3a79b2c44636 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:22.292379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab43c9c3-47e7-42ce-aca1-3a79b2c44636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:22.382655 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3278/6554] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:22.391958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366689b-9cdc-4bd9-b78c-dfcb236cf1b8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.392284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366689b-9cdc-4bd9-b78c-dfcb236cf1b8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.392719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366689b-9cdc-4bd9-b78c-dfcb236cf1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.466841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0366689b-9cdc-4bd9-b78c-dfcb236cf1b8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:22.469530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0366689b-9cdc-4bd9-b78c-dfcb236cf1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:22.497253 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3277/6555] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:22.501734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43c9c3-47e7-42ce-aca1-3a79b2c44636 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.513362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43c9c3-47e7-42ce-aca1-3a79b2c44636 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.540321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.540969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.541290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.541572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.542383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:22.543467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.545105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.565887 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3279/6556] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:22.569046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366689b-9cdc-4bd9-b78c-dfcb236cf1b8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.588119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366689b-9cdc-4bd9-b78c-dfcb236cf1b8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.616303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.618011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.618531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.620103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.621223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.626285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.684813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.697025 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3278/6557] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:22 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:22.697642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e649d-5597-4dee-abb2-fb8d3087c619 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.790416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.791775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.792315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.792836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.793557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.795992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.839029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.839994 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3280/6558] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:22 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:22.855247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f6cc3-4af4-481b-b83f-59ec36b075ca admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.860887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.861527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.861781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.862096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.863103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.865259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.885890 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3279/6559] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:22.889430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.907342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348c1b8-1b23-4ed3-98a7-aa4325e341eb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.922693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c70f7-f47e-420b-957e-6428a6b4da2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.923329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c70f7-f47e-420b-957e-6428a6b4da2f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.923621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c70f7-f47e-420b-957e-6428a6b4da2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.923889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c70f7-f47e-420b-957e-6428a6b4da2f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:22.924549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c70f7-f47e-420b-957e-6428a6b4da2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:22.926242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c70f7-f47e-420b-957e-6428a6b4da2f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:22.947470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:22.949072 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3281/6560] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:22.955171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:22.964560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95529c26-c3c2-42ae-8566-724b6931e110 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:22.988336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:22.989093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:22.989408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:22.989688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] PATH_INFO: `/v3/projects/47bf53336d3c4159baf93bbfadc2035c/users/f25963e8ba5943e080f6b80562ecf25c/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.008801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bf53336d3c4159baf93bbfadc2035c, user_id=f25963e8ba5943e080f6b80562ecf25c, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.012339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.031144 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3280/6561] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:22 2022] PUT /identity/v3/projects/47bf53336d3c4159baf93bbfadc2035c/users/f25963e8ba5943e080f6b80562ecf25c/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:23.043096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e9e6-d49f-4018-a497-681e645decf3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.043385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e9e6-d49f-4018-a497-681e645decf3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.043663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e9e6-d49f-4018-a497-681e645decf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.080484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f3a1e9e6-d49f-4018-a497-681e645decf3 None None] MFA Rules not processed for user `f25963e8ba5943e080f6b80562ecf25c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:23.082136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f3a1e9e6-d49f-4018-a497-681e645decf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:23.110061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:23.162937 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3282/6562] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:23.168426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.249241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090ff83-16eb-4239-a565-05df84fed660 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.254505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.255118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.255571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.255922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.256438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.258452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.303415 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3281/6563] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:23 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:23.306150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e9e6-d49f-4018-a497-681e645decf3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.319859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e9e6-d49f-4018-a497-681e645decf3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.414819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.415036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.415366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.415854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.416946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.419505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.456792 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3283/6564] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:23 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:23.457371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.473498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bbfa4-f80e-4cfa-9109-a9b55fb68138 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.480192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.481088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.481088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.482636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.483305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.485054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.501976 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3282/6565] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:23.503161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.515550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd35b-5eb8-499b-bb5c-5a14375a93c9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.520694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.521666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.521666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.521998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] PATH_INFO: `/v3/projects/d158ddef23304d7db041fba545b105d8/users/1d4b8a59bc50499784557007e2f3ec2c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.574526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.575217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d158ddef23304d7db041fba545b105d8, user_id=1d4b8a59bc50499784557007e2f3ec2c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.577681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:23.580655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.595828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b27c1-ae0a-4076-8b2d-655ed8e29a81 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.609841 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3284/6566] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:23 2022] PUT /identity/v3/projects/d158ddef23304d7db041fba545b105d8/users/1d4b8a59bc50499784557007e2f3ec2c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:23.611118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.691730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c5e28-7244-469c-ac7d-9d5193bb8dcb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.693164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.693958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.694356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.694747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] PATH_INFO: `/v3/users/3d54e02a4fe049d89acf354ed6ef62ad` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.696226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d54e02a4fe049d89acf354ed6ef62ad)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.698175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.699047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.699341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.699782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.700063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.700707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.702325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.715316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3285/6567] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:23.719232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.732413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9504b-762c-4628-b0a7-448572f79733 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.738893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed8ee-e1c6-49da-b861-62cf08ffccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.739614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed8ee-e1c6-49da-b861-62cf08ffccb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.739898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed8ee-e1c6-49da-b861-62cf08ffccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.740184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed8ee-e1c6-49da-b861-62cf08ffccb8 admin admin] PATH_INFO: `/v3/projects/d158ddef23304d7db041fba545b105d8/users/1d4b8a59bc50499784557007e2f3ec2c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.743014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ed8ee-e1c6-49da-b861-62cf08ffccb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d158ddef23304d7db041fba545b105d8, user_id=1d4b8a59bc50499784557007e2f3ec2c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.747131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ed8ee-e1c6-49da-b861-62cf08ffccb8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:23.765662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d54e02a4fe049d89acf354ed6ef62ad'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.769514 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3286/6568] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:23 2022] PUT /identity/v3/projects/d158ddef23304d7db041fba545b105d8/users/1d4b8a59bc50499784557007e2f3ec2c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:23.770792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a551d-63b7-47f0-8891-09d5c9ff82dc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.771114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a551d-63b7-47f0-8891-09d5c9ff82dc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.771364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a551d-63b7-47f0-8891-09d5c9ff82dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.784531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:23.794318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d54e02a4fe049d89acf354ed6ef62ad'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.795489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d54e02a4fe049d89acf354ed6ef62ad'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.796663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:23.806494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-519a551d-63b7-47f0-8891-09d5c9ff82dc None None] MFA Rules not processed for user `1d4b8a59bc50499784557007e2f3ec2c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:23.808487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-519a551d-63b7-47f0-8891-09d5c9ff82dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:23.810550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d54e02a4fe049d89acf354ed6ef62ad'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.811179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7e510420-77d5-412c-b97a-372aac274c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d54e02a4fe049d89acf354ed6ef62ad'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:23.843303 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3283/6569] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:23 2022] DELETE /identity/v3/users/3d54e02a4fe049d89acf354ed6ef62ad => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:23.847931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.848365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.848619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.880878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:23.882536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:23.901923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3287/6570] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:23.906856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a551d-63b7-47f0-8891-09d5c9ff82dc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.916723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a551d-63b7-47f0-8891-09d5c9ff82dc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:23.959332 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3284/6571] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:23.974077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:23.975938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:23.989920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:23.990624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:23.991039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:23.991388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:23.992116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:23.993786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invalid input for field 'name': '' is too short Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: On instance['name']: Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: On instance['name']: Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: '' Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise error Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application '' Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.001224 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:15:24.002656 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application '' Sep 10 21:15:24.003671 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.007275 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3288/6572] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:15:23 2022] POST /identity/v3/domains => generated 283 bytes in 100 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 10 21:15:24.007744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.019244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479216fc-427b-44ff-8973-14e026f665a0 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.027149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.028139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.028530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.028882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.029630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.031559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: On instance['name']: Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: On instance['name']: Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise error Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.038696 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 10 21:15:24.039877 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application On instance['name']: Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 10 21:15:24.040912 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.044206 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3289/6573] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:15:24 2022] POST /identity/v3/domains => generated 412 bytes in 37 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 10 21:15:24.045141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.056053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721a006-0930-4300-9f48-2b11889a2ed5 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.061805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.062396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.062685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.062967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.063797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.065410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.077172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f3e7f-ee96-43e7-a870-bee8dd350379 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.081928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.082918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.083497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.084686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.085902 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3290/6574] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:15:24 2022] POST /identity/v3/domains => generated 276 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:24.090422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.098385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.100635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.116314 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3285/6575] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:15:24.177519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef9a0-8dc3-403f-a5a8-8803e3a07a4c tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.191706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.192378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.192854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.193084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains/f5cf014665b24e40bb7bbc797863db30` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.225853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f5cf014665b24e40bb7bbc797863db30)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.225853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Sep 10 21:15:24.238104 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 10 21:15:24.239178 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.240332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3291/6576] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:15:24 2022] DELETE /identity/v3/domains/f5cf014665b24e40bb7bbc797863db30 => generated 120 bytes in 150 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:15:24.244001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.258073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792f194-e0bd-4de1-9502-ba49dadd8237 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.267230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.267873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.268159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.268435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains/f5cf014665b24e40bb7bbc797863db30` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.269699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f5cf014665b24e40bb7bbc797863db30)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.272117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.296209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.296981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.337457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.338989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.345222 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3286/6577] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:15:24 2022] PATCH /identity/v3/domains/f5cf014665b24e40bb7bbc797863db30 => generated 277 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:24.349119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.351480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:24.370071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d486fa0-c6a6-4f78-8020-8fd03bb5854b tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.391774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:24.485045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.485692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.486201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.486487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains/f5cf014665b24e40bb7bbc797863db30` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.493168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.496431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:24.499292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f5cf014665b24e40bb7bbc797863db30)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.504421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.549397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.552506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:24.555728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.592803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf4cc8-a185-4f8a-b4a2-f0cfe78e6f5e tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.598966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.599633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.599915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.600195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] PATH_INFO: `/v3/projects/b32d92ed96cb4b289d4cf1b80ab78136` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.601340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32d92ed96cb4b289d4cf1b80ab78136)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.603408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.622446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30', 'request_id': 'req-c031af63-2d8f-4409-bdbc-ac4675548a59'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.637313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30', 'request_id': 'req-c031af63-2d8f-4409-bdbc-ac4675548a59'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.649319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30', 'request_id': 'req-c031af63-2d8f-4409-bdbc-ac4675548a59'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.662696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32d92ed96cb4b289d4cf1b80ab78136', 'request_id': 'req-ff507e39-e824-47ad-8dba-64d0fe0ef980'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.665791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] The token cache is being invalidate because project f5cf014665b24e40bb7bbc797863db30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:24.666381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.671284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32d92ed96cb4b289d4cf1b80ab78136', 'request_id': 'req-ff507e39-e824-47ad-8dba-64d0fe0ef980'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.675270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.684515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32d92ed96cb4b289d4cf1b80ab78136', 'request_id': 'req-ff507e39-e824-47ad-8dba-64d0fe0ef980'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.686293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30', 'request_id': 'req-c031af63-2d8f-4409-bdbc-ac4675548a59'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.687231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30', 'request_id': 'req-c031af63-2d8f-4409-bdbc-ac4675548a59'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.694010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f5cf014665b24e40bb7bbc797863db30', 'request_id': 'req-c031af63-2d8f-4409-bdbc-ac4675548a59'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.703408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3292/6578] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:15:24 2022] DELETE /identity/v3/domains/f5cf014665b24e40bb7bbc797863db30 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:24.708263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.709998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:24.722592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] The token cache is being invalidate because project b32d92ed96cb4b289d4cf1b80ab78136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:24.722974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.725426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ff507e39-e824-47ad-8dba-64d0fe0ef980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:24.727221 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3287/6579] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:24 2022] DELETE /identity/v3/projects/b32d92ed96cb4b289d4cf1b80ab78136 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:24.733403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c813c9-acfe-436b-942c-b817202aff18 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.733765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c813c9-acfe-436b-942c-b817202aff18 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.734130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c813c9-acfe-436b-942c-b817202aff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.761537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-03c813c9-acfe-436b-942c-b817202aff18 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:24.762931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-03c813c9-acfe-436b-942c-b817202aff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:24.782615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:24.806408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031af63-2d8f-4409-bdbc-ac4675548a59 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.845861 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3288/6580] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:24.854343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4770161-fd97-494b-bf33-f966c948481b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.855538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4770161-fd97-494b-bf33-f966c948481b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.855538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4770161-fd97-494b-bf33-f966c948481b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.885546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f4770161-fd97-494b-bf33-f966c948481b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:24.887246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f4770161-fd97-494b-bf33-f966c948481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:24.888706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.889550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.890123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.890259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains/575c54171d184c19be68ccb820797c34` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.897621 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3289/6581] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:24.902750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4770161-fd97-494b-bf33-f966c948481b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.908618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Could not find domain: 575c54171d184c19be68ccb820797c34.: keystone.exception.DomainNotFound: Could not find domain: 575c54171d184c19be68ccb820797c34. Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 575c54171d184c19be68ccb820797c34. Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:24.914235 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:24.915433 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:15:24.917348 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 575c54171d184c19be68ccb820797c34. Sep 10 21:15:24.918600 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:24.918684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4770161-fd97-494b-bf33-f966c948481b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.920151 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3293/6582] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:15:24 2022] DELETE /identity/v3/domains/575c54171d184c19be68ccb820797c34 => generated 112 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:24.925318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.926159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.927318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.927638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:24.928290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.928582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.929532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:24.929644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.932475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.945557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877da20-1ffb-4be2-af4d-937549111fdb tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.949786 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3290/6583] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:15:24.954509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:24.955500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.956554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.958502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.958790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.959480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.961129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:24.974316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b3773-4cb1-46c4-ab8f-e5e8654cfa8e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:24.980474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:24.981163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:24.981544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:24.981923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:24.982632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:24.984746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.002130 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3294/6584] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:15:24 2022] POST /identity/v3/domains => generated 253 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:25.005235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.024998 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3291/6585] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:24 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:25.030117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.067443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:25.109360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead012f-d5db-4690-8f41-a1f0aa478144 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.144268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e993-7046-4a0c-ba53-6c682aadabc4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.156456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.156456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.156456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.156976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.157812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.159910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.184818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.185452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.185786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.186100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.187553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.187669 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3292/6586] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:25 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:25.188670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.190789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.196934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.sql.core [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1167277214' for key 'project.ixu_project_name_domain_id'") Sep 10 21:15:25.196934 ncdv-master-1 devstack@keystone.service[93527]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 21:15:25.196934 ncdv-master-1 devstack@keystone.service[93527]: [parameters: {'id': '0096dfc63a0547c6a47b6ef5622ebd6b', 'name': 'tempest-domain-dup-1167277214', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 10 21:15:25.196934 ncdv-master-1 devstack@keystone.service[93527]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93527) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 21:15:25.203193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673571f-2300-4ba4-b8df-66bf2639d503 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.208085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.208945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.209231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.209508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.210177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.211803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.223282 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3293/6587] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:15:25.227334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.238964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59327552-f40e-4cf8-9dba-d9f3cf91ec9f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.245364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.246646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.246960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.247211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.248171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.249937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.262529 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3294/6588] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:15:25.266452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.278357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27f693-ac24-4516-9d09-f4fba4b0f733 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.283207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d8add-7e08-4c2a-8d53-b639c34dd793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.283966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8add-7e08-4c2a-8d53-b639c34dd793 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.284281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8add-7e08-4c2a-8d53-b639c34dd793 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.284628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8add-7e08-4c2a-8d53-b639c34dd793 admin admin] PATH_INFO: `/v3/projects/fe0d9347a2d64323bb563b6f1f0b0cd6/users/93648515d8fb42a7aaf61e245db0e07d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1167277214.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1167277214. Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application conn.query(q) Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 21:15:25.301727 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application result.read() Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-1167277214' for key 'project.ixu_project_name_domain_id'") Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 10 21:15:25.302871 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application next(self.gen) Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application yield resource Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application next(self.gen) Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise self.value Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application session.commit() Sep 10 21:15:25.304085 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.session.flush() Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application compat.raise_( Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 21:15:25.305243 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 21:15:25.306334 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application conn.query(q) Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application result.read() Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 21:15:25.307382 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1167277214' for key 'project.ixu_project_name_domain_id'") Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [parameters: {'id': '0096dfc63a0547c6a47b6ef5622ebd6b', 'name': 'tempest-domain-dup-1167277214', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.308634 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-1167277214. Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:25.309414 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1167277214. Sep 10 21:15:25.310581 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.311524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d8add-7e08-4c2a-8d53-b639c34dd793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe0d9347a2d64323bb563b6f1f0b0cd6, user_id=93648515d8fb42a7aaf61e245db0e07d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.311524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d8add-7e08-4c2a-8d53-b639c34dd793 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.311687 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3295/6589] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:15:25 2022] POST /identity/v3/domains => generated 212 bytes in 308 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 21:15:25.314315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.327714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e88e9f-01dd-469e-85ff-0b50d7bc09fa tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.329681 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3295/6590] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:25 2022] PUT /identity/v3/projects/fe0d9347a2d64323bb563b6f1f0b0cd6/users/93648515d8fb42a7aaf61e245db0e07d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:25.340944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.341329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.341609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.363314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:25.377082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] MFA Rules not processed for user `93648515d8fb42a7aaf61e245db0e07d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:25.378828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:25.411872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.412627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.412937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.413226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains/f8542373a2cb4cd6aeadb4f3c70b6e45` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.427748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f8542373a2cb4cd6aeadb4f3c70b6e45)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.429539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.456429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.457951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.486800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.487538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.487938 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3296/6591] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:25.490618 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3296/6592] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:15:25 2022] PATCH /identity/v3/domains/f8542373a2cb4cd6aeadb4f3c70b6e45 => generated 254 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:25.494332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.494475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.495763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:25.495899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:25.573940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:25.581968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dc014-e93c-4dd9-831a-098c840294ca tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.585307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe9ebb-0690-4400-bd2a-04ff71464b05 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.587071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.587713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.588632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.588917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains/f8542373a2cb4cd6aeadb4f3c70b6e45` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.601777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f8542373a2cb4cd6aeadb4f3c70b6e45)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.604406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.634395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.638083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.666735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.667817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.668209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.668596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.669511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.671684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.701319 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3297/6593] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:25 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:25.702950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45', 'request_id': 'req-0fd06496-7572-4bfd-8fb9-a0a00485c71d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.704888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:25.705496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.707271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:25.710133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45', 'request_id': 'req-0fd06496-7572-4bfd-8fb9-a0a00485c71d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.717882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45', 'request_id': 'req-0fd06496-7572-4bfd-8fb9-a0a00485c71d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.733338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] The token cache is being invalidate because project f8542373a2cb4cd6aeadb4f3c70b6e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:25.733633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.734318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.743198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45', 'request_id': 'req-0fd06496-7572-4bfd-8fb9-a0a00485c71d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.743859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45', 'request_id': 'req-0fd06496-7572-4bfd-8fb9-a0a00485c71d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.748072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f8542373a2cb4cd6aeadb4f3c70b6e45', 'request_id': 'req-0fd06496-7572-4bfd-8fb9-a0a00485c71d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:25.758153 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3297/6594] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:15:25 2022] DELETE /identity/v3/domains/f8542373a2cb4cd6aeadb4f3c70b6e45 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:25.763192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.764640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:25.774323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7253657-9597-4c8c-9258-f6ddc9d4bb77 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.779671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.780167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.780452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.780728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.781354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.783329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.804881 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3298/6595] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:25 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:25.809472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.811155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:25.823590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51debab-3a5c-4765-8db3-02cda73db537 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.828509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.829157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.829445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.829722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.830403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.831995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.834599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd06496-7572-4bfd-8fb9-a0a00485c71d tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.846438 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3299/6596] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:15:25.847920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.859460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493821a-7194-4475-9187-6b032358cee1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.864207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.864879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.865165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.865449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] PATH_INFO: `/v3/projects/2ba991d7c6a946739e76f00bbbf420ba/users/9e2e31abcac04c0bb71fbed775b80d6f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.881149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba991d7c6a946739e76f00bbbf420ba, user_id=9e2e31abcac04c0bb71fbed775b80d6f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.883799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.894838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d4b8a59bc50499784557007e2f3ec2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd158ddef23304d7db041fba545b105d8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.895469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.895753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.896020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] PATH_INFO: `/v3/domains/f5cf014665b24e40bb7bbc797863db30` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.896748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3300/6597] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:25 2022] PUT /identity/v3/projects/2ba991d7c6a946739e76f00bbbf420ba/users/9e2e31abcac04c0bb71fbed775b80d6f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:25.900682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.906354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Could not find domain: f5cf014665b24e40bb7bbc797863db30.: keystone.exception.DomainNotFound: Could not find domain: f5cf014665b24e40bb7bbc797863db30. Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f5cf014665b24e40bb7bbc797863db30. Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:25.911452 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:25.912488 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:15:25.913435 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f5cf014665b24e40bb7bbc797863db30. Sep 10 21:15:25.914648 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:25.914648 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3298/6598] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:15:25 2022] PATCH /identity/v3/domains/f5cf014665b24e40bb7bbc797863db30 => generated 112 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:15:25.916897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.918727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:25.933475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369718f-fa89-4b0b-b8d3-b49cb3be1365 tempest-DomainsNegativeTestJSON-63886734 tempest-DomainsNegativeTestJSON-63886734-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.949267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:25.967974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640fc5-c069-4a4d-93fc-2d8807e8f123 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:25.972983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.973753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.974082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.974364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.975009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.976617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:25.987957 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3301/6599] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:15:25.991980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:25.995515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:25.996168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:25.996438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:25.996717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] PATH_INFO: `/v3/users/f25963e8ba5943e080f6b80562ecf25c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:25.997888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25963e8ba5943e080f6b80562ecf25c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:25.999843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:26.002365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:26.002507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac771b-418d-41fb-be9d-f9898618c432 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:26.007664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f748c-032a-454a-aab1-8d7582b9bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:26.008281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f748c-032a-454a-aab1-8d7582b9bf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:26.008580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f748c-032a-454a-aab1-8d7582b9bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:26.009025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f748c-032a-454a-aab1-8d7582b9bf3d admin admin] PATH_INFO: `/v3/projects/2ba991d7c6a946739e76f00bbbf420ba/users/9e2e31abcac04c0bb71fbed775b80d6f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:26.012278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f748c-032a-454a-aab1-8d7582b9bf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba991d7c6a946739e76f00bbbf420ba, user_id=9e2e31abcac04c0bb71fbed775b80d6f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:26.015483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f748c-032a-454a-aab1-8d7582b9bf3d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:26.027833 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3302/6600] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:25 2022] PUT /identity/v3/projects/2ba991d7c6a946739e76f00bbbf420ba/users/9e2e31abcac04c0bb71fbed775b80d6f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:26.040378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:26.040691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:26.040980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:26.053423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25963e8ba5943e080f6b80562ecf25c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.067137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25963e8ba5943e080f6b80562ecf25c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.070922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] MFA Rules not processed for user `9e2e31abcac04c0bb71fbed775b80d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:26.072565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:26.074631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25963e8ba5943e080f6b80562ecf25c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.082891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25963e8ba5943e080f6b80562ecf25c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.083556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-534e6b21-f29c-4daa-b4ef-c37ba84a3dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25963e8ba5943e080f6b80562ecf25c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.091436 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3299/6601] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:25 2022] DELETE /identity/v3/users/f25963e8ba5943e080f6b80562ecf25c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:26.096090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:26.096384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:26.096663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:26.116013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:26.121058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:26.166897 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3303/6602] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:26.178409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:26.180724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:26.196209 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3300/6603] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:26.203485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:26.205314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:26.249316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4cc26-7f17-4437-b18f-e1212500d6e4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:26.277651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f361a08-2d60-4603-bac3-613c394330fe None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:26.282701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:26.283650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:26.283944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:26.284224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:26.295609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:26.297215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:26.309317 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3301/6604] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:15:26.310551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:26.311183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:26.311470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:26.311749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:26.313554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:26.339200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:26.394015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:26.395689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:26.406722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3304/6605] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:15:26.763022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:26.764548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:26.783455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a8a78-d23f-465d-8348-1698746b06a4 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:26.792921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:26.793764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:26.794120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:26.794401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] PATH_INFO: `/v3/projects/47bf53336d3c4159baf93bbfadc2035c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:26.795566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bf53336d3c4159baf93bbfadc2035c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:26.797891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:26.830453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:26.875521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bf53336d3c4159baf93bbfadc2035c', 'request_id': 'req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.882307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bf53336d3c4159baf93bbfadc2035c', 'request_id': 'req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.889068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bf53336d3c4159baf93bbfadc2035c', 'request_id': 'req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.903015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] The token cache is being invalidate because project 47bf53336d3c4159baf93bbfadc2035c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:26.903307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.903955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:26.905459 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3302/6606] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:26 2022] DELETE /identity/v3/projects/47bf53336d3c4159baf93bbfadc2035c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:26.909527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:26.922991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcefa9-b8f3-4fea-8b05-f636c6a50191 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:26.999961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:27.000841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:27.002253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:27.002812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] PATH_INFO: `/v3/users/1d4b8a59bc50499784557007e2f3ec2c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:27.004150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4b8a59bc50499784557007e2f3ec2c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:27.006313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:27.066822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4b8a59bc50499784557007e2f3ec2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.075066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4b8a59bc50499784557007e2f3ec2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.075850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4b8a59bc50499784557007e2f3ec2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.083675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4b8a59bc50499784557007e2f3ec2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.084966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3753f0c7-8edf-411c-9177-729213f53d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4b8a59bc50499784557007e2f3ec2c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.112439 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3305/6607] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:26 2022] DELETE /identity/v3/users/1d4b8a59bc50499784557007e2f3ec2c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:27.408825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:27.410766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:27.425820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afc8ab-da07-4f00-a4e0-e5b25d72d21e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:27.492200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:27.492752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:27.493069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:27.493470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] PATH_INFO: `/v3/projects/d158ddef23304d7db041fba545b105d8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:27.494874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d158ddef23304d7db041fba545b105d8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:27.496799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:27.549980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd158ddef23304d7db041fba545b105d8', 'request_id': 'req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.558018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd158ddef23304d7db041fba545b105d8', 'request_id': 'req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.565151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd158ddef23304d7db041fba545b105d8', 'request_id': 'req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.579429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] The token cache is being invalidate because project d158ddef23304d7db041fba545b105d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:27.579829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.580409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34dfb25c-15d6-41a2-a2b6-a7de99190a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:27.585256 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3303/6608] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:27 2022] DELETE /identity/v3/projects/d158ddef23304d7db041fba545b105d8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:27.588105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ffc78-f386-4d9f-a456-5ffcb3483f15 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:27.588919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ffc78-f386-4d9f-a456-5ffcb3483f15 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:27.588919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ffc78-f386-4d9f-a456-5ffcb3483f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:27.610216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8d2ffc78-f386-4d9f-a456-5ffcb3483f15 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:27.611517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8d2ffc78-f386-4d9f-a456-5ffcb3483f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:27.649794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:27.680450 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3306/6609] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:27.688646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9fb7-9035-4c36-9783-e7b49a10b200 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:27.688941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9fb7-9035-4c36-9783-e7b49a10b200 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:27.689241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9fb7-9035-4c36-9783-e7b49a10b200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:27.714733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-465b9fb7-9035-4c36-9783-e7b49a10b200 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:27.715469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-465b9fb7-9035-4c36-9783-e7b49a10b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:27.724580 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3304/6610] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:27.730410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ffc78-f386-4d9f-a456-5ffcb3483f15 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:27.744100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ffc78-f386-4d9f-a456-5ffcb3483f15 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:27.760783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:27.762748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:27.763287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:27.763840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:27.764722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:27.765243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:27.767188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:27.793721 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3307/6611] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:27.797850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9fb7-9035-4c36-9783-e7b49a10b200 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:27.810801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b9fb7-9035-4c36-9783-e7b49a10b200 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:27.821738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:27.823152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:27.823152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:27.823152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:27.824424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:27.825604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:27.887457 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3305/6612] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:27 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:27.888810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:27.907574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4d4a6-c048-4965-832b-e764f3f5d232 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:27.985954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:27.986587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:27.986925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:27.987136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:27.987954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:27.990122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.033785 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3308/6613] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:27 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:28.038714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.056082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ef5e6-3ebf-45dc-ad4f-4f56387f12ba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.070648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.071402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.071697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.071991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.073227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.079937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.082894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:28.254840 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3306/6614] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:28.258807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.268914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d17e78-0272-40e3-bb5c-88e7fdadaea7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.273503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba66b0-dc30-4460-b718-00aa33fe364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.274143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba66b0-dc30-4460-b718-00aa33fe364e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.274428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba66b0-dc30-4460-b718-00aa33fe364e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.274705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba66b0-dc30-4460-b718-00aa33fe364e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.275911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba66b0-dc30-4460-b718-00aa33fe364e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.277645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba66b0-dc30-4460-b718-00aa33fe364e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.288791 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3309/6615] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:28.293647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.304211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68815ea3-36e7-4658-a58b-67c98ebe5af0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.309042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.309775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.310164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.310508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] PATH_INFO: `/v3/projects/42d72778dd2b432ea38b61a62a37082b/users/d95e1912f1a64de997c99c3d671d9256/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.328242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d72778dd2b432ea38b61a62a37082b, user_id=d95e1912f1a64de997c99c3d671d9256, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.331627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.352969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3307/6616] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:28 2022] PUT /identity/v3/projects/42d72778dd2b432ea38b61a62a37082b/users/d95e1912f1a64de997c99c3d671d9256/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:28.442215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64998489-8930-4477-9afb-8acafba9d78c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.442584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64998489-8930-4477-9afb-8acafba9d78c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.442769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64998489-8930-4477-9afb-8acafba9d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.479445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-64998489-8930-4477-9afb-8acafba9d78c None None] MFA Rules not processed for user `d95e1912f1a64de997c99c3d671d9256`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:28.480931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64998489-8930-4477-9afb-8acafba9d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:28.555058 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3310/6617] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:28.560414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.628148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54881174-4be4-477a-9806-325add8f95ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.632925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.633546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.633992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.634278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.635494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.637546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.671320 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3308/6618] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:28 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:28.672785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64998489-8930-4477-9afb-8acafba9d78c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.685142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64998489-8930-4477-9afb-8acafba9d78c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.719642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:28.748514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.749397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.749635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.750018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.750744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.755713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.777256 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3311/6619] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:28 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:28.781671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.793238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767caa8-23e4-4f0d-a6f8-080953fec081 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.798099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.798791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.799137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.799484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.800123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.801881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.813512 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3309/6620] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:15:28.818101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.828481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee696adc-a6ef-4d6c-ba8c-3cbcc48b0d21 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.833286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.833978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.834266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.834544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] PATH_INFO: `/v3/projects/4eda3311a5a540b18142b01326e53f01/users/d221383aea734a08a54eceaa931d06c8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.850014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eda3311a5a540b18142b01326e53f01, user_id=d221383aea734a08a54eceaa931d06c8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.852759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.867422 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3312/6621] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:28 2022] PUT /identity/v3/projects/4eda3311a5a540b18142b01326e53f01/users/d221383aea734a08a54eceaa931d06c8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:28.872076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.883366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8a9c4-3bc9-420a-a25a-78f623aab930 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.941012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546b521-106b-4501-91a7-24ccbb7c4906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.941615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b521-106b-4501-91a7-24ccbb7c4906 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.942391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b521-106b-4501-91a7-24ccbb7c4906 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.942391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546b521-106b-4501-91a7-24ccbb7c4906 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.942969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546b521-106b-4501-91a7-24ccbb7c4906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.944598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546b521-106b-4501-91a7-24ccbb7c4906 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:28.955647 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3310/6622] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:28.959810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:28.976942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b4f5c-5452-4eb4-b6e6-b6415bb99c7c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:28.985238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:28.986307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:28.986767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:28.987056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] PATH_INFO: `/v3/projects/4eda3311a5a540b18142b01326e53f01/users/d221383aea734a08a54eceaa931d06c8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:28.989294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eda3311a5a540b18142b01326e53f01, user_id=d221383aea734a08a54eceaa931d06c8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:28.992666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.046427 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3313/6623] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:28 2022] PUT /identity/v3/projects/4eda3311a5a540b18142b01326e53f01/users/d221383aea734a08a54eceaa931d06c8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:29.049157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.049332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.049626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.050121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.051362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.099869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] MFA Rules not processed for user `d221383aea734a08a54eceaa931d06c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:29.103252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.138753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da066c-cda6-46cd-9541-2c9ef97c72cf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.148902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.150841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.153609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.156204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.164845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.169337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:29.318898 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3311/6624] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:29.324064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.327048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.350049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.352200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.364960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3314/6625] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:29.365974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.378363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2feec10-997d-430e-9e46-fb2c4f438469 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.384498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.385805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.386298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.387631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.387631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.388277 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:29.388277 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:15:29.388277 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:15:29.389368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.415717 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3315/6626] 10.222.0.33 () {60 vars in 1185 bytes} [Sat Sep 10 21:15:29 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:29.420031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.443965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381fa29-36f0-44c7-a222-f55672d2fc56 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.458531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.459443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.460491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.461640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f7711f3e28874b65b9b740dd54b87ee3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.462380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=f7711f3e28874b65b9b740dd54b87ee3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.465756 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:29.465756 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:15:29.465756 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:15:29.467592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.484902 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3316/6627] 10.222.0.33 () {58 vars in 1263 bytes} [Sat Sep 10 21:15:29 2022] GET /identity/v3/OS-OAUTH1/consumers/f7711f3e28874b65b9b740dd54b87ee3 => generated 220 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:29.488785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.495778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966ba35-6f85-4d8d-b390-c32ef79228ae None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.506083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c4de-06e4-4e0f-9acb-1d5d268e479e tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.507526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.508248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.508562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.508861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] PATH_INFO: `/v3/users/a16673f69a1445f982ad5c5c800ca3af` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.511126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16673f69a1445f982ad5c5c800ca3af)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.516337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.520708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.521355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.521635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.522067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/f7711f3e28874b65b9b740dd54b87ee3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.522658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=f7711f3e28874b65b9b740dd54b87ee3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.525527 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:29.525527 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:15:29.525527 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:15:29.527648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.527938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invalidating token cache because consumer f7711f3e28874b65b9b740dd54b87ee3 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:29.528226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.530212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.566630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:29.578979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'f7711f3e28874b65b9b740dd54b87ee3', 'request_id': 'req-ba82c80d-1917-4150-8bc2-3ea845c2b24b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.603053 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3317/6628] 10.222.0.33 () {58 vars in 1266 bytes} [Sat Sep 10 21:15:29 2022] DELETE /identity/v3/OS-OAUTH1/consumers/f7711f3e28874b65b9b740dd54b87ee3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:29.609045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.610768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.628405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82c80d-1917-4150-8bc2-3ea845c2b24b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.639228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.640494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.642443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.644259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.645220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.648585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.661448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16673f69a1445f982ad5c5c800ca3af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.688789 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3318/6629] 10.222.0.33 () {60 vars in 1185 bytes} [Sat Sep 10 21:15:29 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:29.694611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.710974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16673f69a1445f982ad5c5c800ca3af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.720130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374242e-0ffd-40fc-90b7-2611f44bbeb2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.729618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16673f69a1445f982ad5c5c800ca3af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.735019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.735684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.735967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.736242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.737087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=b5b7a74eda9f4d2ea04c8fa8359b9630)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.740485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.750391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16673f69a1445f982ad5c5c800ca3af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.755145 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3319/6630] 10.222.0.33 () {58 vars in 1263 bytes} [Sat Sep 10 21:15:29 2022] GET /identity/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630 => generated 221 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:29.756776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc04196f-e037-429a-84ac-06f418c99d74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16673f69a1445f982ad5c5c800ca3af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.760968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.765465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.774386 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3312/6631] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:29 2022] DELETE /identity/v3/users/a16673f69a1445f982ad5c5c800ca3af => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:29.777600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.777937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.778220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.802776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37886b2f-65d0-4d73-9e95-59746e1b91e4 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:29.804769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:29.807166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.872003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:29.906985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3313/6632] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:29.940250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.942237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:29.942914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:29.943201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:29.943488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:29.943815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:29.944309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=b5b7a74eda9f4d2ea04c8fa8359b9630)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:29.946654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:29.946936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invalidating token cache because consumer b5b7a74eda9f4d2ea04c8fa8359b9630 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:29.947889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.949757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.966266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'b5b7a74eda9f4d2ea04c8fa8359b9630', 'request_id': 'req-669ab353-1f7a-4b01-9404-a672263707a2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:29.987867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3320/6633] 10.222.0.33 () {58 vars in 1266 bytes} [Sat Sep 10 21:15:29 2022] DELETE /identity/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:29.992872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:29.995652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:30.016868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ab353-1f7a-4b01-9404-a672263707a2 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.024308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.024945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.026221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.026533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.027215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=b5b7a74eda9f4d2ea04c8fa8359b9630)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.031655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.048274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaf1af-e512-4ac3-bf62-775e02c8be53 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.052128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.053031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.053310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.053582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.055735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Sep 10 21:15:30.063301 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 10 21:15:30.064517 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:15:30.071508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3321/6634] 10.222.0.33 () {58 vars in 1263 bytes} [Sat Sep 10 21:15:29 2022] GET /identity/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630 => generated 74 bytes in 76 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 21:15:30.075796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.077683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.082022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.098493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead81341-14f8-4946-b8be-46642eb3b45c tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.105163 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3314/6635] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:30.112489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.113547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.114181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.115128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.115992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=b5b7a74eda9f4d2ea04c8fa8359b9630)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.118252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.118781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invalidating token cache because consumer b5b7a74eda9f4d2ea04c8fa8359b9630 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:30.121287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.122987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Sep 10 21:15:30.152139 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 10 21:15:30.153245 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:15:30.160742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.169170 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3322/6636] 10.222.0.33 () {58 vars in 1266 bytes} [Sat Sep 10 21:15:30 2022] DELETE /identity/v3/OS-OAUTH1/consumers/b5b7a74eda9f4d2ea04c8fa8359b9630 => generated 74 bytes in 94 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 21:15:30.173705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc40e21-a60a-4cff-9ce9-61fed8c0ee21 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.179347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.179986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.180271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.180550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.181201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.182382 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:30.182382 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:15:30.182382 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:15:30.183287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.214740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.216487 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3315/6637] 10.222.0.33 () {60 vars in 1185 bytes} [Sat Sep 10 21:15:30 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:30.217452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:30.235389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a69e-d8cb-4447-9edf-f537fab96170 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.246732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.247374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.247713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.247994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.248703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.252178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.273167 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3323/6638] 10.222.0.33 () {60 vars in 1185 bytes} [Sat Sep 10 21:15:30 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:30.277659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.290598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3ddae-4564-468e-ba8d-22e5160f4f45 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.301886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.302549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.302821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.303104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.303683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.304707 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:30.304707 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:15:30.304707 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:15:30.305893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.317144 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3316/6639] 10.222.0.33 () {58 vars in 1164 bytes} [Sat Sep 10 21:15:30 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 536 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:30.317712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.330187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d89f3-3b6d-4853-93f2-29b801f66af3 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.346991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.361953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.363323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.363712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/9434dba12e18460b9abbfc526c8fda01` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.365104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=9434dba12e18460b9abbfc526c8fda01)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.372207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.374773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invalidating token cache because consumer 9434dba12e18460b9abbfc526c8fda01 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:30.375967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.378768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.415482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '9434dba12e18460b9abbfc526c8fda01', 'request_id': 'req-b38c8f33-805a-4acb-9161-ef347efe221a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.442178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3324/6640] 10.222.0.33 () {58 vars in 1266 bytes} [Sat Sep 10 21:15:30 2022] DELETE /identity/v3/OS-OAUTH1/consumers/9434dba12e18460b9abbfc526c8fda01 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:30.446876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.450849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:30.467470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e627e-b979-4655-80f8-9518d8391bad tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.472934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.473723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.474165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.474515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/1b1dafc2912a4749963cba8ce1e13460` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.475177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=1b1dafc2912a4749963cba8ce1e13460)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.476562 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:30.476562 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:15:30.476562 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:15:30.477581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.477990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invalidating token cache because consumer 1b1dafc2912a4749963cba8ce1e13460 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:30.478969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.479741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.504227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '1b1dafc2912a4749963cba8ce1e13460', 'request_id': 'req-d2237847-0806-4364-8a88-2ca7bf97e31b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.540820 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3317/6641] 10.222.0.33 () {58 vars in 1266 bytes} [Sat Sep 10 21:15:30 2022] DELETE /identity/v3/OS-OAUTH1/consumers/1b1dafc2912a4749963cba8ce1e13460 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:30.541461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.554374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c8f33-805a-4acb-9161-ef347efe221a tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.561287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.561795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.562139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.562540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.564061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.566297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.586911 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3325/6642] 10.222.0.33 () {60 vars in 1185 bytes} [Sat Sep 10 21:15:30 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:30.590558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.592729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:30.617437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2237847-0806-4364-8a88-2ca7bf97e31b tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.624281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.625201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.625637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.626104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/e3d7c395bd364e26907c48d9a73114b8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.627525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=e3d7c395bd364e26907c48d9a73114b8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.629520 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:30.629520 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:15:30.629520 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:15:30.630763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.658597 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3318/6643] 10.222.0.33 () {60 vars in 1285 bytes} [Sat Sep 10 21:15:30 2022] PATCH /identity/v3/OS-OAUTH1/consumers/e3d7c395bd364e26907c48d9a73114b8 => generated 221 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:15:30.658827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.674889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc669385-4688-4ee3-a06c-679d6dfc710f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.683165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.684528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.685002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.685482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/e3d7c395bd364e26907c48d9a73114b8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.686484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=e3d7c395bd364e26907c48d9a73114b8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.688432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.701183 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3326/6644] 10.222.0.33 () {58 vars in 1263 bytes} [Sat Sep 10 21:15:30 2022] GET /identity/v3/OS-OAUTH1/consumers/e3d7c395bd364e26907c48d9a73114b8 => generated 221 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:30.703844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.731781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cb8fe-fb30-498a-b69e-29cc205a69d1 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.737378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd221383aea734a08a54eceaa931d06c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4eda3311a5a540b18142b01326e53f01', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.738239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.738630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.741616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/e3d7c395bd364e26907c48d9a73114b8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.742456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=e3d7c395bd364e26907c48d9a73114b8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.754142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.754142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invalidating token cache because consumer e3d7c395bd364e26907c48d9a73114b8 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:30.754142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.755103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.779100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'e3d7c395bd364e26907c48d9a73114b8', 'request_id': 'req-f5785b75-ea90-409b-8ed3-ca2a3556e56f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.798010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.799990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:30.811807 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3319/6645] 10.222.0.33 () {58 vars in 1266 bytes} [Sat Sep 10 21:15:30 2022] DELETE /identity/v3/OS-OAUTH1/consumers/e3d7c395bd364e26907c48d9a73114b8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:30.814701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:30.817130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:30.829559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe243-53b7-40f9-9db0-c5e5be0667b9 tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.835830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.836542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.836834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.837590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] PATH_INFO: `/v3/projects/3ae9868b7610448b90be6b692fd087d8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.838627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5785b75-ea90-409b-8ed3-ca2a3556e56f tempest-OAUTHConsumersV3Test-998586014 tempest-OAUTHConsumersV3Test-998586014-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:30.839122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae9868b7610448b90be6b692fd087d8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.841388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.846276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:30.847683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:30.848643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:30.849557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] PATH_INFO: `/v3/users/d95e1912f1a64de997c99c3d671d9256` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:30.852878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95e1912f1a64de997c99c3d671d9256)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:30.858199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:30.883443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:30.924799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae9868b7610448b90be6b692fd087d8', 'request_id': 'req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.934609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95e1912f1a64de997c99c3d671d9256'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.948613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=12,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:30.951379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95e1912f1a64de997c99c3d671d9256'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.962316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95e1912f1a64de997c99c3d671d9256'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.975801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae9868b7610448b90be6b692fd087d8', 'request_id': 'req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.982302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95e1912f1a64de997c99c3d671d9256'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:30.983721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e9569914-2e82-445f-9bcc-3c38d8853f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95e1912f1a64de997c99c3d671d9256'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:31.002470 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3320/6646] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:30 2022] DELETE /identity/v3/users/d95e1912f1a64de997c99c3d671d9256 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:31.006857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.007315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.007596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.018602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae9868b7610448b90be6b692fd087d8', 'request_id': 'req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:31.062513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] The token cache is being invalidate because project 3ae9868b7610448b90be6b692fd087d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:31.063080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:31.064371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c9ee3b9-8489-4fb6-a01b-43bec374dfc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:31.078173 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3327/6647] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:30 2022] DELETE /identity/v3/projects/3ae9868b7610448b90be6b692fd087d8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:31.079937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94dff8-293a-49da-bce7-c9a4a923dfe0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.081655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94dff8-293a-49da-bce7-c9a4a923dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.082709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94dff8-293a-49da-bce7-c9a4a923dfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.089408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:31.092281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:31.131410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3f94dff8-293a-49da-bce7-c9a4a923dfe0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:31.136469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3f94dff8-293a-49da-bce7-c9a4a923dfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:31.277893 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3321/6648] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:31.291105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:31.298396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:31.309750 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3328/6649] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:31.318570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc0b8c-bc36-4e85-a0f2-626578475779 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.318863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc0b8c-bc36-4e85-a0f2-626578475779 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.320647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc0b8c-bc36-4e85-a0f2-626578475779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.351086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-09fc0b8c-bc36-4e85-a0f2-626578475779 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:31.352154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-09fc0b8c-bc36-4e85-a0f2-626578475779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:31.367906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3329/6650] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:31.370217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc0b8c-bc36-4e85-a0f2-626578475779 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:31.389462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc0b8c-bc36-4e85-a0f2-626578475779 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:31.395747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:31.396388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.396734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.397041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.397682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:31.398031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:31.399669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:31.419815 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3330/6651] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:31.420836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:31.432366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490877d-ff9f-4577-bc13-8871805eb142 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:31.438837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:31.439573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.439912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.440283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.441621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:31.443909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:31.448890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571843c5-0f2f-46a8-9b78-364456e137de None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:31.457180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:31.458127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.459675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.459967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.472419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:31.472892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:31.491875 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3331/6652] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:31 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:31.498384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:31.503762 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3322/6653] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:31.614642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ec4e1-fd01-4473-bbc6-0fd4b1e6c3a3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:31.628223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:31.629223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.635388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.635911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.636926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:31.639976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:31.656660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:31.693416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3332/6654] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:31 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:31.697695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:31.713956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5164e60-ef6a-4cc4-bed7-23355a45f7dc service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:31.734585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:31.735526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.735959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.736408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.738625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:31.740427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:31.781955 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3323/6655] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:31.786018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:31.806216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984355-decc-480f-ad79-c347e23685f1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:31.814532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262528a-77ad-46b0-b610-e5c4a125c2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:31.815294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262528a-77ad-46b0-b610-e5c4a125c2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.816524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262528a-77ad-46b0-b610-e5c4a125c2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.816828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262528a-77ad-46b0-b610-e5c4a125c2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.817694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262528a-77ad-46b0-b610-e5c4a125c2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:31.834766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262528a-77ad-46b0-b610-e5c4a125c2f8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:31.868361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3333/6656] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:31.872809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:31.888606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18bf3-49fc-4ca9-bcd7-48518426941d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:31.905244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:31.907115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:31.907786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:31.908226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] PATH_INFO: `/v3/projects/3d9f98a3f93d409a94ee87eec97206c1/users/eac0a8d66da042e096cab5c715f40a27/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:31.969131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9f98a3f93d409a94ee87eec97206c1, user_id=eac0a8d66da042e096cab5c715f40a27, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:31.985486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:32.018306 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3324/6657] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:31 2022] PUT /identity/v3/projects/3d9f98a3f93d409a94ee87eec97206c1/users/eac0a8d66da042e096cab5c715f40a27/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:32.020097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a09c5-ba85-4ea1-ae69-a6ab5cca3d17 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:32.020402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a09c5-ba85-4ea1-ae69-a6ab5cca3d17 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:32.020721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a09c5-ba85-4ea1-ae69-a6ab5cca3d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:32.063264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-523a09c5-ba85-4ea1-ae69-a6ab5cca3d17 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:32.064881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-523a09c5-ba85-4ea1-ae69-a6ab5cca3d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:32.126874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:32.128854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:32.248686 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3334/6658] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:32.249362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:32.256639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a09c5-ba85-4ea1-ae69-a6ab5cca3d17 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:32.292233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ab32f-3871-4162-8000-b1f8b5bd1a4c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:32.305511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:32.306072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:32.306969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:32.307721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] PATH_INFO: `/v3/projects/42d72778dd2b432ea38b61a62a37082b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:32.309288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d72778dd2b432ea38b61a62a37082b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:32.312349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:32.439205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a09c5-ba85-4ea1-ae69-a6ab5cca3d17 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:32.447837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:32.449218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:32.449718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:32.450304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:32.472178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:32.473986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:32.487524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d72778dd2b432ea38b61a62a37082b', 'request_id': 'req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.502234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d72778dd2b432ea38b61a62a37082b', 'request_id': 'req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.503690 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3335/6659] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:32.517312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d72778dd2b432ea38b61a62a37082b', 'request_id': 'req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.543875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] The token cache is being invalidate because project 42d72778dd2b432ea38b61a62a37082b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:32.544365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.545493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.547757 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3325/6660] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:32 2022] DELETE /identity/v3/projects/42d72778dd2b432ea38b61a62a37082b => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:32.552046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:32.571649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989732d8-1d74-435b-b2ef-090ceb70a2cb service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:32.724996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:32.725539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:32.725876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:32.726118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] PATH_INFO: `/v3/users/d221383aea734a08a54eceaa931d06c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:32.727563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d221383aea734a08a54eceaa931d06c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:32.734687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:32.810631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:32.845575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd221383aea734a08a54eceaa931d06c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.858164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd221383aea734a08a54eceaa931d06c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.859595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd221383aea734a08a54eceaa931d06c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.874291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd221383aea734a08a54eceaa931d06c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.875682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-babd4dd8-8b66-40c4-b341-30bdd9ec4fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd221383aea734a08a54eceaa931d06c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:32.956546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3336/6661] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:32 2022] DELETE /identity/v3/users/d221383aea734a08a54eceaa931d06c8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:33.816993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:33.819084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:33.847075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d0fa7-a598-4253-8ef5-82aa3d99f603 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:34.019886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:34.037009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.039144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.040349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] PATH_INFO: `/v3/projects/4eda3311a5a540b18142b01326e53f01` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.044191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eda3311a5a540b18142b01326e53f01)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:34.048117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:34.140654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eda3311a5a540b18142b01326e53f01', 'request_id': 'req-a8232e70-6da4-4e25-94f1-5bad8168cad8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:34.153053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eda3311a5a540b18142b01326e53f01', 'request_id': 'req-a8232e70-6da4-4e25-94f1-5bad8168cad8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:34.164013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eda3311a5a540b18142b01326e53f01', 'request_id': 'req-a8232e70-6da4-4e25-94f1-5bad8168cad8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:34.188869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] The token cache is being invalidate because project 4eda3311a5a540b18142b01326e53f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:34.189230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:34.189964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a8232e70-6da4-4e25-94f1-5bad8168cad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:34.191848 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3326/6662] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:33 2022] DELETE /identity/v3/projects/4eda3311a5a540b18142b01326e53f01 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:34.196418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a245f1-8fab-4a50-a54c-0d9add05c66e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.196896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a245f1-8fab-4a50-a54c-0d9add05c66e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.197539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a245f1-8fab-4a50-a54c-0d9add05c66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.256837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-65a245f1-8fab-4a50-a54c-0d9add05c66e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:34.260331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-65a245f1-8fab-4a50-a54c-0d9add05c66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:34.379154 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3337/6663] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:34.383390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ae27-a97a-47a6-b9d8-b555b0709810 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.384166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ae27-a97a-47a6-b9d8-b555b0709810 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.385455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ae27-a97a-47a6-b9d8-b555b0709810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.397877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:34.422328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2f33ae27-a97a-47a6-b9d8-b555b0709810 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:34.423778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2f33ae27-a97a-47a6-b9d8-b555b0709810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:34.433955 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3327/6664] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:34.438055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a245f1-8fab-4a50-a54c-0d9add05c66e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:34.455556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a245f1-8fab-4a50-a54c-0d9add05c66e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:34.460437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:34.461062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.461341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.461698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.462403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:34.462795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:34.464970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:34.491473 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3338/6665] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:34.493257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ae27-a97a-47a6-b9d8-b555b0709810 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:34.510441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ae27-a97a-47a6-b9d8-b555b0709810 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:34.519678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:34.521052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.521359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.521654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.522453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:34.524618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:34.582679 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3328/6666] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:34 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:34.586893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:34.601432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58c27d-7474-40d0-9886-e6ca6c0a9090 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:34.694017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:34.694803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.695150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.696209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.696857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:34.699455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:34.731025 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3339/6667] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:34 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:34.735412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:34.748040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b52a7e-4df9-4e2f-843b-01ec11ff2a98 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:34.754187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:34.754812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.755099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.755368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.756016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:34.757728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:34.770366 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3329/6668] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:34.774914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:34.785680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730b554-8cea-478f-9ff1-f15e9c6ed53c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:34.791584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89522a-7bd8-4d03-a6fb-2b9b90b77832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:34.792691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89522a-7bd8-4d03-a6fb-2b9b90b77832 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.792979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89522a-7bd8-4d03-a6fb-2b9b90b77832 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.793282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89522a-7bd8-4d03-a6fb-2b9b90b77832 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.793927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89522a-7bd8-4d03-a6fb-2b9b90b77832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:34.795558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89522a-7bd8-4d03-a6fb-2b9b90b77832 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:34.809865 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3340/6669] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:34.813305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:34.824257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763be21b-95a8-4604-b8b7-5844409775d6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:34.829408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:34.830203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.830550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.830895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] PATH_INFO: `/v3/projects/c7df49e781fa4479945f9744b143e145/users/1d3ce5d199cd4f7b8065fc7bb2a10932/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.847915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7df49e781fa4479945f9744b143e145, user_id=1d3ce5d199cd4f7b8065fc7bb2a10932, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:34.850771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:34.869232 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3330/6670] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:34 2022] PUT /identity/v3/projects/c7df49e781fa4479945f9744b143e145/users/1d3ce5d199cd4f7b8065fc7bb2a10932/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:34.880187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308552f-b83f-4f24-b9c3-522b01ff1aeb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:34.880475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308552f-b83f-4f24-b9c3-522b01ff1aeb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:34.880753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308552f-b83f-4f24-b9c3-522b01ff1aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:34.919329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:34.925419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b308552f-b83f-4f24-b9c3-522b01ff1aeb None None] MFA Rules not processed for user `1d3ce5d199cd4f7b8065fc7bb2a10932`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:34.936161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b308552f-b83f-4f24-b9c3-522b01ff1aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:35.039587 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3341/6671] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:35.044383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:35.147220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604100c-9cea-42e3-9ae9-453453de95aa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:35.156580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:35.157215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:35.157503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:35.157784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:35.158477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:35.160496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:35.230310 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3331/6672] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:35 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:35.234795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308552f-b83f-4f24-b9c3-522b01ff1aeb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:35.256084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308552f-b83f-4f24-b9c3-522b01ff1aeb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:35.352014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:35.352779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:35.353450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:35.353957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:35.355191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:35.357612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:35.391617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3342/6673] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:35 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:35.396746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:35.416589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a6ff0-0331-4a38-a22d-4e71770aacde admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:35.422727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:35.423416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:35.423703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:35.423981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:35.425315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:35.426779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:35.444791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3332/6674] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:35.449819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:35.463069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7a26-f934-4bed-9235-da819278b730 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:35.471219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:35.472052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:35.472381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:35.472694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] PATH_INFO: `/v3/projects/887f3c4f88de4498969fce46cfb14f73/users/98c9d144111444a0bf63ea18ca157ce2/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:35.521435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887f3c4f88de4498969fce46cfb14f73, user_id=98c9d144111444a0bf63ea18ca157ce2, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:35.524783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:35.564766 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3343/6675] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:35 2022] PUT /identity/v3/projects/887f3c4f88de4498969fce46cfb14f73/users/98c9d144111444a0bf63ea18ca157ce2/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:35.568794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:35.614650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79984ac-5058-4d48-afa0-11d512247e73 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:35.736915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46102bac-8415-41bc-8578-afef10c2a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:35.737902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46102bac-8415-41bc-8578-afef10c2a222 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:35.738337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46102bac-8415-41bc-8578-afef10c2a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:35.738766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46102bac-8415-41bc-8578-afef10c2a222 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:35.739619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46102bac-8415-41bc-8578-afef10c2a222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:35.741659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46102bac-8415-41bc-8578-afef10c2a222 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:35.771810 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3333/6676] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:35.771985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:35.790086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3495f-9f28-491c-81a9-951ec56fe6f0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:35.799248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:35.799933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:35.800200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:35.800474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] PATH_INFO: `/v3/projects/887f3c4f88de4498969fce46cfb14f73/users/98c9d144111444a0bf63ea18ca157ce2/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:35.803295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=887f3c4f88de4498969fce46cfb14f73, user_id=98c9d144111444a0bf63ea18ca157ce2, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:35.807136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:35.826234 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3344/6677] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:35 2022] PUT /identity/v3/projects/887f3c4f88de4498969fce46cfb14f73/users/98c9d144111444a0bf63ea18ca157ce2/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:35.832908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e57e-6faa-449b-b82b-3bc4bf26d923 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:35.833438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e57e-6faa-449b-b82b-3bc4bf26d923 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:35.834872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e57e-6faa-449b-b82b-3bc4bf26d923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:35.873192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:35.887340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d2b0e57e-6faa-449b-b82b-3bc4bf26d923 None None] MFA Rules not processed for user `98c9d144111444a0bf63ea18ca157ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:35.897544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d2b0e57e-6faa-449b-b82b-3bc4bf26d923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:35.989060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3334/6678] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:35.993715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.008891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850ddfe-2ae0-414b-825c-f97fcac7ea6c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.015247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.016505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.016986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.018021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.019550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.022191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:36.080125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bddf90-964c-4bb1-a4aa-6b5c1b3f4089 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.080416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bddf90-964c-4bb1-a4aa-6b5c1b3f4089 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.080794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bddf90-964c-4bb1-a4aa-6b5c1b3f4089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.104517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3345/6679] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:35 2022] POST /identity/v3/projects => generated 359 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:36.109474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-72bddf90-964c-4bb1-a4aa-6b5c1b3f4089 None None] MFA Rules not processed for user `d83b6ceb599746b394e379d52683b70c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:36.111772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-72bddf90-964c-4bb1-a4aa-6b5c1b3f4089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:36.116210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.213714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:36.272557 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3335/6680] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:36.302710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcd240-f929-4295-96eb-27713d3e8157 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.314103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.314781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.315067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.315346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1072678849` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.316588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=cf570db2918b4f948c545a4c84b1513b, value=tempest-tag-1072678849)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.318890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:36.476187 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1072678849. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:15:36.476187 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:15:36.476187 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:15:36.495785 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3346/6681] 10.222.0.33 () {60 vars in 1333 bytes} [Sat Sep 10 21:15:36 2022] PUT /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1072678849 => generated 0 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 291 bytes (1 switches on core 0) Sep 10 21:15:36.497244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bddf90-964c-4bb1-a4aa-6b5c1b3f4089 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.518146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bddf90-964c-4bb1-a4aa-6b5c1b3f4089 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.529448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.530482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.531058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.532217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1072678849` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.550174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=cf570db2918b4f948c545a4c84b1513b, value=tempest-tag-1072678849)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.553371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:36.570507 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3336/6682] 10.222.0.33 () {58 vars in 1314 bytes} [Sat Sep 10 21:15:36 2022] GET /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1072678849 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:36.575358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.590771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ecab5-9da9-4ed4-af19-ec4e94de4af8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.597287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.598037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.598324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.598602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.600382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=cf570db2918b4f948c545a4c84b1513b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.603855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:36.677448 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3347/6683] 10.222.0.33 () {60 vars in 1265 bytes} [Sat Sep 10 21:15:36 2022] PUT /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags => generated 142 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:36.681500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.699633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35162ac-b8b8-4542-98b9-b25f857100fd tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.705811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.706660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.707664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.707956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.723626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=cf570db2918b4f948c545a4c84b1513b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.725600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:36.746862 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3337/6684] 10.222.0.33 () {58 vars in 1245 bytes} [Sat Sep 10 21:15:36 2022] GET /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags => generated 142 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:36.750886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.764251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d75873-336e-443f-9012-07728e14d9d8 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.770957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.771625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.771915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.772429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1597631178` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.773621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=cf570db2918b4f948c545a4c84b1513b, value=tempest-tag-1597631178)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.776584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:36.829403 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1597631178. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:15:36.829403 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:15:36.829403 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:15:36.836509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3348/6685] 10.222.0.33 () {58 vars in 1317 bytes} [Sat Sep 10 21:15:36 2022] DELETE /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1597631178 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:36.841035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.854452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f93e2-119a-4a3f-893e-e6f9249ebae4 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.863783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.864807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.865321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.865808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1597631178` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.880462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=cf570db2918b4f948c545a4c84b1513b, value=tempest-tag-1597631178)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.883165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Could not find project tag: tempest-tag-1597631178.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1597631178. Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Sep 10 21:15:36.911968 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Sep 10 21:15:36.913093 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:36.913093 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:36.913093 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Sep 10 21:15:36.913093 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Sep 10 21:15:36.913093 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1597631178. Sep 10 21:15:36.913093 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:36.915452 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3338/6686] 10.222.0.33 () {58 vars in 1314 bytes} [Sat Sep 10 21:15:36 2022] GET /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags/tempest-tag-1597631178 => generated 107 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:36.919409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:36.931149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fe23f-cf50-4975-88e0-3d81f74e5898 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:36.937062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:36.937721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:36.938062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:36.938341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:36.939510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=cf570db2918b4f948c545a4c84b1513b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:36.941813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:37.017886 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3349/6687] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:15:36 2022] DELETE /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:37.019660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:37.033492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b7fcb-27d0-47f8-b9ef-337ef92e89e3 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:37.043327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:37.044236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:37.044521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:37.044845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:37.064532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=cf570db2918b4f948c545a4c84b1513b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:37.067422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:37.089773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:37.097015 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3339/6688] 10.222.0.33 () {58 vars in 1245 bytes} [Sat Sep 10 21:15:37 2022] GET /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b/tags => generated 66 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 10 21:15:37.102660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b44eb-2b41-4ac3-8dec-d6fb36b3b301 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:37.109250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c9d144111444a0bf63ea18ca157ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '887f3c4f88de4498969fce46cfb14f73', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:37.110068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:37.110506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:37.110910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] PATH_INFO: `/v3/projects/cf570db2918b4f948c545a4c84b1513b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:37.112323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cf570db2918b4f948c545a4c84b1513b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:37.114683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:37.161151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:37.191447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf570db2918b4f948c545a4c84b1513b', 'request_id': 'req-83b9fe29-b949-4b72-9252-e7b60177dc42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.201508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf570db2918b4f948c545a4c84b1513b', 'request_id': 'req-83b9fe29-b949-4b72-9252-e7b60177dc42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.213482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf570db2918b4f948c545a4c84b1513b', 'request_id': 'req-83b9fe29-b949-4b72-9252-e7b60177dc42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.230675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] The token cache is being invalidate because project cf570db2918b4f948c545a4c84b1513b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:37.231443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.232622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-83b9fe29-b949-4b72-9252-e7b60177dc42 tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.240709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:37.241775 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3350/6689] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:37 2022] DELETE /identity/v3/projects/cf570db2918b4f948c545a4c84b1513b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:37.243584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:37.271628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:37.375506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a55cd-aa5d-4df4-854a-c451769150ae tempest-IdentityV3ProjectTagsTest-1321615153 tempest-IdentityV3ProjectTagsTest-1321615153-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:37.519046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:37.523386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:37.529197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:37.529401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] PATH_INFO: `/v3/users/1d3ce5d199cd4f7b8065fc7bb2a10932` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:37.535368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d3ce5d199cd4f7b8065fc7bb2a10932)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:37.556055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:37.709191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d3ce5d199cd4f7b8065fc7bb2a10932'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.740290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d3ce5d199cd4f7b8065fc7bb2a10932'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.770130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d3ce5d199cd4f7b8065fc7bb2a10932'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.800301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d3ce5d199cd4f7b8065fc7bb2a10932'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.800301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d3ce5d199cd4f7b8065fc7bb2a10932'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:37.810730 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3340/6690] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:37 2022] DELETE /identity/v3/users/1d3ce5d199cd4f7b8065fc7bb2a10932 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:37.814698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:37.815162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:37.815258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:37.848163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:37.851905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:38.008116 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3351/6691] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:38.041928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:38.043636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:38.144092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:38.187150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a2c2-3510-4e6b-b323-72c496010037 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:38.195291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842008a-1274-4560-842a-40a40c7c057d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:38.195925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842008a-1274-4560-842a-40a40c7c057d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:38.196212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842008a-1274-4560-842a-40a40c7c057d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:38.196490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842008a-1274-4560-842a-40a40c7c057d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:38.216915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842008a-1274-4560-842a-40a40c7c057d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:38.218534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842008a-1274-4560-842a-40a40c7c057d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:38.241562 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3341/6692] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:38.757367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:38.760134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:38.776155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6b6c3-beef-4566-9c85-5fc27b1bf7d6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:38.783749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:38.784516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:38.791630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:38.792013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] PATH_INFO: `/v3/projects/c7df49e781fa4479945f9744b143e145` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:38.795091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7df49e781fa4479945f9744b143e145)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:38.798611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:38.904142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f132f9-0c30-4ecd-bf96-3a3f322af5a5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:38.904436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f132f9-0c30-4ecd-bf96-3a3f322af5a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:38.904722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f132f9-0c30-4ecd-bf96-3a3f322af5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:38.969543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-98f132f9-0c30-4ecd-bf96-3a3f322af5a5 None None] MFA Rules not processed for user `93648515d8fb42a7aaf61e245db0e07d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:38.972197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-98f132f9-0c30-4ecd-bf96-3a3f322af5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:38.992807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7df49e781fa4479945f9744b143e145', 'request_id': 'req-89c95348-c8c1-4000-9da6-785c32c9af16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.012426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7df49e781fa4479945f9744b143e145', 'request_id': 'req-89c95348-c8c1-4000-9da6-785c32c9af16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.052084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7df49e781fa4479945f9744b143e145', 'request_id': 'req-89c95348-c8c1-4000-9da6-785c32c9af16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.069917 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3342/6693] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:39.116664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] The token cache is being invalidate because project c7df49e781fa4479945f9744b143e145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:39.117083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.117852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-89c95348-c8c1-4000-9da6-785c32c9af16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.130730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3352/6694] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:38 2022] DELETE /identity/v3/projects/c7df49e781fa4479945f9744b143e145 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:39.132006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f132f9-0c30-4ecd-bf96-3a3f322af5a5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:39.314436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f132f9-0c30-4ecd-bf96-3a3f322af5a5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:39.319698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:39.320509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:39.320882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:39.321230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] PATH_INFO: `/v3/users/98c9d144111444a0bf63ea18ca157ce2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:39.322942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c9d144111444a0bf63ea18ca157ce2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:39.325354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:39.416108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c9d144111444a0bf63ea18ca157ce2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.456720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c9d144111444a0bf63ea18ca157ce2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.474156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c9d144111444a0bf63ea18ca157ce2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.485590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c9d144111444a0bf63ea18ca157ce2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.486590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c9d144111444a0bf63ea18ca157ce2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:39.513489 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3343/6695] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:39 2022] DELETE /identity/v3/users/98c9d144111444a0bf63ea18ca157ce2 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:39.732447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41ad5a-7056-40ab-9db1-1e845b252f20 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:39.732903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41ad5a-7056-40ab-9db1-1e845b252f20 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:39.733287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41ad5a-7056-40ab-9db1-1e845b252f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:39.765687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1a41ad5a-7056-40ab-9db1-1e845b252f20 None None] MFA Rules not processed for user `2c96909ab4eb449d81769b4c1a52ec88`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:39.767184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a41ad5a-7056-40ab-9db1-1e845b252f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:39.874826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:39.994011 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3353/6696] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:40.724302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:40.726710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:40.739567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:40.855155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c10e05-b040-4aa7-afcf-4e71e245e90d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:40.862649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:40.863769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:40.864273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:40.864776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] PATH_INFO: `/v3/projects/887f3c4f88de4498969fce46cfb14f73` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:40.866391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887f3c4f88de4498969fce46cfb14f73)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:40.869460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:40.956046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887f3c4f88de4498969fce46cfb14f73', 'request_id': 'req-ee1becb4-ca99-4190-8807-17ec6038e5f1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:40.969651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887f3c4f88de4498969fce46cfb14f73', 'request_id': 'req-ee1becb4-ca99-4190-8807-17ec6038e5f1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:40.991242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887f3c4f88de4498969fce46cfb14f73', 'request_id': 'req-ee1becb4-ca99-4190-8807-17ec6038e5f1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:41.014961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] The token cache is being invalidate because project 887f3c4f88de4498969fce46cfb14f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:41.015607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:41.022619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ee1becb4-ca99-4190-8807-17ec6038e5f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:41.027849 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3344/6697] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:40 2022] DELETE /identity/v3/projects/887f3c4f88de4498969fce46cfb14f73 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:41.033475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e0641-a4fc-41d9-93a9-53656559300b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.034040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e0641-a4fc-41d9-93a9-53656559300b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.034282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e0641-a4fc-41d9-93a9-53656559300b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.067694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2a1e0641-a4fc-41d9-93a9-53656559300b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:41.071024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a1e0641-a4fc-41d9-93a9-53656559300b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:41.173900 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3354/6698] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:41.178127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.178565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.178711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.205387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:41.207166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:41.223804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e0641-a4fc-41d9-93a9-53656559300b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:41.232295 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3345/6699] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:41.237243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e0641-a4fc-41d9-93a9-53656559300b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:41.243278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:41.246549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:41.256919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:41.258481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.259837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.260150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.262859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:41.263511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:41.268087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:41.302022 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3355/6700] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:41.309482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:41.329408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2efa1-7964-47fa-a810-5066536e9826 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:41.347065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:41.347704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.347991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.348427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.349071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:41.352779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:41.395839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615997b1-092a-44b9-bd50-8c8de4c33990 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:41.402837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:41.403384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.403742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.404083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.406440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:41.416416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3356/6701] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:41 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:41.418091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:41.425569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:41.428123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:41.442829 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3346/6702] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:15:41.512652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4b753-bf0c-4f16-9d32-3ae11d83e033 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:41.518283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:41.519200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.519494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.519779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.520414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:41.522397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:41.553870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3357/6703] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:41 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:41.557762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:41.576708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610263c-b69f-4445-aa2d-8e26087f9cc0 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:41.587552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:41.588193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.588685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.588970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.591754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:41.593566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:41.611047 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3347/6704] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:41.612248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:41.624757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430fba-c305-46ed-8a8e-052449c95fde admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:41.632314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961ed1d-86bf-4562-a82c-8dada05044a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:41.632642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961ed1d-86bf-4562-a82c-8dada05044a7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.633313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961ed1d-86bf-4562-a82c-8dada05044a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.633313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961ed1d-86bf-4562-a82c-8dada05044a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.634113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961ed1d-86bf-4562-a82c-8dada05044a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:41.636508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961ed1d-86bf-4562-a82c-8dada05044a7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:41.649876 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3358/6705] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:41.653957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:41.666986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b780f45-45ea-4069-be23-06039c663bff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:41.673352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:41.673868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.674280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.675103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] PATH_INFO: `/v3/projects/4e45918817c54d74828ee9186ead2e9c/users/99f93c6b6f7b42fd9b7072d64d1b3b66/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.698468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e45918817c54d74828ee9186ead2e9c, user_id=99f93c6b6f7b42fd9b7072d64d1b3b66, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:41.701250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:41.704763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:41.729264 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3348/6706] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:41 2022] PUT /identity/v3/projects/4e45918817c54d74828ee9186ead2e9c/users/99f93c6b6f7b42fd9b7072d64d1b3b66/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:41.739342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591694cc-f1dc-4997-9d1c-09f5188fab16 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:41.740046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591694cc-f1dc-4997-9d1c-09f5188fab16 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:41.740521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591694cc-f1dc-4997-9d1c-09f5188fab16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:41.804716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-591694cc-f1dc-4997-9d1c-09f5188fab16 None None] MFA Rules not processed for user `99f93c6b6f7b42fd9b7072d64d1b3b66`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:41.806070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-591694cc-f1dc-4997-9d1c-09f5188fab16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:41.830869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:41.987468 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3359/6707] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:41.988780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:42.087038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfc900-93dc-425d-8ad2-0eefecaa4133 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:42.095212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:42.095953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:42.096885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:42.097264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:42.098294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:42.100349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:42.166576 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3349/6708] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:41 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:42.169665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591694cc-f1dc-4997-9d1c-09f5188fab16 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:42.185755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591694cc-f1dc-4997-9d1c-09f5188fab16 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:42.314344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:42.316583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:42.317039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:42.317379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:42.318162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:42.324154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:42.379389 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3360/6709] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:42 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:42.383643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:42.402454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0750d-c800-487e-969b-37d9147564a6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:42.413347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:42.414243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:42.414647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:42.416060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:42.418624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:42.423044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:42.452159 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3350/6710] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:15:42.457786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:42.472088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9069732-587d-443c-9f9a-70e16854cb4a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:42.485962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:42.488197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:42.489396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:42.489723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] PATH_INFO: `/v3/projects/a4574ff1d55148f5a0e407721efe09a2/users/4a014358bbe94beca8f14732b665b521/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:42.528149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4574ff1d55148f5a0e407721efe09a2, user_id=4a014358bbe94beca8f14732b665b521, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:42.531468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:42.554776 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3361/6711] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:42 2022] PUT /identity/v3/projects/a4574ff1d55148f5a0e407721efe09a2/users/4a014358bbe94beca8f14732b665b521/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:42.559673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:42.572719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cfb31-eefc-4942-906e-8ce2405185b3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:42.699023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf321a-e115-4c29-8301-48b1d1112aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:42.701241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf321a-e115-4c29-8301-48b1d1112aee admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:42.701662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf321a-e115-4c29-8301-48b1d1112aee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:42.702105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf321a-e115-4c29-8301-48b1d1112aee admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:42.704257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf321a-e115-4c29-8301-48b1d1112aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:42.708370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf321a-e115-4c29-8301-48b1d1112aee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:42.749450 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3351/6712] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:42.750999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:42.775866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3225a0-68c0-4f27-aa6f-86b5ba9e32eb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:42.818384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:42.818384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:42.818384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:42.818384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] PATH_INFO: `/v3/projects/a4574ff1d55148f5a0e407721efe09a2/users/4a014358bbe94beca8f14732b665b521/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:42.821877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4574ff1d55148f5a0e407721efe09a2, user_id=4a014358bbe94beca8f14732b665b521, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:42.825122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:42.863969 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3362/6713] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:42 2022] PUT /identity/v3/projects/a4574ff1d55148f5a0e407721efe09a2/users/4a014358bbe94beca8f14732b665b521/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:42.876393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c59a2c-3a73-4e35-91de-f44daaca16db None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:42.876677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c59a2c-3a73-4e35-91de-f44daaca16db None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:42.876958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c59a2c-3a73-4e35-91de-f44daaca16db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:42.941722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-65c59a2c-3a73-4e35-91de-f44daaca16db None None] MFA Rules not processed for user `4a014358bbe94beca8f14732b665b521`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:42.948342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65c59a2c-3a73-4e35-91de-f44daaca16db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:43.084740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:43.134134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:43.136304 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3352/6714] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:43.191432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80d237-3246-40d2-9106-ae2f70c2bcb9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:43.202972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a014358bbe94beca8f14732b665b521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4574ff1d55148f5a0e407721efe09a2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:43.203610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:43.203899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:43.204728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:43.207268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:43.209722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:43.256414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c59a2c-3a73-4e35-91de-f44daaca16db None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:43.259214 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3363/6715] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:43 2022] POST /identity/v3/projects => generated 358 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:43.381112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c59a2c-3a73-4e35-91de-f44daaca16db None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:43.447133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f93c6b6f7b42fd9b7072d64d1b3b66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e45918817c54d74828ee9186ead2e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:43.447930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:43.448341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:43.448697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] PATH_INFO: `/v3/projects/b16cee97fe35490381280a5962279e35` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:43.450023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] RBAC: Authorizing `identity:delete_project(project_id=b16cee97fe35490381280a5962279e35)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Sep 10 21:15:43.459292 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 10 21:15:43.460187 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:43.460806 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3353/6716] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:43 2022] DELETE /identity/v3/projects/b16cee97fe35490381280a5962279e35 => generated 136 bytes in 205 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:43.465818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:43.555904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:43.622594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a3e7-071d-405b-b09f-9f93fe93beac tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:43.634913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a014358bbe94beca8f14732b665b521', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4574ff1d55148f5a0e407721efe09a2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:43.635986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:43.636470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:43.636898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] PATH_INFO: `/v3/projects/b16cee97fe35490381280a5962279e35` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:43.641431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b16cee97fe35490381280a5962279e35)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:43.647264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:43.766900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16cee97fe35490381280a5962279e35', 'request_id': 'req-3c51531d-0631-434b-810e-af1d9385d682'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:43.784731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16cee97fe35490381280a5962279e35', 'request_id': 'req-3c51531d-0631-434b-810e-af1d9385d682'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:43.799555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16cee97fe35490381280a5962279e35', 'request_id': 'req-3c51531d-0631-434b-810e-af1d9385d682'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:43.832144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] The token cache is being invalidate because project b16cee97fe35490381280a5962279e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:43.832493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:43.833749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c51531d-0631-434b-810e-af1d9385d682 tempest-ProjectsNegativeTestJSON-2091024983 tempest-ProjectsNegativeTestJSON-2091024983-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:43.835767 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3364/6717] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:43 2022] DELETE /identity/v3/projects/b16cee97fe35490381280a5962279e35 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:43.841574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:43.844072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:44.032312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d97b34-6ec2-4761-ba8c-da4f1d41ae4c tempest-ProjectsNegativeTestJSON-1882342387 tempest-ProjectsNegativeTestJSON-1882342387-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:44.132119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:44.152109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:44.152791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:44.153180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:44.153780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] PATH_INFO: `/v3/users/99f93c6b6f7b42fd9b7072d64d1b3b66` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:44.154862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f93c6b6f7b42fd9b7072d64d1b3b66)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:44.156949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:44.206386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f93c6b6f7b42fd9b7072d64d1b3b66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:44.223314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f93c6b6f7b42fd9b7072d64d1b3b66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:44.231551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f93c6b6f7b42fd9b7072d64d1b3b66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:44.241225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f93c6b6f7b42fd9b7072d64d1b3b66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:44.242162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f93c6b6f7b42fd9b7072d64d1b3b66'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:44.252030 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3354/6718] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:43 2022] DELETE /identity/v3/users/99f93c6b6f7b42fd9b7072d64d1b3b66 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:44.256838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:44.257339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:44.257434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:44.278732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:44.280442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:44.366355 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3365/6719] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:44.376491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:44.379142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:44.455933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921e363-0077-4bd6-8351-36f3ea7ac5c2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:44.462187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:44.463420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:44.463705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:44.463986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:44.475458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:44.477103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:44.497092 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3355/6720] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:15:44.864438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:44.866158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:44.896854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed49ad-94fe-417a-bac6-48962b5fdc07 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:44.910690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:44.911848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:44.912404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:44.912972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] PATH_INFO: `/v3/projects/4e45918817c54d74828ee9186ead2e9c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:44.915080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e45918817c54d74828ee9186ead2e9c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:44.918153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:44.998116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e45918817c54d74828ee9186ead2e9c', 'request_id': 'req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.006812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e45918817c54d74828ee9186ead2e9c', 'request_id': 'req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.018855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e45918817c54d74828ee9186ead2e9c', 'request_id': 'req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.022286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:45.038387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] The token cache is being invalidate because project 4e45918817c54d74828ee9186ead2e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:45.042281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.043109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3d0802bd-245c-4bb0-92b6-9d13fcd512d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.044938 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3366/6721] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:44 2022] DELETE /identity/v3/projects/4e45918817c54d74828ee9186ead2e9c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:45.048737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:45.061565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f87715-9152-4a12-9624-9744d6ce3d45 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:45.150738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:45.151537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:45.152025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:45.152377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] PATH_INFO: `/v3/users/4a014358bbe94beca8f14732b665b521` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:45.153650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a014358bbe94beca8f14732b665b521)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:45.155709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:45.184489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:45.215789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a014358bbe94beca8f14732b665b521'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.230301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a014358bbe94beca8f14732b665b521'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.237589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a014358bbe94beca8f14732b665b521'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.245936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a014358bbe94beca8f14732b665b521'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.246616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a014358bbe94beca8f14732b665b521'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.253118 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3356/6722] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:45 2022] DELETE /identity/v3/users/4a014358bbe94beca8f14732b665b521 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:45.541648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628224e2-00fd-42c8-b725-47a8bafa3af5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:45.542130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628224e2-00fd-42c8-b725-47a8bafa3af5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:45.542377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628224e2-00fd-42c8-b725-47a8bafa3af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:45.561034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:45.562635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:45.610611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-628224e2-00fd-42c8-b725-47a8bafa3af5 None None] MFA Rules not processed for user `eac0a8d66da042e096cab5c715f40a27`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:45.612038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-628224e2-00fd-42c8-b725-47a8bafa3af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:45.661839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf4908-ec12-4b63-b366-96393744969e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:45.669509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:45.670308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:45.671188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:45.671554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] PATH_INFO: `/v3/projects/a4574ff1d55148f5a0e407721efe09a2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:45.673172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4574ff1d55148f5a0e407721efe09a2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:45.675136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:45.790041 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3367/6723] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:45.802753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:45.803317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:45.803744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:45.805748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4574ff1d55148f5a0e407721efe09a2', 'request_id': 'req-ccf72069-7120-419c-aa85-77b0fbf91855'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.822069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4574ff1d55148f5a0e407721efe09a2', 'request_id': 'req-ccf72069-7120-419c-aa85-77b0fbf91855'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.837913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4574ff1d55148f5a0e407721efe09a2', 'request_id': 'req-ccf72069-7120-419c-aa85-77b0fbf91855'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.861053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] MFA Rules not processed for user `eac0a8d66da042e096cab5c715f40a27`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:45.862893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] The token cache is being invalidate because project a4574ff1d55148f5a0e407721efe09a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:45.863374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.864777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ccf72069-7120-419c-aa85-77b0fbf91855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:45.864969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:45.867470 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3357/6724] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:45 2022] DELETE /identity/v3/projects/a4574ff1d55148f5a0e407721efe09a2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:45.872540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a67a0e-45b9-4d69-aaa7-d1aa7dab42d2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:45.873037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a67a0e-45b9-4d69-aaa7-d1aa7dab42d2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:45.873498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a67a0e-45b9-4d69-aaa7-d1aa7dab42d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:45.911458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e4a67a0e-45b9-4d69-aaa7-d1aa7dab42d2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:45.912846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e4a67a0e-45b9-4d69-aaa7-d1aa7dab42d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:45.959165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:46.010426 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3368/6725] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:46.023430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:46.024710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:46.088592 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3358/6726] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:46.091647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dc447-029d-46e1-8ef1-9270a2c15517 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.092332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dc447-029d-46e1-8ef1-9270a2c15517 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.092757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dc447-029d-46e1-8ef1-9270a2c15517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.162803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-847dc447-029d-46e1-8ef1-9270a2c15517 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:46.164471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-847dc447-029d-46e1-8ef1-9270a2c15517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:46.177864 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3359/6727] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:46.181615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dc447-029d-46e1-8ef1-9270a2c15517 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:46.202355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dc447-029d-46e1-8ef1-9270a2c15517 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:46.207672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:46.208330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.208611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.209102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.209402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24836763-a3db-41a3-af61-3fc545deadf5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:46.209967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:15:46.210546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:46.213870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:46.225300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:46.225974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.226400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.226683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.229041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:46.235941 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3360/6728] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:15:46.240086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:46.250532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:46.253090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:46.253688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28e3d6-6277-4812-8a80-48edf3cd60f0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:46.260044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:46.261520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.262187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.262568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.263316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:46.266333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:46.267696 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3369/6729] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:15:46.351599 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3361/6730] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:46 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:46.354138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:46.372242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b6148-b207-43b4-999f-c7ebf017609b service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:46.382247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:46.464464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:46.465145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.465430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.465700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.466363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:46.469575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:46.507526 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3370/6731] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:46 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:46.508754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:46.525298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdc9ba-925a-4310-b1de-9761bbd84fd6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:46.533955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:46.534549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.534777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.535052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.535780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:46.538439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:46.558454 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3362/6732] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:46.560910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:46.606995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a012d1-c8a2-4433-a645-afd530455425 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:46.622057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92589c18-71b9-4d59-ad71-a099f2d83ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:46.623750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92589c18-71b9-4d59-ad71-a099f2d83ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.624292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92589c18-71b9-4d59-ad71-a099f2d83ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.624717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92589c18-71b9-4d59-ad71-a099f2d83ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.626303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92589c18-71b9-4d59-ad71-a099f2d83ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:46.627940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92589c18-71b9-4d59-ad71-a099f2d83ee6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:46.642915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3371/6733] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:46.646883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:46.659860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8a585-764f-41f3-9477-fd3b9977dc6f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:46.664737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:46.665744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.665744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.666298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] PATH_INFO: `/v3/projects/c471963f013f4f66b23c29851e719a89/users/49b1b9f8857a40bb8266b9a626e4bcc5/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.683519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c471963f013f4f66b23c29851e719a89, user_id=49b1b9f8857a40bb8266b9a626e4bcc5, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:46.686207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:46.711583 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3363/6734] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:46 2022] PUT /identity/v3/projects/c471963f013f4f66b23c29851e719a89/users/49b1b9f8857a40bb8266b9a626e4bcc5/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:46.714257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dc7b7-5779-4bcd-9a9b-8da9fe7bee71 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:46.714551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dc7b7-5779-4bcd-9a9b-8da9fe7bee71 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:46.714887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dc7b7-5779-4bcd-9a9b-8da9fe7bee71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:46.775109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f28dc7b7-5779-4bcd-9a9b-8da9fe7bee71 None None] MFA Rules not processed for user `49b1b9f8857a40bb8266b9a626e4bcc5`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:46.778776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f28dc7b7-5779-4bcd-9a9b-8da9fe7bee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:46.908782 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3372/6735] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:46.914551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.000835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0a1fd-6de7-492b-8ee2-20a8465988b4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:47.005692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:47.006342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.007572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.007849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.008841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:47.011910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:47.087132 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3364/6736] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:15:46 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:47.091567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28dc7b7-5779-4bcd-9a9b-8da9fe7bee71 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.110153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28dc7b7-5779-4bcd-9a9b-8da9fe7bee71 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:47.191639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:47.193678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.194046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.194333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.195034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:47.198146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:47.228349 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3373/6737] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:47 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:47.229560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.242637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99960ab-3aba-47f2-a21d-a654e6c415b7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:47.250815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:47.251899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.252416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.252839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.253726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:47.255842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:47.274022 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3365/6738] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:47.282048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.305457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc210e-ba9d-430b-be74-0fe4d27042dd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:47.314161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:47.314844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.315128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.315402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] PATH_INFO: `/v3/projects/be2e2b9a7452448f86ac2b665a05d67f/users/2b19eec13d3e4c0083e3399abf4a5acf/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.318977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:47.334271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2e2b9a7452448f86ac2b665a05d67f, user_id=2b19eec13d3e4c0083e3399abf4a5acf, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:47.338642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:47.362677 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3374/6739] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:47 2022] PUT /identity/v3/projects/be2e2b9a7452448f86ac2b665a05d67f/users/2b19eec13d3e4c0083e3399abf4a5acf/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:47.368318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.389481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5a3dd-34d2-4fc2-a9ef-352870eeb923 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:47.429721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:47.481415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85cc4d-7034-4bec-95ec-74dd9f02f841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:47.481990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85cc4d-7034-4bec-95ec-74dd9f02f841 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.482516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85cc4d-7034-4bec-95ec-74dd9f02f841 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.483425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85cc4d-7034-4bec-95ec-74dd9f02f841 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.484264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85cc4d-7034-4bec-95ec-74dd9f02f841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:47.486145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85cc4d-7034-4bec-95ec-74dd9f02f841 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:47.508722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3366/6740] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:47.510400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.528073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bac047-6342-4cb7-9e52-d2c3bac7a12a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:47.536195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:47.537093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.537381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.537713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] PATH_INFO: `/v3/projects/be2e2b9a7452448f86ac2b665a05d67f/users/2b19eec13d3e4c0083e3399abf4a5acf/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.541757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2e2b9a7452448f86ac2b665a05d67f, user_id=2b19eec13d3e4c0083e3399abf4a5acf, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:47.544881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:47.599765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7433-daaa-485b-990a-91d9f5e2584a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.600200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7433-daaa-485b-990a-91d9f5e2584a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.600462 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3375/6741] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:47 2022] PUT /identity/v3/projects/be2e2b9a7452448f86ac2b665a05d67f/users/2b19eec13d3e4c0083e3399abf4a5acf/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:47.600921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7433-daaa-485b-990a-91d9f5e2584a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.640663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a10f7433-daaa-485b-990a-91d9f5e2584a None None] MFA Rules not processed for user `2b19eec13d3e4c0083e3399abf4a5acf`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:47.641798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a10f7433-daaa-485b-990a-91d9f5e2584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:47.763507 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3367/6742] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:47.768285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.783168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d0cc-1891-4d56-a709-a60b0122fa7e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:47.817814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:47.818725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:47.819099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:47.819317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:47.820149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:47.826093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:47.876340 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3376/6743] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:15:47 2022] POST /identity/v3/projects => generated 329 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:47.877516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7433-daaa-485b-990a-91d9f5e2584a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:47.892856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7433-daaa-485b-990a-91d9f5e2584a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:48.050375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:48.051045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.051446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.051744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.052417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:48.054391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:48.107425 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3368/6744] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:47 2022] POST /identity/v3/users => generated 416 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:48.111926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:48.132612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15292ad-1fe7-499c-876a-04dc9c88fc58 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:48.144098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:48.145522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.147591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.147949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.148643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:48.150529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:48.200064 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3377/6745] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:48 2022] POST /identity/v3/roles => generated 243 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:48.201441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:48.215438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eb32a-e667-4560-afce-260217f23d96 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:48.221245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:48.224170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.225448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.226162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.227109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:48.230163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:48.251799 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3369/6746] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:48 2022] POST /identity/v3/roles => generated 246 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:48.255585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:48.270339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6687e-e382-4f56-a42d-b77e5a28683e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:48.276797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:48.277486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.277753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.278060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/9fdea5f47c22436b85bbdc7181ada2b9/users/1a3fdd01f6ed4351a131a150219270a5/roles/5b31d6d4ef9744fb9ca3c18c9010b31d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.327623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=9fdea5f47c22436b85bbdc7181ada2b9, user_id=1a3fdd01f6ed4351a131a150219270a5, role_id=5b31d6d4ef9744fb9ca3c18c9010b31d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:48.331511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:48.348748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3378/6747] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:48 2022] PUT /identity/v3/projects/9fdea5f47c22436b85bbdc7181ada2b9/users/1a3fdd01f6ed4351a131a150219270a5/roles/5b31d6d4ef9744fb9ca3c18c9010b31d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:48.354676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:48.375115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6adde-1d95-4ebc-9883-ae013d502f8c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:48.461098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:48.535655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bd430-c89e-42c3-981c-f17336740873 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:48.536502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bd430-c89e-42c3-981c-f17336740873 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.536803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bd430-c89e-42c3-981c-f17336740873 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.537402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bd430-c89e-42c3-981c-f17336740873 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/9fdea5f47c22436b85bbdc7181ada2b9/users/1a3fdd01f6ed4351a131a150219270a5/roles/8a470cbf93bc4157a87d2916d4f06fb1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.540504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bd430-c89e-42c3-981c-f17336740873 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=9fdea5f47c22436b85bbdc7181ada2b9, user_id=1a3fdd01f6ed4351a131a150219270a5, role_id=8a470cbf93bc4157a87d2916d4f06fb1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:48.544316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bd430-c89e-42c3-981c-f17336740873 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:48.564330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3370/6748] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:48 2022] PUT /identity/v3/projects/9fdea5f47c22436b85bbdc7181ada2b9/users/1a3fdd01f6ed4351a131a150219270a5/roles/8a470cbf93bc4157a87d2916d4f06fb1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:48.568443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:48.581592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db7235-7df5-4570-8d51-ccf29308c1dc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:48.680091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641d1fc-3379-43fd-a644-e263f824c79b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:48.680838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641d1fc-3379-43fd-a644-e263f824c79b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.682038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641d1fc-3379-43fd-a644-e263f824c79b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.682324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641d1fc-3379-43fd-a644-e263f824c79b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.683361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641d1fc-3379-43fd-a644-e263f824c79b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:48.685352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641d1fc-3379-43fd-a644-e263f824c79b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:48.732558 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3379/6749] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:48 2022] GET /identity/v3/users => generated 7774 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:48.737613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731e8a-ed99-4bcc-8061-313d598b9832 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.737922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731e8a-ed99-4bcc-8061-313d598b9832 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.738202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731e8a-ed99-4bcc-8061-313d598b9832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.784216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-06731e8a-ed99-4bcc-8061-313d598b9832 None None] MFA Rules not processed for user `1a3fdd01f6ed4351a131a150219270a5`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:48.785877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-06731e8a-ed99-4bcc-8061-313d598b9832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:48.862156 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3371/6750] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:48.872741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d715f2-16d6-488d-aba8-e0a3575a7df3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.873028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d715f2-16d6-488d-aba8-e0a3575a7df3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.873299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d715f2-16d6-488d-aba8-e0a3575a7df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.893594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-91d715f2-16d6-488d-aba8-e0a3575a7df3 None None] MFA Rules not processed for user `1a3fdd01f6ed4351a131a150219270a5`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:48.894834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-91d715f2-16d6-488d-aba8-e0a3575a7df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:48.902840 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3380/6751] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:48.907265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731e8a-ed99-4bcc-8061-313d598b9832 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:48.917001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731e8a-ed99-4bcc-8061-313d598b9832 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:48.921168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a3fdd01f6ed4351a131a150219270a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fdea5f47c22436b85bbdc7181ada2b9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-326342232', 'tempest-DelegatedRole-488333726'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:48.921775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:48.922088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:48.922363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:48.923029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] RBAC: Authorizing `identity:create_trust()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:48.924176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:49.013881 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3372/6752] 10.222.0.33 () {60 vars in 1174 bytes} [Sat Sep 10 21:15:48 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:49.018116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d715f2-16d6-488d-aba8-e0a3575a7df3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:49.031259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d715f2-16d6-488d-aba8-e0a3575a7df3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:49.035887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:49.036500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:49.036786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:49.037081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:49.037697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:49.038750 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:15:49.038750 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:15:49.038750 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:15:49.039637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:49.047206 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3381/6753] 10.222.0.33 () {58 vars in 1155 bytes} [Sat Sep 10 21:15:49 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:15:49.051264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:49.061413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaa33f-890b-4133-b72c-faec5bc664bf tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:49.065841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a3fdd01f6ed4351a131a150219270a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fdea5f47c22436b85bbdc7181ada2b9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-326342232', 'tempest-DelegatedRole-488333726'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:49.066480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:49.066766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:49.067045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] PATH_INFO: `/v3/OS-TRUST/trusts/89d7908371a64ff5977979945f03c557` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:49.076113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] RBAC: Authorizing `identity:delete_trust(trust_id=89d7908371a64ff5977979945f03c557)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:49.077580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:49.099820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '89d7908371a64ff5977979945f03c557', 'request_id': 'req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.100502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '89d7908371a64ff5977979945f03c557', 'request_id': 'req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.102372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:49.110537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '89d7908371a64ff5977979945f03c557', 'request_id': 'req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.118097 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3373/6754] 10.222.0.33 () {58 vars in 1254 bytes} [Sat Sep 10 21:15:49 2022] DELETE /identity/v3/OS-TRUST/trusts/89d7908371a64ff5977979945f03c557 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:49.123094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:49.125934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:49.140464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763954c-808c-47ee-9a0b-0ac358e62a9d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:49.146415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a3fdd01f6ed4351a131a150219270a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fdea5f47c22436b85bbdc7181ada2b9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-326342232', 'tempest-DelegatedRole-488333726'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:49.147181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:49.147535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:49.148037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] PATH_INFO: `/v3/OS-TRUST/trusts/89d7908371a64ff5977979945f03c557` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Could not find trust: 89d7908371a64ff5977979945f03c557.: keystone.exception.TrustNotFound: Could not find trust: 89d7908371a64ff5977979945f03c557. Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 10 21:15:49.202915 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 89d7908371a64ff5977979945f03c557. Sep 10 21:15:49.204072 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:15:49.204884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3382/6755] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:49 2022] GET /identity/v3/OS-TRUST/trusts/89d7908371a64ff5977979945f03c557 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:49.208861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:49.213972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:49.246182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a8c69-bf13-4e6d-9e75-a92fd9e47da2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:49.251598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:49.252377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:49.252673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:49.253083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/8a470cbf93bc4157a87d2916d4f06fb1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:49.259588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8a470cbf93bc4157a87d2916d4f06fb1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:49.261807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:49.308395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role 8a470cbf93bc4157a87d2916d4f06fb1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:49.309040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.309969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.312068 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3374/6756] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:49 2022] DELETE /identity/v3/roles/8a470cbf93bc4157a87d2916d4f06fb1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:49.316504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:49.318465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:49.335656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9977be2-e06c-4098-a223-149b8cc051a2 tempest-TrustsV3TestJSON-1151154674 tempest-user-1482182346] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:49.382831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:49.402066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:49.402722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:49.403013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:49.403298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/5b31d6d4ef9744fb9ca3c18c9010b31d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:49.404747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5b31d6d4ef9744fb9ca3c18c9010b31d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:49.406414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:49.464530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role 5b31d6d4ef9744fb9ca3c18c9010b31d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:49.464998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.465756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.469132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3383/6757] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:49 2022] DELETE /identity/v3/roles/5b31d6d4ef9744fb9ca3c18c9010b31d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:49.473105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:49.488872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e05c9-889a-4576-ac7a-2bc77add084e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:49.621602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:49.622823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:49.623328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:49.623834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users/1a3fdd01f6ed4351a131a150219270a5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:49.627811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1a3fdd01f6ed4351a131a150219270a5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:49.630124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:49.700272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3fdd01f6ed4351a131a150219270a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.718559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3fdd01f6ed4351a131a150219270a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.726424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3fdd01f6ed4351a131a150219270a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.740420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3fdd01f6ed4351a131a150219270a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.741112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3fdd01f6ed4351a131a150219270a5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:49.752546 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3375/6758] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:49 2022] DELETE /identity/v3/users/1a3fdd01f6ed4351a131a150219270a5 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:49.753855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:49.755996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:49.771513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85afc2-27ca-4a1b-8586-5d75c03b2654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:49.899812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:49.900914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:49.902955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:49.902955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/9fdea5f47c22436b85bbdc7181ada2b9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:49.907856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9fdea5f47c22436b85bbdc7181ada2b9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:49.910041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.003344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fdea5f47c22436b85bbdc7181ada2b9', 'request_id': 'req-e43df504-13b1-40f1-8a0c-b907dc395984'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:50.019850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fdea5f47c22436b85bbdc7181ada2b9', 'request_id': 'req-e43df504-13b1-40f1-8a0c-b907dc395984'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:50.051813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fdea5f47c22436b85bbdc7181ada2b9', 'request_id': 'req-e43df504-13b1-40f1-8a0c-b907dc395984'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:50.066577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] The token cache is being invalidate because project 9fdea5f47c22436b85bbdc7181ada2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:50.066872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:50.067558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:50.069736 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3384/6759] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:49 2022] DELETE /identity/v3/projects/9fdea5f47c22436b85bbdc7181ada2b9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:50.075439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:50.077011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:50.100605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb32193-f5e8-444a-9d64-0ec579d1c5b6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:50.148467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=2,keystone:DELETE=14,keystone:SELECT=84 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:50.186622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:50.187065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.187543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.187832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.188581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:50.190570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.226319 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3376/6760] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:15:50 2022] POST /identity/v3/projects => generated 329 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:50.231069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:50.232582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:50.250134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df504-13b1-40f1-8a0c-b907dc395984 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:50.300511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:50.333074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:50.333695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.334037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.334321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.334974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:50.336945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.367035 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3385/6761] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:50 2022] POST /identity/v3/users => generated 416 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:50.368187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:50.381118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c388b7a-c252-4d9e-93e3-c3e4138692f4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:50.389185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:50.389901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.390480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.390768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.391443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:50.393094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.435106 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3377/6762] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:50 2022] POST /identity/v3/roles => generated 244 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:50.444017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:50.460958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98573bfe-ee12-426e-a3de-dfd967a88702 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:50.469903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:50.470758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.471432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.471751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.472736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:50.475133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.497403 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3386/6763] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:50 2022] POST /identity/v3/roles => generated 247 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:50.498859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:50.513172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701575d2-f71e-406c-8fa2-506caae5ad3f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:50.518231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:50.518902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.519194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.519477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/dbfffdf0ebe74050973d3bb3da13f7cc/users/589b4215c1a8463bb99cc29ac044c502/roles/6599c0fcf1c04c1bb343a32821bbbdcf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.535647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=dbfffdf0ebe74050973d3bb3da13f7cc, user_id=589b4215c1a8463bb99cc29ac044c502, role_id=6599c0fcf1c04c1bb343a32821bbbdcf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:50.538520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.563837 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3378/6764] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:50 2022] PUT /identity/v3/projects/dbfffdf0ebe74050973d3bb3da13f7cc/users/589b4215c1a8463bb99cc29ac044c502/roles/6599c0fcf1c04c1bb343a32821bbbdcf => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:50.568833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:50.613149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f70ef-3558-437a-8527-3a07e5ee835c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:50.701117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec325b-edaa-4647-96bc-4d4aa66622c8 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:50.701888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec325b-edaa-4647-96bc-4d4aa66622c8 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.702363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec325b-edaa-4647-96bc-4d4aa66622c8 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.702363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec325b-edaa-4647-96bc-4d4aa66622c8 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/dbfffdf0ebe74050973d3bb3da13f7cc/users/589b4215c1a8463bb99cc29ac044c502/roles/d7bff64c019f4db08f694801a20acc01` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.704711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec325b-edaa-4647-96bc-4d4aa66622c8 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=dbfffdf0ebe74050973d3bb3da13f7cc, user_id=589b4215c1a8463bb99cc29ac044c502, role_id=d7bff64c019f4db08f694801a20acc01)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:50.708186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec325b-edaa-4647-96bc-4d4aa66622c8 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.722817 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3387/6765] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:50 2022] PUT /identity/v3/projects/dbfffdf0ebe74050973d3bb3da13f7cc/users/589b4215c1a8463bb99cc29ac044c502/roles/d7bff64c019f4db08f694801a20acc01 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:50.726921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:50.740216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffeb00-4400-469f-82ed-ae740b4bf6a9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:50.808880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5159a-76aa-4f3e-9f43-df7ceeb78b20 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:50.809124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5159a-76aa-4f3e-9f43-df7ceeb78b20 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.809520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5159a-76aa-4f3e-9f43-df7ceeb78b20 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.809977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5159a-76aa-4f3e-9f43-df7ceeb78b20 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.812860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5159a-76aa-4f3e-9f43-df7ceeb78b20 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:50.816136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5159a-76aa-4f3e-9f43-df7ceeb78b20 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:50.861804 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3379/6766] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:50 2022] GET /identity/v3/users => generated 7774 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:50.863288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74402c08-6c7b-4a35-849f-433504dd5bff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:50.863760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74402c08-6c7b-4a35-849f-433504dd5bff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:50.864136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74402c08-6c7b-4a35-849f-433504dd5bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:50.910201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-74402c08-6c7b-4a35-849f-433504dd5bff None None] MFA Rules not processed for user `589b4215c1a8463bb99cc29ac044c502`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:50.911930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74402c08-6c7b-4a35-849f-433504dd5bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:51.001678 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3388/6767] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:51.012180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db85e8e-6cb8-49c4-9607-647bc40067ee None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.012472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db85e8e-6cb8-49c4-9607-647bc40067ee None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.012749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db85e8e-6cb8-49c4-9607-647bc40067ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.037428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3db85e8e-6cb8-49c4-9607-647bc40067ee None None] MFA Rules not processed for user `589b4215c1a8463bb99cc29ac044c502`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:51.038743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3db85e8e-6cb8-49c4-9607-647bc40067ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:51.047142 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3380/6768] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:51.051028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74402c08-6c7b-4a35-849f-433504dd5bff None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:51.060978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74402c08-6c7b-4a35-849f-433504dd5bff None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:51.065136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589b4215c1a8463bb99cc29ac044c502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbfffdf0ebe74050973d3bb3da13f7cc', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2088608919', 'tempest-DelegatedRole-1521293663'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:51.065763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.066073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.066348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.067035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: Authorizing `identity:create_trust()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:51.068230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:51.101637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:51.164689 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3389/6769] 10.222.0.33 () {60 vars in 1174 bytes} [Sat Sep 10 21:15:51 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:51.169566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db85e8e-6cb8-49c4-9607-647bc40067ee None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:51.179508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db85e8e-6cb8-49c4-9607-647bc40067ee None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:51.185367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589b4215c1a8463bb99cc29ac044c502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbfffdf0ebe74050973d3bb3da13f7cc', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2088608919', 'tempest-DelegatedRole-1521293663'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:51.186045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.186353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.186631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.187310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:51.188695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:51.195185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3381/6770] 10.222.0.33 () {58 vars in 1252 bytes} [Sat Sep 10 21:15:51 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=589b4215c1a8463bb99cc29ac044c502 => generated 611 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:51.199917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:51.217491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a650-13ae-4d36-b06a-9ab3d352bcba tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:51.222107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589b4215c1a8463bb99cc29ac044c502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbfffdf0ebe74050973d3bb3da13f7cc', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2088608919', 'tempest-DelegatedRole-1521293663'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:51.222765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.223053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.223330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] PATH_INFO: `/v3/OS-TRUST/trusts/e51c63749c65457eaa1cf3ca33d67906` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.255256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: Authorizing `identity:delete_trust(trust_id=e51c63749c65457eaa1cf3ca33d67906)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:51.257134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:51.283504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'e51c63749c65457eaa1cf3ca33d67906', 'request_id': 'req-e66b5107-1be9-403e-ab18-0462e9c512b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.294652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'e51c63749c65457eaa1cf3ca33d67906', 'request_id': 'req-e66b5107-1be9-403e-ab18-0462e9c512b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.295456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'e51c63749c65457eaa1cf3ca33d67906', 'request_id': 'req-e66b5107-1be9-403e-ab18-0462e9c512b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.311683 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3390/6771] 10.222.0.33 () {58 vars in 1254 bytes} [Sat Sep 10 21:15:51 2022] DELETE /identity/v3/OS-TRUST/trusts/e51c63749c65457eaa1cf3ca33d67906 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:51.313235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:51.315397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:51.329909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fd4a0-3954-4ba9-bf47-ad4717a75313 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:51.336825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589b4215c1a8463bb99cc29ac044c502', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dbfffdf0ebe74050973d3bb3da13f7cc', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2088608919', 'tempest-DelegatedRole-1521293663'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:51.337944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.338608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.338972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] PATH_INFO: `/v3/OS-TRUST/trusts/e51c63749c65457eaa1cf3ca33d67906` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Could not find trust: e51c63749c65457eaa1cf3ca33d67906.: keystone.exception.TrustNotFound: Could not find trust: e51c63749c65457eaa1cf3ca33d67906. Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 10 21:15:51.350959 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: e51c63749c65457eaa1cf3ca33d67906. Sep 10 21:15:51.352164 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:51.354460 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3382/6772] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:51 2022] GET /identity/v3/OS-TRUST/trusts/e51c63749c65457eaa1cf3ca33d67906 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:51.359601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:51.362248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:51.389415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b5107-1be9-403e-ab18-0462e9c512b7 tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:51.398705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:51.401153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.401609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.402090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/d7bff64c019f4db08f694801a20acc01` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.404843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d7bff64c019f4db08f694801a20acc01)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:51.407065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:51.491936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role d7bff64c019f4db08f694801a20acc01 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:51.492287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.494621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.502569 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3391/6773] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:51 2022] DELETE /identity/v3/roles/d7bff64c019f4db08f694801a20acc01 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:51.504002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:51.505887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:51.534462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a1349-4982-4126-a4c9-d408ebb5f33e tempest-TrustsV3TestJSON-1290633088 tempest-user-1970995678] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:51.630615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:51.652751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:51.653414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.653706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.654024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/6599c0fcf1c04c1bb343a32821bbbdcf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.658956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6599c0fcf1c04c1bb343a32821bbbdcf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:51.660604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:51.744926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role 6599c0fcf1c04c1bb343a32821bbbdcf has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:51.745407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.747383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.750848 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3383/6774] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:51 2022] DELETE /identity/v3/roles/6599c0fcf1c04c1bb343a32821bbbdcf => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:51.754755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:51.768336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db148d0d-24b7-4aee-85a3-df5e598148c2 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:51.844441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:51.845109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:51.845418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:51.845714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users/589b4215c1a8463bb99cc29ac044c502` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:51.848064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_user(user_id=589b4215c1a8463bb99cc29ac044c502)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:51.850791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:51.936750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:51.954148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589b4215c1a8463bb99cc29ac044c502'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.973867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589b4215c1a8463bb99cc29ac044c502'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.974484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589b4215c1a8463bb99cc29ac044c502'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.993906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589b4215c1a8463bb99cc29ac044c502'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:51.994900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589b4215c1a8463bb99cc29ac044c502'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:52.028348 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3392/6775] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:51 2022] DELETE /identity/v3/users/589b4215c1a8463bb99cc29ac044c502 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:52.030332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.032738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:52.056972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62029cee-2ef0-4e28-9566-b6ebbd80c06b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.155489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.156483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.157106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.157553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/dbfffdf0ebe74050973d3bb3da13f7cc` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.159218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dbfffdf0ebe74050973d3bb3da13f7cc)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.161244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:52.227741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbfffdf0ebe74050973d3bb3da13f7cc', 'request_id': 'req-af6387ad-ec27-4c07-8d5a-096932a5dee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:52.239279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbfffdf0ebe74050973d3bb3da13f7cc', 'request_id': 'req-af6387ad-ec27-4c07-8d5a-096932a5dee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:52.249182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbfffdf0ebe74050973d3bb3da13f7cc', 'request_id': 'req-af6387ad-ec27-4c07-8d5a-096932a5dee6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:52.285351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] The token cache is being invalidate because project dbfffdf0ebe74050973d3bb3da13f7cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:52.285351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:52.285864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:52.291505 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3384/6776] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:52 2022] DELETE /identity/v3/projects/dbfffdf0ebe74050973d3bb3da13f7cc => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:52.308506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.311693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:52.328475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6408b7-8079-4016-8a34-af20e29a1474 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.421201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.422546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.422992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.424687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.426134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.428828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:52.469860 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3393/6777] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:15:52 2022] POST /identity/v3/projects => generated 328 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:52.471155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.472641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:52.488211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6387ad-ec27-4c07-8d5a-096932a5dee6 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.553707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:52.581423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.582213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.582673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.583002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.584965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.589709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:52.619714 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3385/6778] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:52 2022] POST /identity/v3/users => generated 413 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:52.621063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.633569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42c248-02fc-4edd-9b6a-9c80f0f78597 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.642282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.643029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.643389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.643731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.645178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.646945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:52.669934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3394/6779] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:52 2022] POST /identity/v3/roles => generated 244 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:52.673707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.689613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397001f-fdc9-450d-8798-6afabe296654 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.695219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.695869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.696155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.696444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.697677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.699317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:52.721535 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3386/6780] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:52 2022] POST /identity/v3/roles => generated 246 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:52.722642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.734579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18014e6-8efd-456a-b0af-02ab182c6656 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.740119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.740721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.741010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.741289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/673e9140ccd444258798d3b1ed804572/users/777642fc130b47829ac7048bf0c0d89b/roles/fd5f042c570c4b33964ef41d6e53c88b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.757301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=673e9140ccd444258798d3b1ed804572, user_id=777642fc130b47829ac7048bf0c0d89b, role_id=fd5f042c570c4b33964ef41d6e53c88b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.761125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:52.779582 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3395/6781] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:52 2022] PUT /identity/v3/projects/673e9140ccd444258798d3b1ed804572/users/777642fc130b47829ac7048bf0c0d89b/roles/fd5f042c570c4b33964ef41d6e53c88b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:52.783766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.796442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ad2fa-f045-4d99-a413-ea23a3a0fd68 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.863531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff63348-9345-4512-a301-45af2fb2d00e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.864203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff63348-9345-4512-a301-45af2fb2d00e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.864612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff63348-9345-4512-a301-45af2fb2d00e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.864901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff63348-9345-4512-a301-45af2fb2d00e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/673e9140ccd444258798d3b1ed804572/users/777642fc130b47829ac7048bf0c0d89b/roles/d7b2931af46f4773ae11b9c5effc7852` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.867153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff63348-9345-4512-a301-45af2fb2d00e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=673e9140ccd444258798d3b1ed804572, user_id=777642fc130b47829ac7048bf0c0d89b, role_id=d7b2931af46f4773ae11b9c5effc7852)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.869878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff63348-9345-4512-a301-45af2fb2d00e tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:52.887193 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3387/6782] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:52 2022] PUT /identity/v3/projects/673e9140ccd444258798d3b1ed804572/users/777642fc130b47829ac7048bf0c0d89b/roles/d7b2931af46f4773ae11b9c5effc7852 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:52.888383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:52.899502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07ef69-d2aa-4d62-9ac5-b342693aa6dd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:52.967645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6bf45-d13c-4bc7-9cf7-263055b75754 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:52.967645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6bf45-d13c-4bc7-9cf7-263055b75754 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:52.967645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6bf45-d13c-4bc7-9cf7-263055b75754 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:52.968458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6bf45-d13c-4bc7-9cf7-263055b75754 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:52.968458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6bf45-d13c-4bc7-9cf7-263055b75754 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:52.968458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6bf45-d13c-4bc7-9cf7-263055b75754 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.000344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3396/6783] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:52 2022] GET /identity/v3/users => generated 7771 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:53.002256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27432a3-c766-4f0f-9115-c0e9382ad89b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.002678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27432a3-c766-4f0f-9115-c0e9382ad89b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.003186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27432a3-c766-4f0f-9115-c0e9382ad89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.061935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c27432a3-c766-4f0f-9115-c0e9382ad89b None None] MFA Rules not processed for user `777642fc130b47829ac7048bf0c0d89b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:53.063335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c27432a3-c766-4f0f-9115-c0e9382ad89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:53.158607 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3388/6784] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:53.160705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ca63e-9eee-4429-baec-0fc597448c36 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.161053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ca63e-9eee-4429-baec-0fc597448c36 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.161484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ca63e-9eee-4429-baec-0fc597448c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.169844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:53.186596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-301ca63e-9eee-4429-baec-0fc597448c36 None None] MFA Rules not processed for user `777642fc130b47829ac7048bf0c0d89b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:53.188377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-301ca63e-9eee-4429-baec-0fc597448c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:53.199625 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3397/6785] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:53.204160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27432a3-c766-4f0f-9115-c0e9382ad89b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.215770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27432a3-c766-4f0f-9115-c0e9382ad89b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.220935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.221816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.222302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.222867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.223696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:create_trust()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.226081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.267786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:53.315443 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3389/6786] 10.222.0.33 () {60 vars in 1174 bytes} [Sat Sep 10 21:15:53 2022] POST /identity/v3/OS-TRUST/trusts => generated 859 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:53.316995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ca63e-9eee-4429-baec-0fc597448c36 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.328630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ca63e-9eee-4429-baec-0fc597448c36 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.335997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.335997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.335997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.336378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.343774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:get_trust(trust_id=984e2576b90847eabe1053461dba1f18)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.345284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.357791 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3398/6787] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:53 2022] GET /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18 => generated 859 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:53.359937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.371691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00e20-415b-466d-8908-b07efad77483 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.375963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.376653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.376934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.377207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.386894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=984e2576b90847eabe1053461dba1f18)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.388255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.404693 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3390/6788] 10.222.0.33 () {58 vars in 1269 bytes} [Sat Sep 10 21:15:53 2022] GET /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles => generated 371 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:53.405971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.416852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b7fa6-91f9-4edb-81ca-a340aac25489 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.421479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.422234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.422604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.422948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/fd5f042c570c4b33964ef41d6e53c88b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.431966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:get_role_for_trust(trust_id=984e2576b90847eabe1053461dba1f18, role_id=fd5f042c570c4b33964ef41d6e53c88b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.433412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.440920 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3399/6789] 10.222.0.33 () {58 vars in 1368 bytes} [Sat Sep 10 21:15:53 2022] GET /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/fd5f042c570c4b33964ef41d6e53c88b => generated 244 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:53.445025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.454719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1b9a8-5355-462b-a38f-f2b8dfec67ba tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.458895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.459530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.459807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.460078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/fd5f042c570c4b33964ef41d6e53c88b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.467209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:get_role_for_trust(trust_id=984e2576b90847eabe1053461dba1f18, role_id=fd5f042c570c4b33964ef41d6e53c88b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.468647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.476351 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3391/6790] 10.222.0.33 () {58 vars in 1369 bytes} [Sat Sep 10 21:15:53 2022] HEAD /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/fd5f042c570c4b33964ef41d6e53c88b => generated 0 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 10 21:15:53.481542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.491970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeaa02-300f-4340-a042-0cc1be6a54c5 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.497982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.498436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.498831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.499261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/d7b2931af46f4773ae11b9c5effc7852` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.506327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:get_role_for_trust(trust_id=984e2576b90847eabe1053461dba1f18, role_id=d7b2931af46f4773ae11b9c5effc7852)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.507794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Could not find role: d7b2931af46f4773ae11b9c5effc7852.: keystone.exception.RoleNotFound: Could not find role: d7b2931af46f4773ae11b9c5effc7852. Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 21:15:53.520645 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: d7b2931af46f4773ae11b9c5effc7852. Sep 10 21:15:53.521637 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:15:53.522123 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3400/6791] 10.222.0.33 () {58 vars in 1368 bytes} [Sat Sep 10 21:15:53 2022] GET /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/d7b2931af46f4773ae11b9c5effc7852 => generated 110 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:15:53.526924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.536255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dd335-9b0a-43bb-a0e5-6d8f3d76e4f2 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.541105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.542092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.542439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.542780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/d7b2931af46f4773ae11b9c5effc7852` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.548982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:get_role_for_trust(trust_id=984e2576b90847eabe1053461dba1f18, role_id=d7b2931af46f4773ae11b9c5effc7852)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.550619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Could not find role: d7b2931af46f4773ae11b9c5effc7852.: keystone.exception.RoleNotFound: Could not find role: d7b2931af46f4773ae11b9c5effc7852. Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 21:15:53.562163 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: d7b2931af46f4773ae11b9c5effc7852. Sep 10 21:15:53.563702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:53.564830 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3392/6792] 10.222.0.33 () {58 vars in 1369 bytes} [Sat Sep 10 21:15:53 2022] HEAD /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18/roles/d7b2931af46f4773ae11b9c5effc7852 => generated 0 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 10 21:15:53.569226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.581577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6aabe-4dec-471e-a3e1-032a59bbfab7 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.586498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.587188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.587848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.587848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.595855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorizing `identity:delete_trust(trust_id=984e2576b90847eabe1053461dba1f18)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.597917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.622220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '984e2576b90847eabe1053461dba1f18', 'request_id': 'req-39160021-9707-4497-9fdc-cf946339d272'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:53.633116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '984e2576b90847eabe1053461dba1f18', 'request_id': 'req-39160021-9707-4497-9fdc-cf946339d272'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:53.633314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '984e2576b90847eabe1053461dba1f18', 'request_id': 'req-39160021-9707-4497-9fdc-cf946339d272'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:53.641442 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3401/6793] 10.222.0.33 () {58 vars in 1254 bytes} [Sat Sep 10 21:15:53 2022] DELETE /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:53.645629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.648148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:53.673075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f612fe-aeb3-4c6c-a7b5-6688f0d55091 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.677636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '777642fc130b47829ac7048bf0c0d89b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '673e9140ccd444258798d3b1ed804572', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-826221862', 'tempest-DelegatedRole-1760643586'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.678319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.678630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.678870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] PATH_INFO: `/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Could not find trust: 984e2576b90847eabe1053461dba1f18.: keystone.exception.TrustNotFound: Could not find trust: 984e2576b90847eabe1053461dba1f18. Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 10 21:15:53.691076 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 984e2576b90847eabe1053461dba1f18. Sep 10 21:15:53.692245 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:53.693088 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3393/6794] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:53 2022] GET /identity/v3/OS-TRUST/trusts/984e2576b90847eabe1053461dba1f18 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:53.697112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.698990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:53.721441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39160021-9707-4497-9fdc-cf946339d272 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.730506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.732399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.732950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.733410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/d7b2931af46f4773ae11b9c5effc7852` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.735664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d7b2931af46f4773ae11b9c5effc7852)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.739043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.809349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role d7b2931af46f4773ae11b9c5effc7852 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:53.809701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:53.810525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:53.812419 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3402/6795] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:53 2022] DELETE /identity/v3/roles/d7b2931af46f4773ae11b9c5effc7852 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:53.816520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.818322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:53.833476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4927b0-dd32-4cdb-b7ae-030f54fc0313 tempest-TrustsV3TestJSON-292206875 tempest-user-479734875] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:53.894143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:53.894779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:53.895069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:53.895349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/fd5f042c570c4b33964ef41d6e53c88b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:53.896764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=fd5f042c570c4b33964ef41d6e53c88b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:53.898422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:53.939398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role fd5f042c570c4b33964ef41d6e53c88b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:53.939693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:53.940426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:53.942545 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3394/6796] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:53 2022] DELETE /identity/v3/roles/fd5f042c570c4b33964ef41d6e53c88b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:53.946812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:53.957899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189e483-d664-42d6-aa54-36bccd469b69 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.035012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.035649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.035925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.036202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users/777642fc130b47829ac7048bf0c0d89b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.037355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_user(user_id=777642fc130b47829ac7048bf0c0d89b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.039317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.061959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:54.086178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '777642fc130b47829ac7048bf0c0d89b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.093931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '777642fc130b47829ac7048bf0c0d89b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.094620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '777642fc130b47829ac7048bf0c0d89b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.101459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '777642fc130b47829ac7048bf0c0d89b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.102113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '777642fc130b47829ac7048bf0c0d89b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.120938 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3403/6797] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:53 2022] DELETE /identity/v3/users/777642fc130b47829ac7048bf0c0d89b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:54.124979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.126493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:54.146513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f9f98-80f5-4a13-b6d5-ba960a47e864 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.174689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:54.207324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.208278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.208410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.208627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/673e9140ccd444258798d3b1ed804572` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.209766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_project(project_id=673e9140ccd444258798d3b1ed804572)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.211739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.263553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673e9140ccd444258798d3b1ed804572', 'request_id': 'req-973cf0f5-8d9c-456e-957c-0d30f68722fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.269399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673e9140ccd444258798d3b1ed804572', 'request_id': 'req-973cf0f5-8d9c-456e-957c-0d30f68722fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.276649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673e9140ccd444258798d3b1ed804572', 'request_id': 'req-973cf0f5-8d9c-456e-957c-0d30f68722fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.299568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] The token cache is being invalidate because project 673e9140ccd444258798d3b1ed804572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:54.299860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.300499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:54.302027 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3395/6798] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:54 2022] DELETE /identity/v3/projects/673e9140ccd444258798d3b1ed804572 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:54.309415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.311060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:54.327349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2586cf-e74c-4307-81a3-5a7f6177ae15 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.390745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.390745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.390745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.391175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.391518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.393548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.428844 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3404/6799] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:15:54 2022] POST /identity/v3/projects => generated 329 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:54.430087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.431718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:54.444883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cf0f5-8d9c-456e-957c-0d30f68722fb tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.502543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.503146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.503431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.503703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.504339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.506295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.531517 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3396/6800] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:54 2022] POST /identity/v3/users => generated 416 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:54.536228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.552152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec477c-d9aa-4e78-a468-34f1716c4d05 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.558905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.559603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.560048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.560380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.561113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.563388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.584265 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3405/6801] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:54 2022] POST /identity/v3/roles => generated 244 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:54.585401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.596891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e9f4-3cd3-48af-b6ec-4f6011a33b03 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.601682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.602346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.602833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.603278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.604121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.606031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.624492 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3397/6802] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:54 2022] POST /identity/v3/roles => generated 246 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:54.629213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.641916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d758f-ef1b-4ca7-aacc-44b02bb34e32 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.648686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.649368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.650394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.650502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/d3044c8eb8e045b890c7d4b502bf8896/users/ae0a7ec855b94c4683b28cbe291d7a2a/roles/41157af62de34c37b7dfc6b447d1c979` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.669814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d3044c8eb8e045b890c7d4b502bf8896, user_id=ae0a7ec855b94c4683b28cbe291d7a2a, role_id=41157af62de34c37b7dfc6b447d1c979)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.673138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.694007 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3406/6803] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:54 2022] PUT /identity/v3/projects/d3044c8eb8e045b890c7d4b502bf8896/users/ae0a7ec855b94c4683b28cbe291d7a2a/roles/41157af62de34c37b7dfc6b447d1c979 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:54.695953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.709285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375494a0-283c-4d0e-9e00-8bc3760791d5 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.750187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:54.783948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d96c0-5659-4495-a005-5f864cdaa09f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.784806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d96c0-5659-4495-a005-5f864cdaa09f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.785067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d96c0-5659-4495-a005-5f864cdaa09f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.785441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d96c0-5659-4495-a005-5f864cdaa09f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/d3044c8eb8e045b890c7d4b502bf8896/users/ae0a7ec855b94c4683b28cbe291d7a2a/roles/c2bfb311320f4b209f79391aff973025` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.787842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d96c0-5659-4495-a005-5f864cdaa09f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=d3044c8eb8e045b890c7d4b502bf8896, user_id=ae0a7ec855b94c4683b28cbe291d7a2a, role_id=c2bfb311320f4b209f79391aff973025)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.791408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d96c0-5659-4495-a005-5f864cdaa09f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.821790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3398/6804] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:54 2022] PUT /identity/v3/projects/d3044c8eb8e045b890c7d4b502bf8896/users/ae0a7ec855b94c4683b28cbe291d7a2a/roles/c2bfb311320f4b209f79391aff973025 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:54.826084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:54.839400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1826182-7200-41df-80a0-89e68ec0ae97 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:54.898890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:54.917539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59705380-b11f-4325-bb0b-b6b165b2a72d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:54.918675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59705380-b11f-4325-bb0b-b6b165b2a72d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.919022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59705380-b11f-4325-bb0b-b6b165b2a72d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.919830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59705380-b11f-4325-bb0b-b6b165b2a72d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:54.920157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59705380-b11f-4325-bb0b-b6b165b2a72d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:54.922324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59705380-b11f-4325-bb0b-b6b165b2a72d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:54.959368 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3407/6805] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:54 2022] GET /identity/v3/users => generated 7774 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:54.960731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c1b3-eb5b-499d-8073-7a2809dad279 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:54.960731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c1b3-eb5b-499d-8073-7a2809dad279 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:54.960915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c1b3-eb5b-499d-8073-7a2809dad279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:55.021905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-acb9c1b3-eb5b-499d-8073-7a2809dad279 None None] MFA Rules not processed for user `ae0a7ec855b94c4683b28cbe291d7a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:55.023208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-acb9c1b3-eb5b-499d-8073-7a2809dad279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:55.139943 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3399/6806] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:55.142006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb73604-82fd-4157-a105-0c42e6e88775 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:55.142949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb73604-82fd-4157-a105-0c42e6e88775 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:55.143407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb73604-82fd-4157-a105-0c42e6e88775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:55.168214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9eb73604-82fd-4157-a105-0c42e6e88775 None None] MFA Rules not processed for user `ae0a7ec855b94c4683b28cbe291d7a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:55.169907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9eb73604-82fd-4157-a105-0c42e6e88775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:55.179241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3408/6807] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:55.185898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9c1b3-eb5b-499d-8073-7a2809dad279 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:55.197871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9c1b3-eb5b-499d-8073-7a2809dad279 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:55.202472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae0a7ec855b94c4683b28cbe291d7a2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3044c8eb8e045b890c7d4b502bf8896', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1755465329', 'tempest-NotDelegatedRole-455054864'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:55.203169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:55.203500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:55.203777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:55.205141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] RBAC: Authorizing `identity:create_trust()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:55.206463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Sep 10 21:15:55.323507 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise ValueError(str(e)) Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Sep 10 21:15:55.324501 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Sep 10 21:15:55.325491 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Sep 10 21:15:55.325491 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Sep 10 21:15:55.325491 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Sep 10 21:15:55.325491 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Sep 10 21:15:55.325491 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 10 21:15:55.325491 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:55.325859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3400/6808] 10.222.0.33 () {60 vars in 1174 bytes} [Sat Sep 10 21:15:55 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 141 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 10 21:15:55.330093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb73604-82fd-4157-a105-0c42e6e88775 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:55.345965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb73604-82fd-4157-a105-0c42e6e88775 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:55.351559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:55.352432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:55.353156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:55.353631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/c2bfb311320f4b209f79391aff973025` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:55.356029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c2bfb311320f4b209f79391aff973025)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:55.358005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:55.427964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role c2bfb311320f4b209f79391aff973025 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:55.427964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.428831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.435402 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3409/6809] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:55 2022] DELETE /identity/v3/roles/c2bfb311320f4b209f79391aff973025 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:55.439352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:55.441258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:55.470671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a964ae-a89c-4032-88e3-ecbf539cbfde tempest-TrustsV3TestJSON-1837849008 tempest-user-1549424885] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:55.548694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:55.572647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:55.573529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:55.574059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:55.574600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/41157af62de34c37b7dfc6b447d1c979` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:55.576244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=41157af62de34c37b7dfc6b447d1c979)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:55.578063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:55.621976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role 41157af62de34c37b7dfc6b447d1c979 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:55.622621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.622998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.624784 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3401/6810] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:55 2022] DELETE /identity/v3/roles/41157af62de34c37b7dfc6b447d1c979 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:55.629093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:55.642234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc7c20-86c1-4818-a147-ec2b2fba6bfe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:55.717348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:55.718115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:55.718460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:55.718706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users/ae0a7ec855b94c4683b28cbe291d7a2a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:55.719869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ae0a7ec855b94c4683b28cbe291d7a2a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:55.721839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:55.772749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0a7ec855b94c4683b28cbe291d7a2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.786391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0a7ec855b94c4683b28cbe291d7a2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.792580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0a7ec855b94c4683b28cbe291d7a2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.808950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0a7ec855b94c4683b28cbe291d7a2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.809591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0a7ec855b94c4683b28cbe291d7a2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.836004 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3410/6811] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:55 2022] DELETE /identity/v3/users/ae0a7ec855b94c4683b28cbe291d7a2a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:55.839904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:55.841404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:55.859678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02974f71-bbd8-4894-a19c-5d9b5f57766f tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:55.924099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:55.924657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:55.924997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:55.925305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/d3044c8eb8e045b890c7d4b502bf8896` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:55.926767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d3044c8eb8e045b890c7d4b502bf8896)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:55.928719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:55.979946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3044c8eb8e045b890c7d4b502bf8896', 'request_id': 'req-7db33892-0060-453a-8adf-793d45fe65d7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.985764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3044c8eb8e045b890c7d4b502bf8896', 'request_id': 'req-7db33892-0060-453a-8adf-793d45fe65d7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:55.992547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3044c8eb8e045b890c7d4b502bf8896', 'request_id': 'req-7db33892-0060-453a-8adf-793d45fe65d7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:56.005928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] The token cache is being invalidate because project d3044c8eb8e045b890c7d4b502bf8896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:56.006216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:56.006829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:56.008306 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3402/6812] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:55 2022] DELETE /identity/v3/projects/d3044c8eb8e045b890c7d4b502bf8896 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:56.013959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.015407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:56.031413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454bb77-74ae-4e50-8b76-e673825f25d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.043697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:56.090994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.091605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.091891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.092170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.092796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.094763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.125020 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3411/6813] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:15:56 2022] POST /identity/v3/projects => generated 329 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:56.126267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.127652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:56.141207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db33892-0060-453a-8adf-793d45fe65d7 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.188844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:56.203924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.203924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.203924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.204357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.204978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.206940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.231069 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3403/6814] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:56 2022] POST /identity/v3/users => generated 416 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:56.235249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.246694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82896-745c-47f9-838f-514a5f556ee9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.251363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.251984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.252266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.252544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.253207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.254824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.270589 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3412/6815] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:56 2022] POST /identity/v3/roles => generated 243 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:56.275572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.286609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c01ae-c8c9-4f06-96a6-5a17910ba220 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.292006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.293106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.293396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.293675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.294384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.295997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.312408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3404/6816] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:56 2022] POST /identity/v3/roles => generated 247 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:15:56.316630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.328488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19647004-2c77-4887-9d75-41ef54d59d40 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.333778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.334691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.335040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.335385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/fe4c258f5b554416b9fbeb3d9d7c1545/users/25b47624bf8340c38a08c4c4ef12519c/roles/33ecb41e41e24655b4790356d461fba8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.356527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=fe4c258f5b554416b9fbeb3d9d7c1545, user_id=25b47624bf8340c38a08c4c4ef12519c, role_id=33ecb41e41e24655b4790356d461fba8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.359479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.379204 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3413/6817] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:56 2022] PUT /identity/v3/projects/fe4c258f5b554416b9fbeb3d9d7c1545/users/25b47624bf8340c38a08c4c4ef12519c/roles/33ecb41e41e24655b4790356d461fba8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:56.383286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.395395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1098-1736-4770-8810-dfda71d1f3bc tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.460508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367c13b-9efc-4d74-8e25-af41cf045e45 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.461105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c13b-9efc-4d74-8e25-af41cf045e45 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.461395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c13b-9efc-4d74-8e25-af41cf045e45 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.461678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367c13b-9efc-4d74-8e25-af41cf045e45 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/fe4c258f5b554416b9fbeb3d9d7c1545/users/25b47624bf8340c38a08c4c4ef12519c/roles/0828aff7912f41efabb7d9f717851716` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.464079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367c13b-9efc-4d74-8e25-af41cf045e45 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=fe4c258f5b554416b9fbeb3d9d7c1545, user_id=25b47624bf8340c38a08c4c4ef12519c, role_id=0828aff7912f41efabb7d9f717851716)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.466809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367c13b-9efc-4d74-8e25-af41cf045e45 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.486142 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3405/6818] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:56 2022] PUT /identity/v3/projects/fe4c258f5b554416b9fbeb3d9d7c1545/users/25b47624bf8340c38a08c4c4ef12519c/roles/0828aff7912f41efabb7d9f717851716 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:56.490343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.502803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ad8b-4c4b-4eff-89dc-91c4c8a2a7e4 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.584576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e3d04-0bd5-4d7b-87bc-78e49a13020a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.585191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e3d04-0bd5-4d7b-87bc-78e49a13020a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.585478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e3d04-0bd5-4d7b-87bc-78e49a13020a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.585763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e3d04-0bd5-4d7b-87bc-78e49a13020a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.586456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e3d04-0bd5-4d7b-87bc-78e49a13020a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.589154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e3d04-0bd5-4d7b-87bc-78e49a13020a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.617534 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3414/6819] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:56 2022] GET /identity/v3/users => generated 7774 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:56.622982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35faed-3447-4404-80bf-566e76b88a65 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.623457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35faed-3447-4404-80bf-566e76b88a65 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.623625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35faed-3447-4404-80bf-566e76b88a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.673388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3a35faed-3447-4404-80bf-566e76b88a65 None None] MFA Rules not processed for user `25b47624bf8340c38a08c4c4ef12519c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:56.675097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3a35faed-3447-4404-80bf-566e76b88a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:56.759665 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3406/6820] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:15:56.769807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862adb7c-c164-4b01-8b8d-78425b968cd5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.770655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862adb7c-c164-4b01-8b8d-78425b968cd5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.771092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862adb7c-c164-4b01-8b8d-78425b968cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.794895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-862adb7c-c164-4b01-8b8d-78425b968cd5 None None] MFA Rules not processed for user `25b47624bf8340c38a08c4c4ef12519c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:56.796567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-862adb7c-c164-4b01-8b8d-78425b968cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:56.806276 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3415/6821] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:56.809920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35faed-3447-4404-80bf-566e76b88a65 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.822828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35faed-3447-4404-80bf-566e76b88a65 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.827884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.828629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.829019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.829489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.830318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:create_trust()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.831899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:56.882421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:56.951877 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3407/6822] 10.222.0.33 () {60 vars in 1174 bytes} [Sat Sep 10 21:15:56 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:56.956033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862adb7c-c164-4b01-8b8d-78425b968cd5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:56.969905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862adb7c-c164-4b01-8b8d-78425b968cd5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:56.982851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:56.983243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:56.983531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:56.983818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:56.990816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:get_trust(trust_id=a9afc1ecd7324d838dd66075ba1fda84)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:56.992367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.004171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3416/6823] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:56 2022] GET /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84 => generated 833 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:57.005187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.016263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacb60-165d-4663-9fb5-bcc9dd17e481 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.022369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.023215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.023530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.023782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.031572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=a9afc1ecd7324d838dd66075ba1fda84)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.033914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.042085 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3408/6824] 10.222.0.33 () {58 vars in 1269 bytes} [Sat Sep 10 21:15:57 2022] GET /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles => generated 370 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:57.046172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.055772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589f0d-7ca4-4441-9422-33107f8e2351 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.060418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.061074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.061359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.061639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/33ecb41e41e24655b4790356d461fba8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.069555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:get_role_for_trust(trust_id=a9afc1ecd7324d838dd66075ba1fda84, role_id=33ecb41e41e24655b4790356d461fba8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.072216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.087691 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3417/6825] 10.222.0.33 () {58 vars in 1368 bytes} [Sat Sep 10 21:15:57 2022] GET /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/33ecb41e41e24655b4790356d461fba8 => generated 243 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:15:57.088943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.100120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239932f-adce-4e6a-a038-558ed9c31a63 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.106302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.107016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.107302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.107581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/33ecb41e41e24655b4790356d461fba8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.114651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:get_role_for_trust(trust_id=a9afc1ecd7324d838dd66075ba1fda84, role_id=33ecb41e41e24655b4790356d461fba8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.116409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.127483 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3409/6826] 10.222.0.33 () {58 vars in 1369 bytes} [Sat Sep 10 21:15:57 2022] HEAD /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/33ecb41e41e24655b4790356d461fba8 => generated 0 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 10 21:15:57.132266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.149326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f229ca-1967-4bf2-a12c-76ca8c84b70f tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.158083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.159654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.159907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.160193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/0828aff7912f41efabb7d9f717851716` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.169042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:get_role_for_trust(trust_id=a9afc1ecd7324d838dd66075ba1fda84, role_id=0828aff7912f41efabb7d9f717851716)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.170573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.187216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Could not find role: 0828aff7912f41efabb7d9f717851716.: keystone.exception.RoleNotFound: Could not find role: 0828aff7912f41efabb7d9f717851716. Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 21:15:57.195267 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 0828aff7912f41efabb7d9f717851716. Sep 10 21:15:57.196162 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:15:57.206961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3418/6827] 10.222.0.33 () {58 vars in 1368 bytes} [Sat Sep 10 21:15:57 2022] GET /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/0828aff7912f41efabb7d9f717851716 => generated 110 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:57.210987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.221349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86e45e-f9a7-4fd2-a95f-f3cfc3ac6360 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.225955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.226495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.226778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.227055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/0828aff7912f41efabb7d9f717851716` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.234844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:get_role_for_trust(trust_id=a9afc1ecd7324d838dd66075ba1fda84, role_id=0828aff7912f41efabb7d9f717851716)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.236307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Could not find role: 0828aff7912f41efabb7d9f717851716.: keystone.exception.RoleNotFound: Could not find role: 0828aff7912f41efabb7d9f717851716. Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 21:15:57.249347 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 0828aff7912f41efabb7d9f717851716. Sep 10 21:15:57.250476 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:57.250547 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3410/6828] 10.222.0.33 () {58 vars in 1369 bytes} [Sat Sep 10 21:15:57 2022] HEAD /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84/roles/0828aff7912f41efabb7d9f717851716 => generated 0 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:57.254905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.267252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf705734-3367-47ee-8368-4d8c84573f02 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.272441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.273351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.274095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.275003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.283268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorizing `identity:delete_trust(trust_id=a9afc1ecd7324d838dd66075ba1fda84)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.284723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.315031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'a9afc1ecd7324d838dd66075ba1fda84', 'request_id': 'req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.323238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'a9afc1ecd7324d838dd66075ba1fda84', 'request_id': 'req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.323868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'a9afc1ecd7324d838dd66075ba1fda84', 'request_id': 'req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.330496 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3419/6829] 10.222.0.33 () {58 vars in 1254 bytes} [Sat Sep 10 21:15:57 2022] DELETE /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:57.334638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.336400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:57.349972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfaa9d8-ce4f-4b65-9952-ef3ef0284e3a tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.354945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25b47624bf8340c38a08c4c4ef12519c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-795244532', 'tempest-NotDelegatedRole-1555889019'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.355911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.356190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.356465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] PATH_INFO: `/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Could not find trust: a9afc1ecd7324d838dd66075ba1fda84.: keystone.exception.TrustNotFound: Could not find trust: a9afc1ecd7324d838dd66075ba1fda84. Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 10 21:15:57.365159 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: a9afc1ecd7324d838dd66075ba1fda84. Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:57.366373 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3411/6830] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:57 2022] GET /identity/v3/OS-TRUST/trusts/a9afc1ecd7324d838dd66075ba1fda84 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:57.372440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.373879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:57.393140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48ad0-1d0c-48b4-a9f2-76313edfd5c2 tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.399132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.400253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.400746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.401097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/0828aff7912f41efabb7d9f717851716` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.402855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0828aff7912f41efabb7d9f717851716)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.404744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.447958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role 0828aff7912f41efabb7d9f717851716 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:57.448534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.449355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.451680 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3420/6831] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:57 2022] DELETE /identity/v3/roles/0828aff7912f41efabb7d9f717851716 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:57.455709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.457396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:57.473310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a8827-6287-423b-9807-9054a23d00ab tempest-TrustsV3TestJSON-1092238441 tempest-user-1835141670] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.535884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.536529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.536816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.537093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/33ecb41e41e24655b4790356d461fba8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.538593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=33ecb41e41e24655b4790356d461fba8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.540888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.594859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role 33ecb41e41e24655b4790356d461fba8 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:57.595366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.596120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.599166 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3412/6832] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:57 2022] DELETE /identity/v3/roles/33ecb41e41e24655b4790356d461fba8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:57.603921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.622503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45debc-a39c-4985-a20e-79862c099410 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.708394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.709623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.710598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.710901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users/25b47624bf8340c38a08c4c4ef12519c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.712681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_user(user_id=25b47624bf8340c38a08c4c4ef12519c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.716032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:57.793850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b47624bf8340c38a08c4c4ef12519c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.823301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b47624bf8340c38a08c4c4ef12519c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.824058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b47624bf8340c38a08c4c4ef12519c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.833021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b47624bf8340c38a08c4c4ef12519c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.833670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b47624bf8340c38a08c4c4ef12519c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:57.847413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=83,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=14 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:57.856249 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3421/6833] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:57 2022] DELETE /identity/v3/users/25b47624bf8340c38a08c4c4ef12519c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:57.857887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:57.859501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:57.874393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375bd44-db60-405b-b070-5546e31a149d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:57.907582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:57.941661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:57.942465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:57.942760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:57.943041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/fe4c258f5b554416b9fbeb3d9d7c1545` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:57.944180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fe4c258f5b554416b9fbeb3d9d7c1545)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:57.946153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.007038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'request_id': 'req-3356a04d-dd4f-4275-9900-1478a47fde4b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:58.014524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'request_id': 'req-3356a04d-dd4f-4275-9900-1478a47fde4b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:58.022766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe4c258f5b554416b9fbeb3d9d7c1545', 'request_id': 'req-3356a04d-dd4f-4275-9900-1478a47fde4b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:58.044715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] The token cache is being invalidate because project fe4c258f5b554416b9fbeb3d9d7c1545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:58.045066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:58.045685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:58.052067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3413/6834] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:57 2022] DELETE /identity/v3/projects/fe4c258f5b554416b9fbeb3d9d7c1545 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:58.054436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.056545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:58.072993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433492fc-720a-4a6a-96a8-a0794215e3fe tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.139061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.139684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.139970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.140249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.140975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.143088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.170922 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3422/6835] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:15:58 2022] POST /identity/v3/projects => generated 328 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:58.174728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.176263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:58.191329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356a04d-dd4f-4275-9900-1478a47fde4b tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.256491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.256791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.257055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.257338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.258035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.260379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.288650 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3414/6836] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:15:58 2022] POST /identity/v3/users => generated 416 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:58.293135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.305567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97190d2f-5b58-4c41-a31c-964d9026e126 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.310401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.310997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.311333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.311620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.312293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.314003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.331154 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3423/6837] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:58 2022] POST /identity/v3/roles => generated 244 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:58.334992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.346669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a016f72-3ddf-4761-8517-9106a7279501 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.351302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.351898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.352183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.352457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.353684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.355610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.374251 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3415/6838] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:15:58 2022] POST /identity/v3/roles => generated 246 bytes in 37 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:58.375472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.386330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5fad-67ca-4117-8e5b-453f70f2e7ea tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.390938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.391590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.391873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.392150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/96b0d63c80204b0fbf00d6b60bff360e/users/c5e9b1d37ad34ba8b470da63b2a17309/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.408218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=96b0d63c80204b0fbf00d6b60bff360e, user_id=c5e9b1d37ad34ba8b470da63b2a17309, role_id=c3912ed01d0c4dbeb06ab7a3c6c46d1e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.410973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.427085 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3424/6839] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:58 2022] PUT /identity/v3/projects/96b0d63c80204b0fbf00d6b60bff360e/users/c5e9b1d37ad34ba8b470da63b2a17309/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:58.431334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.442226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891c0a4-ec5f-4258-9979-2c12c1ce0921 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.478882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:58.504657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d7c2a-9b6b-48b1-a4e8-7f9264613d4d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.505317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d7c2a-9b6b-48b1-a4e8-7f9264613d4d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.505607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d7c2a-9b6b-48b1-a4e8-7f9264613d4d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.505924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d7c2a-9b6b-48b1-a4e8-7f9264613d4d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/96b0d63c80204b0fbf00d6b60bff360e/users/c5e9b1d37ad34ba8b470da63b2a17309/roles/ffb14902107445b9ad2cf3116300f31a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.508121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d7c2a-9b6b-48b1-a4e8-7f9264613d4d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:create_grant(project_id=96b0d63c80204b0fbf00d6b60bff360e, user_id=c5e9b1d37ad34ba8b470da63b2a17309, role_id=ffb14902107445b9ad2cf3116300f31a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.511076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d7c2a-9b6b-48b1-a4e8-7f9264613d4d tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.523498 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3416/6840] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:15:58 2022] PUT /identity/v3/projects/96b0d63c80204b0fbf00d6b60bff360e/users/c5e9b1d37ad34ba8b470da63b2a17309/roles/ffb14902107445b9ad2cf3116300f31a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:58.528320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.539467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec6943-9413-4e3c-8b80-143a39a57884 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.599532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648fb50-6dee-4481-b43f-487eb3ad49cd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.600141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648fb50-6dee-4481-b43f-487eb3ad49cd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.600425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648fb50-6dee-4481-b43f-487eb3ad49cd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.600704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648fb50-6dee-4481-b43f-487eb3ad49cd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.601352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648fb50-6dee-4481-b43f-487eb3ad49cd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.603953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648fb50-6dee-4481-b43f-487eb3ad49cd tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.632768 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3425/6841] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:15:58 2022] GET /identity/v3/users => generated 7774 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:15:58.638359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846efa2d-877f-4543-bff9-8ed0223c1b37 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.638651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846efa2d-877f-4543-bff9-8ed0223c1b37 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.638931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846efa2d-877f-4543-bff9-8ed0223c1b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.678585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-846efa2d-877f-4543-bff9-8ed0223c1b37 None None] MFA Rules not processed for user `c5e9b1d37ad34ba8b470da63b2a17309`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:58.679821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-846efa2d-877f-4543-bff9-8ed0223c1b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:58.747091 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3417/6842] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:58.758066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07937f4-1b28-4bb7-b4f3-2a1af1875502 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.758763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07937f4-1b28-4bb7-b4f3-2a1af1875502 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.760295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07937f4-1b28-4bb7-b4f3-2a1af1875502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.780960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a07937f4-1b28-4bb7-b4f3-2a1af1875502 None None] MFA Rules not processed for user `c5e9b1d37ad34ba8b470da63b2a17309`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:15:58.782196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a07937f4-1b28-4bb7-b4f3-2a1af1875502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:58.789396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3426/6843] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:15:58.794042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846efa2d-877f-4543-bff9-8ed0223c1b37 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.803967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846efa2d-877f-4543-bff9-8ed0223c1b37 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.809224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.809856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.810177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.810452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.811108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:create_trust()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.812249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.895649 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3418/6844] 10.222.0.33 () {60 vars in 1174 bytes} [Sat Sep 10 21:15:58 2022] POST /identity/v3/OS-TRUST/trusts => generated 835 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:15:58.900392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07937f4-1b28-4bb7-b4f3-2a1af1875502 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.909581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07937f4-1b28-4bb7-b4f3-2a1af1875502 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.915380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.916402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.916662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.918323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.925469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:get_trust(trust_id=bdcbd93c502c47269701798ab32199c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.927165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.937914 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3427/6845] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:58 2022] GET /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8 => generated 835 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:58.941369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:58.953393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c0c64-308c-45f1-86ca-830c3cc03d8f tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:58.957644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:58.958460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:58.958845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:58.959118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:58.966289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=bdcbd93c502c47269701798ab32199c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:58.967718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:58.989283 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3419/6846] 10.222.0.33 () {58 vars in 1269 bytes} [Sat Sep 10 21:15:58 2022] GET /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles => generated 371 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:58.990687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.002293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490872eb-7bc7-4f4d-b09f-507200031723 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.007676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.008496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.009724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.010114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.022014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:get_role_for_trust(trust_id=bdcbd93c502c47269701798ab32199c8, role_id=c3912ed01d0c4dbeb06ab7a3c6c46d1e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.022014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.026137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:59.034710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.038426 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3428/6847] 10.222.0.33 () {58 vars in 1368 bytes} [Sat Sep 10 21:15:58 2022] GET /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e => generated 244 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:15:59.045667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558eccea-6cbe-48c3-ba53-b437a721a4b9 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.050429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.052384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.052384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.052384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.058305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:get_role_for_trust(trust_id=bdcbd93c502c47269701798ab32199c8, role_id=c3912ed01d0c4dbeb06ab7a3c6c46d1e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.059805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.066386 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3420/6848] 10.222.0.33 () {58 vars in 1369 bytes} [Sat Sep 10 21:15:59 2022] HEAD /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e => generated 0 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 10 21:15:59.070998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.082458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57cc8a-b790-449b-8237-571c3b7e50ab tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.087656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.088407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.088688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.088962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/ffb14902107445b9ad2cf3116300f31a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.095526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:get_role_for_trust(trust_id=bdcbd93c502c47269701798ab32199c8, role_id=ffb14902107445b9ad2cf3116300f31a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.096965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Could not find role: ffb14902107445b9ad2cf3116300f31a.: keystone.exception.RoleNotFound: Could not find role: ffb14902107445b9ad2cf3116300f31a. Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 21:15:59.108653 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ffb14902107445b9ad2cf3116300f31a. Sep 10 21:15:59.109717 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:15:59.109797 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3429/6849] 10.222.0.33 () {58 vars in 1368 bytes} [Sat Sep 10 21:15:59 2022] GET /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/ffb14902107445b9ad2cf3116300f31a => generated 110 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:59.114101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.124389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a730b4b-33cb-41c8-9224-8c9f4a6ab910 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.129036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.129577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.129913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.130196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/ffb14902107445b9ad2cf3116300f31a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.133069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:59.141364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:get_role_for_trust(trust_id=bdcbd93c502c47269701798ab32199c8, role_id=ffb14902107445b9ad2cf3116300f31a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.142880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Could not find role: ffb14902107445b9ad2cf3116300f31a.: keystone.exception.RoleNotFound: Could not find role: ffb14902107445b9ad2cf3116300f31a. Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 21:15:59.155202 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ffb14902107445b9ad2cf3116300f31a. Sep 10 21:15:59.156308 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:59.158155 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3421/6850] 10.222.0.33 () {58 vars in 1369 bytes} [Sat Sep 10 21:15:59 2022] HEAD /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8/roles/ffb14902107445b9ad2cf3116300f31a => generated 0 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 10 21:15:59.163282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.177279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b490a8-7a0b-465f-8aa8-1a45fe67ac2d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.184138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.184784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.185073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.185345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.192822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorizing `identity:delete_trust(trust_id=bdcbd93c502c47269701798ab32199c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.194446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.215906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'bdcbd93c502c47269701798ab32199c8', 'request_id': 'req-3c1b129f-5d27-4194-996e-25dded9be86c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.227157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'bdcbd93c502c47269701798ab32199c8', 'request_id': 'req-3c1b129f-5d27-4194-996e-25dded9be86c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.228127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'bdcbd93c502c47269701798ab32199c8', 'request_id': 'req-3c1b129f-5d27-4194-996e-25dded9be86c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.242344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3430/6851] 10.222.0.33 () {58 vars in 1254 bytes} [Sat Sep 10 21:15:59 2022] DELETE /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:59.245147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.252532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:59.272496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da90a23-0ff0-4797-bf05-96028fbbe52d tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.280131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e9b1d37ad34ba8b470da63b2a17309', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96b0d63c80204b0fbf00d6b60bff360e', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1676970293', 'tempest-NotDelegatedRole-883680747'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.280778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.281144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.281326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] PATH_INFO: `/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Could not find trust: bdcbd93c502c47269701798ab32199c8.: keystone.exception.TrustNotFound: Could not find trust: bdcbd93c502c47269701798ab32199c8. Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 10 21:15:59.291151 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: bdcbd93c502c47269701798ab32199c8. Sep 10 21:15:59.292107 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:15:59.295565 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3422/6852] 10.222.0.33 () {58 vars in 1251 bytes} [Sat Sep 10 21:15:59 2022] GET /identity/v3/OS-TRUST/trusts/bdcbd93c502c47269701798ab32199c8 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:15:59.296695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.300109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:59.319059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b129f-5d27-4194-996e-25dded9be86c tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.328086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.328767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.329052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.329331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/ffb14902107445b9ad2cf3116300f31a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.332657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=ffb14902107445b9ad2cf3116300f31a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.335693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.394761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role ffb14902107445b9ad2cf3116300f31a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:59.395193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.396108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.399696 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3431/6853] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:59 2022] DELETE /identity/v3/roles/ffb14902107445b9ad2cf3116300f31a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:59.402581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.404403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:59.423045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f752e-68d2-47bd-b431-e92a128ff6d0 tempest-TrustsV3TestJSON-216543783 tempest-user-1118292411] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.499633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.500367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.500729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.501017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.502640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c3912ed01d0c4dbeb06ab7a3c6c46d1e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.504477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.572247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invalidating the token cache because role c3912ed01d0c4dbeb06ab7a3c6c46d1e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:15:59.572733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.573563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.576215 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3423/6854] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:59 2022] DELETE /identity/v3/roles/c3912ed01d0c4dbeb06ab7a3c6c46d1e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:15:59.580160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.599999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593cf91-ac87-412e-a75d-dc4ce36fda7a tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.709096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.709814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.710208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.710492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/users/c5e9b1d37ad34ba8b470da63b2a17309` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.711626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_user(user_id=c5e9b1d37ad34ba8b470da63b2a17309)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.713730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.765276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e9b1d37ad34ba8b470da63b2a17309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.773290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e9b1d37ad34ba8b470da63b2a17309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.774043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e9b1d37ad34ba8b470da63b2a17309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.788141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e9b1d37ad34ba8b470da63b2a17309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.788827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e9b1d37ad34ba8b470da63b2a17309'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.791164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:59.816391 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3432/6855] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:15:59 2022] DELETE /identity/v3/users/c5e9b1d37ad34ba8b470da63b2a17309 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:15:59.820658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:15:59.823509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:15:59.842661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213b5c1-8ff8-4385-80fa-ac06edb1cfb9 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:15:59.918577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b19eec13d3e4c0083e3399abf4a5acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be2e2b9a7452448f86ac2b665a05d67f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:15:59.920050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:15:59.920343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:15:59.920628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] PATH_INFO: `/v3/projects/96b0d63c80204b0fbf00d6b60bff360e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:15:59.921798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorizing `identity:delete_project(project_id=96b0d63c80204b0fbf00d6b60bff360e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:15:59.923972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:15:59.980345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:15:59.982948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b0d63c80204b0fbf00d6b60bff360e', 'request_id': 'req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.991003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b0d63c80204b0fbf00d6b60bff360e', 'request_id': 'req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:15:59.998864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b0d63c80204b0fbf00d6b60bff360e', 'request_id': 'req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.011695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] The token cache is being invalidate because project 96b0d63c80204b0fbf00d6b60bff360e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:00.011984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.012613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-92b8568d-14ce-4da4-84f6-3ba3c8b7497c tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.015022 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3424/6856] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:15:59 2022] DELETE /identity/v3/projects/96b0d63c80204b0fbf00d6b60bff360e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:16:00.022403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:00.023788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:00.142117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e282b0-78fc-4320-818f-cf4326732337 tempest-TrustsV3TestJSON-409580453 tempest-TrustsV3TestJSON-409580453-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:00.208348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:00.209023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:00.209311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:00.209590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] PATH_INFO: `/v3/users/49b1b9f8857a40bb8266b9a626e4bcc5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:00.210908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b1b9f8857a40bb8266b9a626e4bcc5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:00.213095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:00.267637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b1b9f8857a40bb8266b9a626e4bcc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.276325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b1b9f8857a40bb8266b9a626e4bcc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.277104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b1b9f8857a40bb8266b9a626e4bcc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.285021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b1b9f8857a40bb8266b9a626e4bcc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.285619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b1b9f8857a40bb8266b9a626e4bcc5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:00.304822 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3433/6857] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:00 2022] DELETE /identity/v3/users/49b1b9f8857a40bb8266b9a626e4bcc5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:00.309379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:00.309669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:00.309970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:00.331585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:00.332801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:00.416326 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3425/6858] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:00.429174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:00.430759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:00.445581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:00.524090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba4dda-1af3-448b-9dd4-cc12dc546476 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:00.528238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968504d-fc6a-4051-8494-820c67617b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:00.529286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968504d-fc6a-4051-8494-820c67617b80 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:00.529782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968504d-fc6a-4051-8494-820c67617b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:00.530103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968504d-fc6a-4051-8494-820c67617b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:00.548328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968504d-fc6a-4051-8494-820c67617b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:00.549860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968504d-fc6a-4051-8494-820c67617b80 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:00.563661 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3434/6859] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:00.938877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:00.941189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:00.969410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:00.969701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:00.978024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619cd7-7a77-4439-84cb-d6adb76ccd16 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:00.983773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:00.992377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:00.994051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:00.994974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:00.994974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] PATH_INFO: `/v3/projects/c471963f013f4f66b23c29851e719a89` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:01.003643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c471963f013f4f66b23c29851e719a89)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:01.020172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:01.021590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:01.025621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:01.108776 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3435/6860] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:01.122914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:01.126548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:01.129705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c471963f013f4f66b23c29851e719a89', 'request_id': 'req-d049fea0-e62c-4df7-8567-523f15d90c6a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.146674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c471963f013f4f66b23c29851e719a89', 'request_id': 'req-d049fea0-e62c-4df7-8567-523f15d90c6a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.157446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c471963f013f4f66b23c29851e719a89', 'request_id': 'req-d049fea0-e62c-4df7-8567-523f15d90c6a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.187312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] The token cache is being invalidate because project c471963f013f4f66b23c29851e719a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:01.187777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.189541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.192620 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3426/6861] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:00 2022] DELETE /identity/v3/projects/c471963f013f4f66b23c29851e719a89 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:01.198612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:01.201176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:01.324929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeae808-f21e-4756-be39-606046a35fc4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:01.410694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fea0-e62c-4df7-8567-523f15d90c6a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:01.416673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:01.417395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:01.417673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:01.418765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] PATH_INFO: `/v3/users/2b19eec13d3e4c0083e3399abf4a5acf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:01.420058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b19eec13d3e4c0083e3399abf4a5acf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:01.422486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:01.424653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:01.464295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:01.488531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70407702-b8b2-4257-b983-21767b48f7fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:01.489167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70407702-b8b2-4257-b983-21767b48f7fc service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:01.489455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70407702-b8b2-4257-b983-21767b48f7fc service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:01.494096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70407702-b8b2-4257-b983-21767b48f7fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:01.498504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70407702-b8b2-4257-b983-21767b48f7fc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:01.509984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b19eec13d3e4c0083e3399abf4a5acf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.534000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b19eec13d3e4c0083e3399abf4a5acf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.547246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b19eec13d3e4c0083e3399abf4a5acf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.559540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b19eec13d3e4c0083e3399abf4a5acf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.560340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b19eec13d3e4c0083e3399abf4a5acf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:01.567882 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3427/6862] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:01 2022] DELETE /identity/v3/users/2b19eec13d3e4c0083e3399abf4a5acf => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:01.600787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70407702-b8b2-4257-b983-21767b48f7fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:01.603959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70407702-b8b2-4257-b983-21767b48f7fc service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:01.617022 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3436/6863] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:16:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:16:02.183045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:02.187103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:02.342109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b3256-b695-459c-a238-198d97effe9c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:02.348073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:02.348737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:02.349221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:02.349516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] PATH_INFO: `/v3/projects/be2e2b9a7452448f86ac2b665a05d67f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:02.350970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2e2b9a7452448f86ac2b665a05d67f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:02.353466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:02.433808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2e2b9a7452448f86ac2b665a05d67f', 'request_id': 'req-cb375c8e-f598-4e8c-8e51-0273c840db39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:02.440724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2e2b9a7452448f86ac2b665a05d67f', 'request_id': 'req-cb375c8e-f598-4e8c-8e51-0273c840db39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:02.449104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2e2b9a7452448f86ac2b665a05d67f', 'request_id': 'req-cb375c8e-f598-4e8c-8e51-0273c840db39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:02.483215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] The token cache is being invalidate because project be2e2b9a7452448f86ac2b665a05d67f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:02.483517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:02.484913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cb375c8e-f598-4e8c-8e51-0273c840db39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:02.490028 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3428/6864] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:02 2022] DELETE /identity/v3/projects/be2e2b9a7452448f86ac2b665a05d67f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:02.491783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a04a07-475b-4b0b-8dcb-1937debd30eb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:02.492252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a04a07-475b-4b0b-8dcb-1937debd30eb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:02.492378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a04a07-475b-4b0b-8dcb-1937debd30eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:02.538070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d1a04a07-475b-4b0b-8dcb-1937debd30eb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:02.542542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d1a04a07-475b-4b0b-8dcb-1937debd30eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:02.632053 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3437/6865] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:16:02.639994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22405cf7-5667-4bdc-9585-8170f8f710fd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:02.640295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22405cf7-5667-4bdc-9585-8170f8f710fd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:02.640577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22405cf7-5667-4bdc-9585-8170f8f710fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:02.673933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-22405cf7-5667-4bdc-9585-8170f8f710fd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:02.675186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-22405cf7-5667-4bdc-9585-8170f8f710fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:02.684556 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3429/6866] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:02.688463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a04a07-475b-4b0b-8dcb-1937debd30eb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:02.704923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a04a07-475b-4b0b-8dcb-1937debd30eb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:02.720864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:02.722516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:02.722796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:02.723071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:02.723735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:16:02.724039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:02.725653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:02.762621 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3438/6867] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:16:02.764212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22405cf7-5667-4bdc-9585-8170f8f710fd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:02.771181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:02.783823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22405cf7-5667-4bdc-9585-8170f8f710fd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:02.795297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:02.795958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:02.796243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:02.796518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:02.797224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:02.799418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:02.862423 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3430/6868] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:02 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:02.867420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:02.897407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696baad-6c23-4e69-8bb9-71e1e8af79f5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:03.030022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:03.030849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:03.031286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:03.031596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:03.032436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:03.034817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:03.079560 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3439/6869] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:02 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:03.083363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:03.098951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93f41b-b712-43fc-b56d-f68d195687d6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:03.106189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:03.107199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:03.107766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:03.108224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:03.109025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:03.110956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:03.133074 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3431/6870] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:03.135854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:03.147859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db55b5-8e15-4e90-8026-32e02c5f6b47 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:03.162090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59116b7f-87d1-4df2-aad7-f0ef27115037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:03.163023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116b7f-87d1-4df2-aad7-f0ef27115037 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:03.163466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116b7f-87d1-4df2-aad7-f0ef27115037 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:03.164565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59116b7f-87d1-4df2-aad7-f0ef27115037 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:03.166025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59116b7f-87d1-4df2-aad7-f0ef27115037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:03.168548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59116b7f-87d1-4df2-aad7-f0ef27115037 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:03.191197 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3440/6871] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:03.195039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:03.213710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14fd95-e218-4790-9b7c-0e56946c59cd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:03.224870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:03.225555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:03.225862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:03.226146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] PATH_INFO: `/v3/projects/f22dee49d0664c33a30034d1185a8e94/users/55301e893f144df190bfeba79bdbea31/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:03.260424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22dee49d0664c33a30034d1185a8e94, user_id=55301e893f144df190bfeba79bdbea31, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:03.263435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:03.307398 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3432/6872] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:03 2022] PUT /identity/v3/projects/f22dee49d0664c33a30034d1185a8e94/users/55301e893f144df190bfeba79bdbea31/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:16:03.309153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04909a1f-4152-42b4-9060-6c89ab20f037 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:03.309449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04909a1f-4152-42b4-9060-6c89ab20f037 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:03.309723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04909a1f-4152-42b4-9060-6c89ab20f037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:03.320357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:03.395406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-04909a1f-4152-42b4-9060-6c89ab20f037 None None] MFA Rules not processed for user `55301e893f144df190bfeba79bdbea31`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:03.399237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-04909a1f-4152-42b4-9060-6c89ab20f037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:03.536599 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3441/6873] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:03.537854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:03.737397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6132fb-03c8-4a00-8202-f47247ddea83 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:03.757468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:03.758231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:03.758533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:03.760223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:03.760940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:03.768266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:03.826311 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3433/6874] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:03 2022] POST /identity/v3/projects => generated 369 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:03.831499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04909a1f-4152-42b4-9060-6c89ab20f037 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:03.863329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04909a1f-4152-42b4-9060-6c89ab20f037 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:03.970538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:03.971174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:03.971462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:03.971745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:03.972536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:03.974676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:04.019738 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3442/6875] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:03 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:04.022616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:04.039028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d09e-d155-4b29-9dfa-e91d456acd0b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:04.054220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:04.055015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.055404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.056512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.059032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:04.063274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:04.083804 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3434/6876] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:04.088288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:04.109657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1e03-4f75-4c00-84d0-6c4bfdc9ef73 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:04.129051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:04.129485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.129897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.130280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] PATH_INFO: `/v3/projects/fe29ae6d26b34e718d03a5e93fae962a/users/bdffa358f833486693ac6fe9f6031c3d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.151740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe29ae6d26b34e718d03a5e93fae962a, user_id=bdffa358f833486693ac6fe9f6031c3d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:04.155753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:04.178936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3443/6877] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:04 2022] PUT /identity/v3/projects/fe29ae6d26b34e718d03a5e93fae962a/users/bdffa358f833486693ac6fe9f6031c3d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:04.183392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:04.196990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5561bfc-257b-48d7-88d8-bc8b51650d16 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:04.290420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192b8a8-4bcc-4c5f-b2d3-8b0f6aa42799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:04.291643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192b8a8-4bcc-4c5f-b2d3-8b0f6aa42799 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.292030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192b8a8-4bcc-4c5f-b2d3-8b0f6aa42799 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.292423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192b8a8-4bcc-4c5f-b2d3-8b0f6aa42799 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.293173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192b8a8-4bcc-4c5f-b2d3-8b0f6aa42799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:04.294899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192b8a8-4bcc-4c5f-b2d3-8b0f6aa42799 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:04.309727 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3435/6878] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:16:04.313547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:04.330659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92540a74-c0ff-4e10-a0d3-519ca8f331e8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:04.335873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:04.336556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.337665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.337991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] PATH_INFO: `/v3/projects/fe29ae6d26b34e718d03a5e93fae962a/users/bdffa358f833486693ac6fe9f6031c3d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.341429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe29ae6d26b34e718d03a5e93fae962a, user_id=bdffa358f833486693ac6fe9f6031c3d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:04.344520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:04.382892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3444/6879] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:04 2022] PUT /identity/v3/projects/fe29ae6d26b34e718d03a5e93fae962a/users/bdffa358f833486693ac6fe9f6031c3d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:04.394992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1306a7-c8d8-4e83-9ebe-7dc40a5671e4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.395686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1306a7-c8d8-4e83-9ebe-7dc40a5671e4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.396564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1306a7-c8d8-4e83-9ebe-7dc40a5671e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.458688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ac1306a7-c8d8-4e83-9ebe-7dc40a5671e4 None None] MFA Rules not processed for user `bdffa358f833486693ac6fe9f6031c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:04.460244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ac1306a7-c8d8-4e83-9ebe-7dc40a5671e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:04.467864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:04.618910 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3436/6880] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:04.622220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:04.640453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db283dba-ab62-41be-9365-323c6aa8b818 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:04.652748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdffa358f833486693ac6fe9f6031c3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe29ae6d26b34e718d03a5e93fae962a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:04.653759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.654320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.654786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.655606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:04.658836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:04.759896 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3445/6881] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:04 2022] POST /identity/v3/projects => generated 358 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:04.761482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1306a7-c8d8-4e83-9ebe-7dc40a5671e4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:04.773520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1306a7-c8d8-4e83-9ebe-7dc40a5671e4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:04.875544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdffa358f833486693ac6fe9f6031c3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe29ae6d26b34e718d03a5e93fae962a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:04.876302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.876688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.877045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.877778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:04.880020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:04.922474 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3437/6882] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:04 2022] POST /identity/v3/users => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:04.924943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:04.947154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b91385-a761-4255-8c4d-160c89333869 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:04.954538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cc483-d2d2-4ff2-aae9-d39b4fffaa44 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdffa358f833486693ac6fe9f6031c3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe29ae6d26b34e718d03a5e93fae962a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:04.955482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cc483-d2d2-4ff2-aae9-d39b4fffaa44 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:04.955871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cc483-d2d2-4ff2-aae9-d39b4fffaa44 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:04.956153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cc483-d2d2-4ff2-aae9-d39b4fffaa44 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:04.957087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cc483-d2d2-4ff2-aae9-d39b4fffaa44 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:16:04.957624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cc483-d2d2-4ff2-aae9-d39b4fffaa44 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:04.960148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cc483-d2d2-4ff2-aae9-d39b4fffaa44 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:05.007272 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3446/6883] 10.222.0.33 () {58 vars in 1157 bytes} [Sat Sep 10 21:16:04 2022] GET /identity/v3/users?domain_id=default => generated 7449 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:05.012268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:05.048087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1edf5e-0e7d-4bb1-831a-a54bbf5e2aa0 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:05.060027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdffa358f833486693ac6fe9f6031c3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe29ae6d26b34e718d03a5e93fae962a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:05.061474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:05.061623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:05.063041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] PATH_INFO: `/v3/users/34017ba955694555a73a6c2f21336b1f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:05.093519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorizing `identity:update_user(user_id=34017ba955694555a73a6c2f21336b1f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:05.095404 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:16:05.095404 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:16:05.095404 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:16:05.096836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:05.134060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '34017ba955694555a73a6c2f21336b1f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.147169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invalidating the token cache because user 34017ba955694555a73a6c2f21336b1f was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:05.147506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.148249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.154033 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3438/6884] 10.222.0.33 () {60 vars in 1244 bytes} [Sat Sep 10 21:16:05 2022] PATCH /identity/v3/users/34017ba955694555a73a6c2f21336b1f => generated 488 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:16:05.156654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:05.157239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:05.157868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:05.179412 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.auth.plugins.core [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] User is disabled: 34017ba955694555a73a6c2f21336b1f: AssertionError: User is disabled: 34017ba955694555a73a6c2f21336b1f Sep 10 21:16:05.197488 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.server.flask.application [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 10 21:16:05.199103 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3447/6885] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:05 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 44 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 10 21:16:05.203665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:05.205354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:05.229927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb161fa-858d-4ec7-a3e0-bc311721e27d tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:05.245816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdffa358f833486693ac6fe9f6031c3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe29ae6d26b34e718d03a5e93fae962a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:05.246600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:05.246953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:05.247298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] PATH_INFO: `/v3/users/34017ba955694555a73a6c2f21336b1f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:05.278699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorizing `identity:delete_user(user_id=34017ba955694555a73a6c2f21336b1f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:05.281672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:05.379348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:05.400924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34017ba955694555a73a6c2f21336b1f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.436957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34017ba955694555a73a6c2f21336b1f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.452040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34017ba955694555a73a6c2f21336b1f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.478156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34017ba955694555a73a6c2f21336b1f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.479874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34017ba955694555a73a6c2f21336b1f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:05.507990 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3439/6886] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:05 2022] DELETE /identity/v3/users/34017ba955694555a73a6c2f21336b1f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:05.513373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:05.516033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:05.534362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:05.721121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634f84-f9f4-408d-b848-ad6f275c6d1b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:05.786724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdffa358f833486693ac6fe9f6031c3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe29ae6d26b34e718d03a5e93fae962a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:05.787883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:05.788124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:05.788398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] PATH_INFO: `/v3/projects/bf74ce76042a4a18a093a32d0550fee2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:05.803856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bf74ce76042a4a18a093a32d0550fee2)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:05.818469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:05.994447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf74ce76042a4a18a093a32d0550fee2', 'request_id': 'req-61f420bb-922f-43ba-af93-0034cea5c038'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.014170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf74ce76042a4a18a093a32d0550fee2', 'request_id': 'req-61f420bb-922f-43ba-af93-0034cea5c038'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.035049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf74ce76042a4a18a093a32d0550fee2', 'request_id': 'req-61f420bb-922f-43ba-af93-0034cea5c038'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.054952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] The token cache is being invalidate because project bf74ce76042a4a18a093a32d0550fee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:06.055978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.057010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.061936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3448/6887] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:05 2022] DELETE /identity/v3/projects/bf74ce76042a4a18a093a32d0550fee2 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:06.067426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:06.083854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8a0c5-0fce-4655-9e1a-a9c0402fa7b4 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:06.247846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6203acb-7437-4c76-8959-327c4dfecfb6 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdffa358f833486693ac6fe9f6031c3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe29ae6d26b34e718d03a5e93fae962a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:06.248525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6203acb-7437-4c76-8959-327c4dfecfb6 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:06.248752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6203acb-7437-4c76-8959-327c4dfecfb6 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:06.249037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6203acb-7437-4c76-8959-327c4dfecfb6 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:06.249685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6203acb-7437-4c76-8959-327c4dfecfb6 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:06.251704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6203acb-7437-4c76-8959-327c4dfecfb6 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-d6203acb-7437-4c76-8959-327c4dfecfb6 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Could not find domain: b2f20fa836f2445b8d0e2079c0a9960f.: keystone.exception.DomainNotFound: Could not find domain: b2f20fa836f2445b8d0e2079c0a9960f. Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b2f20fa836f2445b8d0e2079c0a9960f. Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:16:06.285744 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Sep 10 21:16:06.287045 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:16:06.288059 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:16:06.289077 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:16:06.289077 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: b2f20fa836f2445b8d0e2079c0a9960f. Sep 10 21:16:06.289077 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:16:06.293497 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3440/6888] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:06 2022] POST /identity/v3/users => generated 112 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:16:06.295722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:06.297684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:06.444350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f420bb-922f-43ba-af93-0034cea5c038 tempest-UsersNegativeTest-763233529 tempest-UsersNegativeTest-763233529-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:06.457198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:06.460154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:06.460708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:06.460844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] PATH_INFO: `/v3/users/55301e893f144df190bfeba79bdbea31` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:06.463354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55301e893f144df190bfeba79bdbea31)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:06.465328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:06.636863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:06.648054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55301e893f144df190bfeba79bdbea31'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.659684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55301e893f144df190bfeba79bdbea31'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.661666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55301e893f144df190bfeba79bdbea31'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.697600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55301e893f144df190bfeba79bdbea31'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.698973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55301e893f144df190bfeba79bdbea31'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:06.735224 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3449/6889] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:06 2022] DELETE /identity/v3/users/55301e893f144df190bfeba79bdbea31 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:06.739606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15fde-f037-41f1-a438-f72eb9b639c3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:06.740089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15fde-f037-41f1-a438-f72eb9b639c3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:06.740520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15fde-f037-41f1-a438-f72eb9b639c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:06.775836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-25d15fde-f037-41f1-a438-f72eb9b639c3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:06.777125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-25d15fde-f037-41f1-a438-f72eb9b639c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:06.917780 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3441/6890] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:06.965599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:06.965599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:07.111156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dab93-412d-4391-9cfd-93bf12b00b33 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:07.124710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:07.125328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:07.125613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:07.126131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:07.157902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:07.159405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:07.196744 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3450/6891] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:16:08.055350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:08.056652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:08.056815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:08.112825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:08.116552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:08.222925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:08.269861 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3442/6892] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:08.284268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:08.290985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:08.414495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac859e0-8a86-4d20-b4da-b947bfb266b1 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:08.436147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:08.437021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:08.437468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:08.437885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:08.460716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:08.463324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:08.493157 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3451/6893] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:16:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:16:08.515023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:08.518208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:08.552878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14e3a-68d4-43fa-8767-6524caf47e0f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:08.568816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:08.569921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:08.570723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:08.574875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] PATH_INFO: `/v3/projects/f22dee49d0664c33a30034d1185a8e94` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:08.579375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22dee49d0664c33a30034d1185a8e94)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:08.585220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:08.716758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22dee49d0664c33a30034d1185a8e94', 'request_id': 'req-6af79211-c219-4e35-afa0-bd68a1b1a01e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.724004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22dee49d0664c33a30034d1185a8e94', 'request_id': 'req-6af79211-c219-4e35-afa0-bd68a1b1a01e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.732098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22dee49d0664c33a30034d1185a8e94', 'request_id': 'req-6af79211-c219-4e35-afa0-bd68a1b1a01e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.759770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] The token cache is being invalidate because project f22dee49d0664c33a30034d1185a8e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:08.760010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.760855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.765586 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3443/6894] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:08 2022] DELETE /identity/v3/projects/f22dee49d0664c33a30034d1185a8e94 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:08.767279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:08.783430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5dec4d-893f-4216-ac45-8cc9d7802dfe service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:08.844641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:08.864080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:08.865120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:08.865601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:08.866094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] PATH_INFO: `/v3/users/bdffa358f833486693ac6fe9f6031c3d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:08.867589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdffa358f833486693ac6fe9f6031c3d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:08.871952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:08.955313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdffa358f833486693ac6fe9f6031c3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.975606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdffa358f833486693ac6fe9f6031c3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.976681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdffa358f833486693ac6fe9f6031c3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.994070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdffa358f833486693ac6fe9f6031c3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:08.995018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdffa358f833486693ac6fe9f6031c3d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:09.034573 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3452/6895] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:08 2022] DELETE /identity/v3/users/bdffa358f833486693ac6fe9f6031c3d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:09.657772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:09.659646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:09.732866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af79211-c219-4e35-afa0-bd68a1b1a01e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:09.897237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:09.898012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:09.898331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:09.898617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] PATH_INFO: `/v3/projects/fe29ae6d26b34e718d03a5e93fae962a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:09.901091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe29ae6d26b34e718d03a5e93fae962a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:09.904213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:10.013501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe29ae6d26b34e718d03a5e93fae962a', 'request_id': 'req-f32207dd-52a3-4715-878b-a48377a91209'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:10.031785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe29ae6d26b34e718d03a5e93fae962a', 'request_id': 'req-f32207dd-52a3-4715-878b-a48377a91209'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:10.053509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe29ae6d26b34e718d03a5e93fae962a', 'request_id': 'req-f32207dd-52a3-4715-878b-a48377a91209'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:10.081398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] The token cache is being invalidate because project fe29ae6d26b34e718d03a5e93fae962a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:10.081699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:10.097446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f32207dd-52a3-4715-878b-a48377a91209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:10.106990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:10.107915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:10.125790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3444/6896] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:09 2022] DELETE /identity/v3/projects/fe29ae6d26b34e718d03a5e93fae962a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:10.128335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231fa2-039d-4f90-b71c-985d4ce6874f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:10.128635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231fa2-039d-4f90-b71c-985d4ce6874f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:10.128915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231fa2-039d-4f90-b71c-985d4ce6874f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:10.192393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-37231fa2-039d-4f90-b71c-985d4ce6874f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:10.195791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-37231fa2-039d-4f90-b71c-985d4ce6874f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:10.210486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:10.382445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172baa3-97df-420a-9a3e-f664361c95ef admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:10.408971 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3445/6897] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:10.417324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda6373-c620-4c4e-8503-8277bbaf050e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:10.417778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda6373-c620-4c4e-8503-8277bbaf050e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:10.418176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda6373-c620-4c4e-8503-8277bbaf050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:10.462903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:10.463799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dbda6373-c620-4c4e-8503-8277bbaf050e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:10.470838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dbda6373-c620-4c4e-8503-8277bbaf050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:10.486885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:10.487799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:10.488357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:10.488781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:10.491027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:10.491770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3446/6898] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:10.497569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda6373-c620-4c4e-8503-8277bbaf050e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:10.511107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:10.517684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:10.523414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda6373-c620-4c4e-8503-8277bbaf050e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:10.540556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:10.541450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:10.541869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:10.542252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:10.543014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:16:10.543473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:10.546592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:10.552531 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3453/6899] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:10.579924 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3447/6900] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:16:10.584259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:10.617174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0e4b0-0398-4544-a713-2e8a3dddf77f service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:10.624554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:10.628796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:10.630059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:10.630627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:10.631524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:10.633643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:10.704357 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3454/6901] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:10 2022] POST /identity/v3/projects => generated 399 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:10.705498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:10.722462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa1068-9867-4871-8324-96ddeb7b07c5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:10.868601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:10.870831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:10.872188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:10.876656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:10.878167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:10.882203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:10.933125 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3448/6902] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:10 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:10.937520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:10.951074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac64bb-a692-46bf-831a-54c1351c46e0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:10.956743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:10.957485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:10.957770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:10.958179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:10.959210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:10.960521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:10.990771 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3455/6903] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:16:10.995840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:11.012656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aee7e-d556-44eb-8a9c-21e262c56a0e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:11.031263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d863-9b21-4e1b-8ea5-b20ac0535f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:11.032467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d863-9b21-4e1b-8ea5-b20ac0535f0f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:11.034400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d863-9b21-4e1b-8ea5-b20ac0535f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:11.034915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d863-9b21-4e1b-8ea5-b20ac0535f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:11.035830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5d863-9b21-4e1b-8ea5-b20ac0535f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:11.040338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5d863-9b21-4e1b-8ea5-b20ac0535f0f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:11.062349 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3449/6904] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:11.066578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:11.083076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45156f21-3cbf-4fa3-8b64-eac786822ed4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:11.100426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:11.101036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:11.101815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:11.101815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] PATH_INFO: `/v3/projects/1e6b8b1c36154ce996c558c83d21fe24/users/6454dafe63724f20b3368df9eeb79c7b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:11.133127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6b8b1c36154ce996c558c83d21fe24, user_id=6454dafe63724f20b3368df9eeb79c7b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:11.137984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:11.166254 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3456/6905] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:11 2022] PUT /identity/v3/projects/1e6b8b1c36154ce996c558c83d21fe24/users/6454dafe63724f20b3368df9eeb79c7b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:11.176868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960b33-823e-4cd1-96db-4aa380df95c6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:11.177166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960b33-823e-4cd1-96db-4aa380df95c6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:11.177609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960b33-823e-4cd1-96db-4aa380df95c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:11.233133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-46960b33-823e-4cd1-96db-4aa380df95c6 None None] MFA Rules not processed for user `6454dafe63724f20b3368df9eeb79c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:11.234528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-46960b33-823e-4cd1-96db-4aa380df95c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:11.380068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:11.400279 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3450/6906] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:11.407759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:11.553680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f855-dbcc-467f-bc13-99e9b32a2b08 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:11.563313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:11.564103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:11.564383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:11.564731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:11.565775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:11.567619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:11.620485 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3457/6907] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:11 2022] POST /identity/v3/projects => generated 397 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:11.624378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960b33-823e-4cd1-96db-4aa380df95c6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:11.641323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960b33-823e-4cd1-96db-4aa380df95c6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:11.745088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:11.745863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:11.746245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:11.746754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:11.747475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:11.749609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:11.788009 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3451/6908] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:11 2022] POST /identity/v3/users => generated 305 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:11.790142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:11.809322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7894fb-2b77-498a-8421-37c2cba160d6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:11.828385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:11.829447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:11.830079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:11.830506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:11.831372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:11.833162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:11.863790 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3458/6909] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:11.868672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:11.892261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daf0d2-49bf-4814-9812-6609d5267cfd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:11.912338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:11.913331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:11.913767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:11.914278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] PATH_INFO: `/v3/projects/c191895b79a94ea9be0d26ff78b93ed6/users/6c1bbddb809040d68df2c5dc3bfaf537/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:11.954963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c191895b79a94ea9be0d26ff78b93ed6, user_id=6c1bbddb809040d68df2c5dc3bfaf537, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:11.958009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:11.985202 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3452/6910] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:11 2022] PUT /identity/v3/projects/c191895b79a94ea9be0d26ff78b93ed6/users/6c1bbddb809040d68df2c5dc3bfaf537/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:11.989234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:12.008907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308eaab-fe3a-4c87-a20f-2d99c34ce028 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:12.150272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56c5f8-6f0a-4d5c-a021-8141571b8aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:12.150980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56c5f8-6f0a-4d5c-a021-8141571b8aad admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:12.151270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56c5f8-6f0a-4d5c-a021-8141571b8aad admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:12.151550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56c5f8-6f0a-4d5c-a021-8141571b8aad admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:12.152199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56c5f8-6f0a-4d5c-a021-8141571b8aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:12.154257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56c5f8-6f0a-4d5c-a021-8141571b8aad admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:12.167837 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3459/6911] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:12.178750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:12.193335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854999a-d9eb-4fb0-a66a-ab1b64bae4c2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:12.199887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:12.201445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:12.202006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:12.203005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] PATH_INFO: `/v3/projects/c191895b79a94ea9be0d26ff78b93ed6/users/6c1bbddb809040d68df2c5dc3bfaf537/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:12.205401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c191895b79a94ea9be0d26ff78b93ed6, user_id=6c1bbddb809040d68df2c5dc3bfaf537, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:12.208132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:12.246186 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3453/6912] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:12 2022] PUT /identity/v3/projects/c191895b79a94ea9be0d26ff78b93ed6/users/6c1bbddb809040d68df2c5dc3bfaf537/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:12.247571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:12.247863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:12.248143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:12.266557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:12.354253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] MFA Rules not processed for user `6c1bbddb809040d68df2c5dc3bfaf537`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:12.356604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:12.549789 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3460/6913] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:16:12.561011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:12.563149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:12.675136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:12.677141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:12.728484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872422b9-308d-4daf-938c-7a6219f96582 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:12.734284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:12.736222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:12.736625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:12.737008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:12.893217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5b776-e16a-4681-be61-5ce5a25a434a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:12.902890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:12.904277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:12.915136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:12.917437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:12.918798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:12.919152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:12.924900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:12.946536 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3454/6914] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:13.082080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:13.083832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:13.107670 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3461/6915] 10.222.0.33 () {60 vars in 1396 bytes} [Sat Sep 10 21:16:12 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:16:13.800495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:13.801978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:13.818655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:13.990686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a695a64-98b2-45e9-8949-94e1816cd584 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:13.998701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:13.999338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:13.999628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:14.000040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:14.011224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:14.012865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:14.032430 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3455/6916] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:22.886517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:22.888216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:22.961369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:23.021609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38346a42-eb6a-4cb6-b96a-96735ac78d1a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:23.033630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef87976-f05b-478f-ba48-ef30d92ab4be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:23.034520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef87976-f05b-478f-ba48-ef30d92ab4be service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:23.034937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef87976-f05b-478f-ba48-ef30d92ab4be service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:23.035345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef87976-f05b-478f-ba48-ef30d92ab4be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:23.038407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ef87976-f05b-478f-ba48-ef30d92ab4be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:23.054931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef87976-f05b-478f-ba48-ef30d92ab4be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:23.057811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef87976-f05b-478f-ba48-ef30d92ab4be service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:23.077279 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3462/6917] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:23.092067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:23.206464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9fdd-61fc-4e4c-a96e-7a1353c6405e service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:23.215468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:23.226237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:23.226237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:23.226237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] PATH_INFO: `/v3/users/6454dafe63724f20b3368df9eeb79c7b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:23.229699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6454dafe63724f20b3368df9eeb79c7b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:23.233114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:23.299078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6454dafe63724f20b3368df9eeb79c7b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.317816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6454dafe63724f20b3368df9eeb79c7b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.325980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6454dafe63724f20b3368df9eeb79c7b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.335905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6454dafe63724f20b3368df9eeb79c7b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.337298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6454dafe63724f20b3368df9eeb79c7b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.344395 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3456/6918] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:23 2022] DELETE /identity/v3/users/6454dafe63724f20b3368df9eeb79c7b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:23.349549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:23.349975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:23.350347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:23.381289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:23.382621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:23.457969 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3463/6919] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:23.467523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:23.469047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:23.482665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:23.549593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44034522-7b32-4d34-aafb-ada20f416755 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:23.553284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:23.553916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:23.554569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:23.554569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:23.565497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:23.567021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:23.578310 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3457/6920] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:23.771885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:23.773449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:23.795415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9aead-fd34-42bd-be1f-65735bb90b86 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:23.800628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:23.801308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:23.801617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:23.801814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] PATH_INFO: `/v3/projects/1e6b8b1c36154ce996c558c83d21fe24` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:23.802956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6b8b1c36154ce996c558c83d21fe24)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:23.804865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:23.858932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6b8b1c36154ce996c558c83d21fe24', 'request_id': 'req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.865281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6b8b1c36154ce996c558c83d21fe24', 'request_id': 'req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.872962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6b8b1c36154ce996c558c83d21fe24', 'request_id': 'req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.883499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] The token cache is being invalidate because project 1e6b8b1c36154ce996c558c83d21fe24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:23.883972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.886331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:23.887733 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3464/6921] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:23 2022] DELETE /identity/v3/projects/1e6b8b1c36154ce996c558c83d21fe24 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:23.891844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:23.904184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4bdfb-76f9-4413-b91c-44d68da84e41 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:23.967374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:23.968394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:23.970885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:23.971186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] PATH_INFO: `/v3/users/6c1bbddb809040d68df2c5dc3bfaf537` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:23.972377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1bbddb809040d68df2c5dc3bfaf537)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:23.974354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:24.034958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1bbddb809040d68df2c5dc3bfaf537'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.049325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1bbddb809040d68df2c5dc3bfaf537'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.056934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1bbddb809040d68df2c5dc3bfaf537'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.064511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1bbddb809040d68df2c5dc3bfaf537'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.065316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26fa4a62-1c50-48e5-abc3-e46f4eaee76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1bbddb809040d68df2c5dc3bfaf537'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.071801 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3458/6922] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:23 2022] DELETE /identity/v3/users/6c1bbddb809040d68df2c5dc3bfaf537 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:24.351180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:24.354040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:24.374047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f116a-1b34-495c-8f0c-a3a6ae3218e6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:24.448333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:24.456382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:24.457413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.458043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.458817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] PATH_INFO: `/v3/projects/c191895b79a94ea9be0d26ff78b93ed6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.460121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c191895b79a94ea9be0d26ff78b93ed6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:24.462789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:24.513747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c191895b79a94ea9be0d26ff78b93ed6', 'request_id': 'req-d7fd7706-67d4-4768-a274-a1e5da8d53e9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.520292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c191895b79a94ea9be0d26ff78b93ed6', 'request_id': 'req-d7fd7706-67d4-4768-a274-a1e5da8d53e9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.529385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c191895b79a94ea9be0d26ff78b93ed6', 'request_id': 'req-d7fd7706-67d4-4768-a274-a1e5da8d53e9'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.542765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] The token cache is being invalidate because project c191895b79a94ea9be0d26ff78b93ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:24.543060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.543791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d7fd7706-67d4-4768-a274-a1e5da8d53e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:24.546546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3465/6923] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:24 2022] DELETE /identity/v3/projects/c191895b79a94ea9be0d26ff78b93ed6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:24.551664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63212-9560-43ec-b7f3-43d85095b218 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.552183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63212-9560-43ec-b7f3-43d85095b218 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.552277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63212-9560-43ec-b7f3-43d85095b218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.572899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-eca63212-9560-43ec-b7f3-43d85095b218 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:24.574053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eca63212-9560-43ec-b7f3-43d85095b218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:24.601144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:24.639484 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3459/6924] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:24.647269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263973-6f71-468c-b044-bc45292718df None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.647589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263973-6f71-468c-b044-bc45292718df None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.647866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263973-6f71-468c-b044-bc45292718df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.669250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6f263973-6f71-468c-b044-bc45292718df None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:24.670495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6f263973-6f71-468c-b044-bc45292718df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:24.679034 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3466/6925] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:24.683799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca63212-9560-43ec-b7f3-43d85095b218 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:24.693982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca63212-9560-43ec-b7f3-43d85095b218 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:24.699378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:24.699994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.700274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.700550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.701205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:16:24.701513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:24.703144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:24.725902 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3460/6926] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:16:24.728565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263973-6f71-468c-b044-bc45292718df None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:24.738716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263973-6f71-468c-b044-bc45292718df None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:24.743635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:24.744244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.744530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.744949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.745587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:24.747557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:24.779085 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3467/6927] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:24 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:24.780467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:24.791551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62bfde-84de-4615-8564-e2141fbb3f0a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:24.852763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:24.853379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.853688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.853998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.854629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:24.856948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:24.884623 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3461/6928] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:24 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:24.886075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:24.896731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833aaff-b032-40ba-8390-09bddf4d841b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:24.901721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:24.902361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.902640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.902910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.903563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:24.905174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:24.916229 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3468/6929] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:24.920687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:24.934128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7fc1e-b8b4-4262-a029-3cdd256b8669 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:24.939132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fef30-2558-4eaa-befa-d63ae7e5aae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:24.939868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fef30-2558-4eaa-befa-d63ae7e5aae7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.940159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fef30-2558-4eaa-befa-d63ae7e5aae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.940452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fef30-2558-4eaa-befa-d63ae7e5aae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.941084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fef30-2558-4eaa-befa-d63ae7e5aae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:24.942719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fef30-2558-4eaa-befa-d63ae7e5aae7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:24.954034 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3462/6930] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:16:24.958530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:24.970617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b211a7-f6e6-470e-aea0-38996fc89cf8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:24.975834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410244f-823c-4f7b-a06e-28d2a13dd0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:24.976571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410244f-823c-4f7b-a06e-28d2a13dd0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:24.977524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410244f-823c-4f7b-a06e-28d2a13dd0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:24.977798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410244f-823c-4f7b-a06e-28d2a13dd0f8 admin admin] PATH_INFO: `/v3/projects/88b99f7f58154e4d9ecee50ef8740cdd/users/de803cb7dded4de9aa74352e8fe33992/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:24.994209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410244f-823c-4f7b-a06e-28d2a13dd0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b99f7f58154e4d9ecee50ef8740cdd, user_id=de803cb7dded4de9aa74352e8fe33992, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:24.996870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410244f-823c-4f7b-a06e-28d2a13dd0f8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.009666 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3469/6931] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:24 2022] PUT /identity/v3/projects/88b99f7f58154e4d9ecee50ef8740cdd/users/de803cb7dded4de9aa74352e8fe33992/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:25.020454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed762ca-82e7-40b9-b341-df6cf7a8b022 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.020741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed762ca-82e7-40b9-b341-df6cf7a8b022 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.021046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed762ca-82e7-40b9-b341-df6cf7a8b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.052936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8ed762ca-82e7-40b9-b341-df6cf7a8b022 None None] MFA Rules not processed for user `de803cb7dded4de9aa74352e8fe33992`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:25.054175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8ed762ca-82e7-40b9-b341-df6cf7a8b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:25.072089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.072716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.072998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.094826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:25.096100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:25.127089 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3463/6932] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:16:25.131805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed762ca-82e7-40b9-b341-df6cf7a8b022 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:25.162683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:25.179773 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3470/6933] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:25.192871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:25.195039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:25.208643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed762ca-82e7-40b9-b341-df6cf7a8b022 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:25.214008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:25.214617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.215255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.215538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.216430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:25.218856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.230147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:25.259019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3464/6934] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:25 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:25.262957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:25.285257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfcaaf-d968-4964-b076-2ca75363a565 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:25.327152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ef219-d075-4fd4-8283-602b3571f6f1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:25.331850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:25.332462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.332745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.333018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.333904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:25.335575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.363854 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3465/6935] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:25 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:25.364546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:25.365674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.366124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.366276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.368515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:25.379457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47229400-1b78-45f2-a0e0-39f2221f1afa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:25.392713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:25.394011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.394290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.394562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.395235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:25.396810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.411318 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3466/6936] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:25.412989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:25.426443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983d1f7-720e-47f3-92f4-87beb7d524b4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:25.429245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:25.431701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.433390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:25.434280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.434443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.434826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] PATH_INFO: `/v3/projects/6fa7d4783e8d4e67b5993ae6dfbec33c/users/d4ab2754f79741c2bb13ddcea7ab98ba/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.448534 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3471/6937] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:16:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 21:16:25.455934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa7d4783e8d4e67b5993ae6dfbec33c, user_id=d4ab2754f79741c2bb13ddcea7ab98ba, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:25.458569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.487165 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3467/6938] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:25 2022] PUT /identity/v3/projects/6fa7d4783e8d4e67b5993ae6dfbec33c/users/d4ab2754f79741c2bb13ddcea7ab98ba/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:16:25.491108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:25.522772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d747852-f7ae-44ae-b62d-75bc0b8caa57 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:25.736838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e359ec-57cc-40e6-8837-08d0899a5b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:25.737508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e359ec-57cc-40e6-8837-08d0899a5b52 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.738076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e359ec-57cc-40e6-8837-08d0899a5b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.738389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e359ec-57cc-40e6-8837-08d0899a5b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.739115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e359ec-57cc-40e6-8837-08d0899a5b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:25.741543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e359ec-57cc-40e6-8837-08d0899a5b52 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.782209 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3472/6939] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:25.791031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:25.809669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043601e-301f-49bc-8170-6615fd98a8ba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:25.820785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:25.821471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.821750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.822060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] PATH_INFO: `/v3/projects/6fa7d4783e8d4e67b5993ae6dfbec33c/users/d4ab2754f79741c2bb13ddcea7ab98ba/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.832816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa7d4783e8d4e67b5993ae6dfbec33c, user_id=d4ab2754f79741c2bb13ddcea7ab98ba, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:25.835525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:25.859091 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3468/6940] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:25 2022] PUT /identity/v3/projects/6fa7d4783e8d4e67b5993ae6dfbec33c/users/d4ab2754f79741c2bb13ddcea7ab98ba/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:25.872852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21042165-8590-42fd-b6d3-3a230e316104 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:25.873064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21042165-8590-42fd-b6d3-3a230e316104 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:25.873329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21042165-8590-42fd-b6d3-3a230e316104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:25.895259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:25.928756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-21042165-8590-42fd-b6d3-3a230e316104 None None] MFA Rules not processed for user `d4ab2754f79741c2bb13ddcea7ab98ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:25.930280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21042165-8590-42fd-b6d3-3a230e316104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:26.094618 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3473/6941] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:26.106245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:26.108087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:26.218568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9fbeb-88e7-4bcd-91db-3d5a9b5613bb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:26.222733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03626ec-63f4-4f0a-9adb-bd78873c3975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:26.223546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03626ec-63f4-4f0a-9adb-bd78873c3975 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:26.223958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03626ec-63f4-4f0a-9adb-bd78873c3975 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:26.224320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03626ec-63f4-4f0a-9adb-bd78873c3975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:26.233779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:26.237233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03626ec-63f4-4f0a-9adb-bd78873c3975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:26.239543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03626ec-63f4-4f0a-9adb-bd78873c3975 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:26.281915 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3469/6942] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:16:30.348858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21042165-8590-42fd-b6d3-3a230e316104 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:30.426788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21042165-8590-42fd-b6d3-3a230e316104 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:30.432356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:30.432994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:30.433275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:30.433551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] PATH_INFO: `/v3/users/de803cb7dded4de9aa74352e8fe33992` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:30.434911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=de803cb7dded4de9aa74352e8fe33992)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:30.437040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:30.487336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de803cb7dded4de9aa74352e8fe33992'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:30.496304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de803cb7dded4de9aa74352e8fe33992'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:30.497096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de803cb7dded4de9aa74352e8fe33992'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:30.506749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de803cb7dded4de9aa74352e8fe33992'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:30.507392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de803cb7dded4de9aa74352e8fe33992'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:30.533038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3474/6943] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:30 2022] DELETE /identity/v3/users/de803cb7dded4de9aa74352e8fe33992 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:30.537681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:30.538366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:30.538670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:30.559003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:30.560233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:30.669107 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3470/6944] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:30.679040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:30.681228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:30.716453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:30.808718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091644f4-a79a-4fde-80cb-636be9fbab8a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:30.812754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:30.813459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:30.813742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:30.814070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:30.826530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:30.828596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:30.856133 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3475/6945] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:31.261689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:31.263520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:31.290497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fedb4b-b030-4fc3-a655-527e251c266d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:31.295765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:31.296537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:31.296921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:31.297297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] PATH_INFO: `/v3/projects/88b99f7f58154e4d9ecee50ef8740cdd` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:31.298580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b99f7f58154e4d9ecee50ef8740cdd)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:31.300613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:31.376469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b99f7f58154e4d9ecee50ef8740cdd', 'request_id': 'req-2003616a-4147-40de-9044-b068d4ead74b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.399367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b99f7f58154e4d9ecee50ef8740cdd', 'request_id': 'req-2003616a-4147-40de-9044-b068d4ead74b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.406591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b99f7f58154e4d9ecee50ef8740cdd', 'request_id': 'req-2003616a-4147-40de-9044-b068d4ead74b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.424213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] The token cache is being invalidate because project 88b99f7f58154e4d9ecee50ef8740cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:31.424608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.425462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.428633 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3471/6946] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:31 2022] DELETE /identity/v3/projects/88b99f7f58154e4d9ecee50ef8740cdd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:31.433957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:31.445053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447f69-a2f3-4d39-bcaa-908fa31200b0 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:31.518816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:31.519472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:31.519754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:31.520032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] PATH_INFO: `/v3/users/d4ab2754f79741c2bb13ddcea7ab98ba` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:31.521359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ab2754f79741c2bb13ddcea7ab98ba)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:31.523373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:31.575069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ab2754f79741c2bb13ddcea7ab98ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.584291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ab2754f79741c2bb13ddcea7ab98ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.585021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ab2754f79741c2bb13ddcea7ab98ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.593996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ab2754f79741c2bb13ddcea7ab98ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.594641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-896ce447-c930-4aad-890f-64a5d0a559c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ab2754f79741c2bb13ddcea7ab98ba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:31.616199 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3476/6947] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:31 2022] DELETE /identity/v3/users/d4ab2754f79741c2bb13ddcea7ab98ba => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:31.809666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:31.811231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:31.826599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003616a-4147-40de-9044-b068d4ead74b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:31.909918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:31.910845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:31.911241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:31.911622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] PATH_INFO: `/v3/projects/6fa7d4783e8d4e67b5993ae6dfbec33c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:31.913043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa7d4783e8d4e67b5993ae6dfbec33c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:31.915732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:31.949966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:32.028884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa7d4783e8d4e67b5993ae6dfbec33c', 'request_id': 'req-3451c798-e336-4edb-b2a2-69e532c0c442'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:32.043847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa7d4783e8d4e67b5993ae6dfbec33c', 'request_id': 'req-3451c798-e336-4edb-b2a2-69e532c0c442'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:32.056463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa7d4783e8d4e67b5993ae6dfbec33c', 'request_id': 'req-3451c798-e336-4edb-b2a2-69e532c0c442'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:32.076896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] The token cache is being invalidate because project 6fa7d4783e8d4e67b5993ae6dfbec33c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:32.077979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:32.079071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3451c798-e336-4edb-b2a2-69e532c0c442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:32.086154 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3472/6948] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:31 2022] DELETE /identity/v3/projects/6fa7d4783e8d4e67b5993ae6dfbec33c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:32.087509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41570-7f01-4ee3-a9bf-9e8dbccbf8d2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.087991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41570-7f01-4ee3-a9bf-9e8dbccbf8d2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.088274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41570-7f01-4ee3-a9bf-9e8dbccbf8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.163432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-99e41570-7f01-4ee3-a9bf-9e8dbccbf8d2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:32.165045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-99e41570-7f01-4ee3-a9bf-9e8dbccbf8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:32.182725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:32.250132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3477/6949] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:32.257904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b48b8-163e-466d-913e-cb95d04dc48e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.258238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b48b8-163e-466d-913e-cb95d04dc48e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.258517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b48b8-163e-466d-913e-cb95d04dc48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.292781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-355b48b8-163e-466d-913e-cb95d04dc48e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:32.294051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-355b48b8-163e-466d-913e-cb95d04dc48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:32.303108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3473/6950] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:32.307615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41570-7f01-4ee3-a9bf-9e8dbccbf8d2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:32.317908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41570-7f01-4ee3-a9bf-9e8dbccbf8d2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:32.323333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:32.324085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.324476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.324853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.325941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:16:32.326359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:32.328385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:32.343698 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3478/6951] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:16:32.348109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b48b8-163e-466d-913e-cb95d04dc48e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:32.360496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b48b8-163e-466d-913e-cb95d04dc48e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:32.369712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:32.370451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.370805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.371153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.371859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:32.373939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:32.444771 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3474/6952] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:32 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:32.446078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:32.468594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6cd66-61ba-43ef-948e-480808000964 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:32.551063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:32.551766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.552122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.552486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.553343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:32.555708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:32.589876 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3479/6953] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:32 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:32.592269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:32.606206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb894ea-fd1c-4558-9edf-6f87c3750211 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:32.611857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:32.612818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.613196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.613508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.614362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:32.618459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:32.632771 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3475/6954] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:32.647906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:32.660178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8e34f-79a5-4436-beb4-3550bf82e5ee admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:32.665948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccac61-d11e-4f46-a74a-64f111f7c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:32.666698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccac61-d11e-4f46-a74a-64f111f7c762 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.667039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccac61-d11e-4f46-a74a-64f111f7c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.667378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccac61-d11e-4f46-a74a-64f111f7c762 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.668143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccac61-d11e-4f46-a74a-64f111f7c762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:32.669923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccac61-d11e-4f46-a74a-64f111f7c762 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:32.684211 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3480/6955] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:32.689389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:32.702880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0757f5-6174-45ca-9eb9-1e2ee46b36ed admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:32.707737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:32.708409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.708696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.708983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] PATH_INFO: `/v3/projects/f71d833e17a4480888dc835bf42d484c/users/95d42077dc824a6ea4955af4a65c8601/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.727491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71d833e17a4480888dc835bf42d484c, user_id=95d42077dc824a6ea4955af4a65c8601, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:32.730221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:32.757896 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3476/6956] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:32 2022] PUT /identity/v3/projects/f71d833e17a4480888dc835bf42d484c/users/95d42077dc824a6ea4955af4a65c8601/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:32.763298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.763701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.764096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:32.795958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] MFA Rules not processed for user `95d42077dc824a6ea4955af4a65c8601`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:32.797340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:32.890306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3481/6957] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:32.901770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:32.903540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:32.987793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb6d2-14e4-41d6-a41a-732a17c61cad admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:32.994904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3719f-6ca7-4b3f-aa76-66b33e62a82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:32.995540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3719f-6ca7-4b3f-aa76-66b33e62a82c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:32.995823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3719f-6ca7-4b3f-aa76-66b33e62a82c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:32.996102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3719f-6ca7-4b3f-aa76-66b33e62a82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:33.006217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3719f-6ca7-4b3f-aa76-66b33e62a82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:33.007657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3719f-6ca7-4b3f-aa76-66b33e62a82c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:33.023021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3477/6958] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:34.624915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:34.627638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:34.656151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:34.721283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059af8-56f1-4c4f-b718-bac4ba23306c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:34.727214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb327ea-29f1-480d-aeec-c6b458d9f139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:34.727852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb327ea-29f1-480d-aeec-c6b458d9f139 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:34.728118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb327ea-29f1-480d-aeec-c6b458d9f139 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:34.728392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb327ea-29f1-480d-aeec-c6b458d9f139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:34.730654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-feb327ea-29f1-480d-aeec-c6b458d9f139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:34.746841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb327ea-29f1-480d-aeec-c6b458d9f139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:34.749194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb327ea-29f1-480d-aeec-c6b458d9f139 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:34.764739 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3482/6959] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:36.215119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:36.215674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:36.215674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:36.236676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] MFA Rules not processed for user `eac0a8d66da042e096cab5c715f40a27`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:36.237914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:36.282661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:36.296753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3478/6960] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:41.546141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374d74d-cb02-433c-98c8-71a716f5edfd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:41.546634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374d74d-cb02-433c-98c8-71a716f5edfd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:41.546717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374d74d-cb02-433c-98c8-71a716f5edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:41.577211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6374d74d-cb02-433c-98c8-71a716f5edfd None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:41.578537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6374d74d-cb02-433c-98c8-71a716f5edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:41.656682 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3483/6961] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:41.670202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:41.671654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:41.764800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d77f-eeba-4062-a1fa-1f6d3676ea6d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:41.773094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:41.774641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:41.775082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:41.775360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:41.786640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:41.788273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:41.798826 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3479/6962] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:16:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:16:51.648347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=54 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:51.794931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=35 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:52.656648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc8252-5df0-4069-ac61-616ac913531c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:52.657121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc8252-5df0-4069-ac61-616ac913531c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:52.657337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc8252-5df0-4069-ac61-616ac913531c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:52.677425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-53cc8252-5df0-4069-ac61-616ac913531c None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:52.678693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-53cc8252-5df0-4069-ac61-616ac913531c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:52.686738 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3484/6963] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:52.701244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:52.714857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467fb57-b46c-4699-96e9-adf2fe548c0f service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:52.720014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096f36e-2db5-4c2e-8727-6825cec5bebe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:52.720618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096f36e-2db5-4c2e-8727-6825cec5bebe service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:52.720905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096f36e-2db5-4c2e-8727-6825cec5bebe service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:52.721186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096f36e-2db5-4c2e-8727-6825cec5bebe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:52.731105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096f36e-2db5-4c2e-8727-6825cec5bebe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:52.733247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096f36e-2db5-4c2e-8727-6825cec5bebe service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:52.741662 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3480/6964] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:16:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:16:52.798160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc8252-5df0-4069-ac61-616ac913531c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:52.892657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc8252-5df0-4069-ac61-616ac913531c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:52.898133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:52.898950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:52.899301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:52.899646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] PATH_INFO: `/v3/users/95d42077dc824a6ea4955af4a65c8601` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:52.900996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d42077dc824a6ea4955af4a65c8601)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:52.903211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:52.958611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d42077dc824a6ea4955af4a65c8601'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:52.971105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d42077dc824a6ea4955af4a65c8601'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:52.974307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d42077dc824a6ea4955af4a65c8601'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:52.984997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d42077dc824a6ea4955af4a65c8601'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:52.985818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d42077dc824a6ea4955af4a65c8601'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:53.015872 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3485/6965] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:52 2022] DELETE /identity/v3/users/95d42077dc824a6ea4955af4a65c8601 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:53.020374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.020881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.020979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.042609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:53.043912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:53.109767 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3481/6966] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:53.121781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:53.123220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:53.189472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eb775-9e1d-4862-bb77-269705c01ecd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:53.193388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c6d76-6c75-4415-86fe-95262838af92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:53.194038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6d76-6c75-4415-86fe-95262838af92 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.194322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6d76-6c75-4415-86fe-95262838af92 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.194604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c6d76-6c75-4415-86fe-95262838af92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.206064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6d76-6c75-4415-86fe-95262838af92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:53.209992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c6d76-6c75-4415-86fe-95262838af92 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:53.220955 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3486/6967] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:16:53.443944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:53.446503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:53.463999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b19a08-e01f-4e82-9b15-dadb102a1557 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:53.469379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:53.470229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.470615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.470978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] PATH_INFO: `/v3/projects/f71d833e17a4480888dc835bf42d484c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.472302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71d833e17a4480888dc835bf42d484c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:53.475487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:53.530187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71d833e17a4480888dc835bf42d484c', 'request_id': 'req-548c74f4-8e6b-43ac-a808-ebd85753ba9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:53.536263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71d833e17a4480888dc835bf42d484c', 'request_id': 'req-548c74f4-8e6b-43ac-a808-ebd85753ba9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:53.543924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71d833e17a4480888dc835bf42d484c', 'request_id': 'req-548c74f4-8e6b-43ac-a808-ebd85753ba9a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:53.557987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] The token cache is being invalidate because project f71d833e17a4480888dc835bf42d484c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:53.558282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:53.558939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-548c74f4-8e6b-43ac-a808-ebd85753ba9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:53.560664 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3482/6968] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:53 2022] DELETE /identity/v3/projects/f71d833e17a4480888dc835bf42d484c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:53.565663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ed577-b35b-4681-b8b1-5a56bba620c8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.565991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ed577-b35b-4681-b8b1-5a56bba620c8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.566272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ed577-b35b-4681-b8b1-5a56bba620c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.588691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c39ed577-b35b-4681-b8b1-5a56bba620c8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:53.590087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c39ed577-b35b-4681-b8b1-5a56bba620c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:53.600995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:53.678352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3487/6969] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:16:53.686624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1225442-4a24-45de-b46a-500401fc9f53 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.686915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1225442-4a24-45de-b46a-500401fc9f53 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.687197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1225442-4a24-45de-b46a-500401fc9f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.707953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d1225442-4a24-45de-b46a-500401fc9f53 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:53.709188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d1225442-4a24-45de-b46a-500401fc9f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:53.720317 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3483/6970] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:53.724406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ed577-b35b-4681-b8b1-5a56bba620c8 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:53.736558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ed577-b35b-4681-b8b1-5a56bba620c8 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:53.741646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:53.743019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.743276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.743560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.744510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:16:53.744664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:53.747599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:53.764066 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3488/6971] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:16:53.765560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1225442-4a24-45de-b46a-500401fc9f53 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:53.778096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1225442-4a24-45de-b46a-500401fc9f53 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:53.783914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:53.784662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.785083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.785590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.786334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:53.788485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:53.819503 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3484/6972] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:53 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:53.823830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:53.834901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc96dda-0835-4d64-b638-e6c7bda0ea80 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:53.905018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:53.906438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:53.906917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:53.907810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:53.908485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:53.916662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:53.951570 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3489/6973] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:53 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:53.956292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:53.989565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0069a-6872-4c77-8313-4f7b487ce13a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:54.005495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:54.006420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.006856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.008252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.009778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:54.017180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:54.047669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:54.049446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:54.054864 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3485/6974] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:54.059334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:54.072428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951301b7-4736-4741-8db0-2222da23e7e3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:54.078474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:54.079320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.079890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.080316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.081327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:54.083113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:54.098870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3486/6975] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:54.106717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:54.118580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af34db-5c6a-49b0-9371-c882d7b44bb0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:54.138794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c767d-b6f5-4c32-978a-bd2f6bfff25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:54.139490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c767d-b6f5-4c32-978a-bd2f6bfff25d admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.139786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c767d-b6f5-4c32-978a-bd2f6bfff25d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.140070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c767d-b6f5-4c32-978a-bd2f6bfff25d admin admin] PATH_INFO: `/v3/projects/91651327efe24c6a86b1c796fd8baaaf/users/8b0b38c2f8f54339a92f43ae8a26aeba/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.170959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c767d-b6f5-4c32-978a-bd2f6bfff25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91651327efe24c6a86b1c796fd8baaaf, user_id=8b0b38c2f8f54339a92f43ae8a26aeba, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:54.174305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c767d-b6f5-4c32-978a-bd2f6bfff25d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:54.177852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:54.200595 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3487/6976] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:54 2022] PUT /identity/v3/projects/91651327efe24c6a86b1c796fd8baaaf/users/8b0b38c2f8f54339a92f43ae8a26aeba/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:54.210532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.211340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.211689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.227986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c5c07-c206-42df-87ca-0e899ff11747 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:54.247494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] MFA Rules not processed for user `8b0b38c2f8f54339a92f43ae8a26aeba`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:54.248807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:54.267481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:54.325986 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3488/6977] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:54.327275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:54.327895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.328180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.328486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.330266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:54.338741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:54.340360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:54.345571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:54.347374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:54.361332 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3490/6978] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:16:54.406733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516c16-8e56-4a55-bc00-d83bce59aee9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:54.411723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32675eed-f0eb-4377-afff-2311cc0ab5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:54.411723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32675eed-f0eb-4377-afff-2311cc0ab5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.411723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32675eed-f0eb-4377-afff-2311cc0ab5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.412230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32675eed-f0eb-4377-afff-2311cc0ab5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.422255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32675eed-f0eb-4377-afff-2311cc0ab5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:54.423726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32675eed-f0eb-4377-afff-2311cc0ab5a8 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:54.433043 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3489/6979] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:16:54.754351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:54.819855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1826bf9-2366-46a6-832f-1786e303e5f4 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:54.824825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:54.825520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.825895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.826237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] PATH_INFO: `/v3/users/8b0b38c2f8f54339a92f43ae8a26aeba` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.827405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0b38c2f8f54339a92f43ae8a26aeba)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:54.829391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:54.877872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0b38c2f8f54339a92f43ae8a26aeba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:54.885412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0b38c2f8f54339a92f43ae8a26aeba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:54.886156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0b38c2f8f54339a92f43ae8a26aeba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:54.893357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0b38c2f8f54339a92f43ae8a26aeba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:54.894206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0b38c2f8f54339a92f43ae8a26aeba'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:54.911994 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3491/6980] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:16:54 2022] DELETE /identity/v3/users/8b0b38c2f8f54339a92f43ae8a26aeba => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:16:54.916467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:54.916756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:54.917060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:54.939223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:54.940424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:55.006722 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3490/6981] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:55.017034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.018481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:55.065674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:55.080714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fa0fe-213d-4af4-98d2-7779328121bc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.084435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca8608-fa35-4938-9e38-ccdd8ac46702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.085212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca8608-fa35-4938-9e38-ccdd8ac46702 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.086118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca8608-fa35-4938-9e38-ccdd8ac46702 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.086398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca8608-fa35-4938-9e38-ccdd8ac46702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.095587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca8608-fa35-4938-9e38-ccdd8ac46702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.097019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca8608-fa35-4938-9e38-ccdd8ac46702 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.106779 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3492/6982] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:16:55.414907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.416365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:55.428303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d1e1-1736-49a0-9939-353fa2384f81 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.433527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.434436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.434583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.435022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] PATH_INFO: `/v3/projects/91651327efe24c6a86b1c796fd8baaaf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.436497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91651327efe24c6a86b1c796fd8baaaf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.438469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.466637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:55.490735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91651327efe24c6a86b1c796fd8baaaf', 'request_id': 'req-ea6c3575-73df-405a-8fa5-bb472fba885e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:55.497178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91651327efe24c6a86b1c796fd8baaaf', 'request_id': 'req-ea6c3575-73df-405a-8fa5-bb472fba885e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:55.504005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91651327efe24c6a86b1c796fd8baaaf', 'request_id': 'req-ea6c3575-73df-405a-8fa5-bb472fba885e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:55.517609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] The token cache is being invalidate because project 91651327efe24c6a86b1c796fd8baaaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:16:55.517941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:55.518596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ea6c3575-73df-405a-8fa5-bb472fba885e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:16:55.520075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3491/6983] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:16:55 2022] DELETE /identity/v3/projects/91651327efe24c6a86b1c796fd8baaaf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:16:55.525583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d5e97-c9f5-4fa6-8135-1898d504a766 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.525939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d5e97-c9f5-4fa6-8135-1898d504a766 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.526226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d5e97-c9f5-4fa6-8135-1898d504a766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.546523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e76d5e97-c9f5-4fa6-8135-1898d504a766 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:55.547864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e76d5e97-c9f5-4fa6-8135-1898d504a766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:55.609744 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3493/6984] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:55.616976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3206e-d3d9-4da1-9512-082e71a55fbc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.617419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3206e-d3d9-4da1-9512-082e71a55fbc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.617702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3206e-d3d9-4da1-9512-082e71a55fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.638280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f2f3206e-d3d9-4da1-9512-082e71a55fbc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:55.639498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f2f3206e-d3d9-4da1-9512-082e71a55fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:55.647629 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3492/6985] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:55.652384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d5e97-c9f5-4fa6-8135-1898d504a766 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.662436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d5e97-c9f5-4fa6-8135-1898d504a766 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.667461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.668071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.668357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.668636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.669314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:16:55.669703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.671275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.683730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3494/6986] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:16:55.688212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3206e-d3d9-4da1-9512-082e71a55fbc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.698661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3206e-d3d9-4da1-9512-082e71a55fbc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.703441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.704054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.704336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.704955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.705235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.707205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.736359 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3493/6987] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:16:55 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:16:55.740423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.750707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe2727-106c-44ea-834d-af43e7c2c8e8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.808268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.808894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.809145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.809422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.810082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.812044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.833343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3495/6988] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:16:55 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:16:55.837412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.847706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ead9-1500-456d-b576-abc032213596 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.853234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.853856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.854144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.854415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.855080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.857134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.868307 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3494/6989] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:55.872527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.884383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d048-1726-41e4-9b89-6960416615cb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.890269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0af0f-5257-452e-b613-be69ae707075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.891023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0af0f-5257-452e-b613-be69ae707075 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.891411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0af0f-5257-452e-b613-be69ae707075 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.891751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0af0f-5257-452e-b613-be69ae707075 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.892493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0af0f-5257-452e-b613-be69ae707075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.894281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0af0f-5257-452e-b613-be69ae707075 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.905526 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3496/6990] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:16:55.909943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:55.926726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bf159-2230-498f-9add-4d6cb0b25e22 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:55.932892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:55.933201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.933481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.933761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] PATH_INFO: `/v3/projects/6877cf3def324994bf7a60560e997ebb/users/8d35af81a56f4854a8b5b7a888d33c8a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:55.949902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6877cf3def324994bf7a60560e997ebb, user_id=8d35af81a56f4854a8b5b7a888d33c8a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:55.952530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:55.972231 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3495/6991] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:16:55 2022] PUT /identity/v3/projects/6877cf3def324994bf7a60560e997ebb/users/8d35af81a56f4854a8b5b7a888d33c8a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:16:55.984239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:55.984527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:55.984804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:56.013468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] MFA Rules not processed for user `8d35af81a56f4854a8b5b7a888d33c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:16:56.014730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:56.058910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:16:56.083102 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3497/6992] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:16:56.093517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:16:56.095235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:16:56.160829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992208b9-198d-4ffc-b1d0-53c84edd92d8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:16:56.165336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:16:56.166055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:16:56.166254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:16:56.166530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:16:56.177790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:16:56.179459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:16:56.190174 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3496/6993] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:06.080378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:06.185716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:11.514290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:11.515820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:11.535616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d3727-334f-4e0d-afbb-4738cbae7a43 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:11.540504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3eb69-a321-4d22-94a1-860033a46823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:11.541456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3eb69-a321-4d22-94a1-860033a46823 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:11.541874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3eb69-a321-4d22-94a1-860033a46823 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:11.542257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3eb69-a321-4d22-94a1-860033a46823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:11.544182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-66a3eb69-a321-4d22-94a1-860033a46823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:11.625707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3eb69-a321-4d22-94a1-860033a46823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:11.627500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3eb69-a321-4d22-94a1-860033a46823 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:11.640860 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3498/6994] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:12.103608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:12.117818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f004faa-5c7c-4a91-97f3-73e03623e3fa service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:12.123928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:12.124642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:12.124993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:12.125340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] PATH_INFO: `/v3/users/8d35af81a56f4854a8b5b7a888d33c8a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:12.126606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d35af81a56f4854a8b5b7a888d33c8a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:12.128571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:12.205345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d35af81a56f4854a8b5b7a888d33c8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.218931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d35af81a56f4854a8b5b7a888d33c8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.226364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d35af81a56f4854a8b5b7a888d33c8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.234603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d35af81a56f4854a8b5b7a888d33c8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.235227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d35af81a56f4854a8b5b7a888d33c8a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.241930 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3497/6995] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:12 2022] DELETE /identity/v3/users/8d35af81a56f4854a8b5b7a888d33c8a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:12.246480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:12.246766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:12.247041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:12.266978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:12.268183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:12.333978 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3499/6996] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:12.342250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:12.343924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:12.409226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe74672-ae69-4582-9f99-b8814fccacba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:12.413184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85cf9b-1eb5-4745-991a-9a92460e9f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:12.413803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85cf9b-1eb5-4745-991a-9a92460e9f06 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:12.414121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85cf9b-1eb5-4745-991a-9a92460e9f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:12.414402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85cf9b-1eb5-4745-991a-9a92460e9f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:12.423925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85cf9b-1eb5-4745-991a-9a92460e9f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:12.425355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85cf9b-1eb5-4745-991a-9a92460e9f06 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:12.435698 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3498/6997] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:12.682479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:12.684653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:12.713071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943c272-dc20-4bf4-b5c6-a649ce098233 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:12.723992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:12.724646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:12.724931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:12.725209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] PATH_INFO: `/v3/projects/6877cf3def324994bf7a60560e997ebb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:12.726454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6877cf3def324994bf7a60560e997ebb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:12.728375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:12.794898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6877cf3def324994bf7a60560e997ebb', 'request_id': 'req-3907b22c-959b-432d-a076-6fe280a7f14d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.801979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6877cf3def324994bf7a60560e997ebb', 'request_id': 'req-3907b22c-959b-432d-a076-6fe280a7f14d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.812512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6877cf3def324994bf7a60560e997ebb', 'request_id': 'req-3907b22c-959b-432d-a076-6fe280a7f14d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.827815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] The token cache is being invalidate because project 6877cf3def324994bf7a60560e997ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:12.828133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.828784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3907b22c-959b-432d-a076-6fe280a7f14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:12.830460 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3500/6998] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:12 2022] DELETE /identity/v3/projects/6877cf3def324994bf7a60560e997ebb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:12.836225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfeb49-b809-48cf-b7ba-a4e2affe811d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:12.837009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfeb49-b809-48cf-b7ba-a4e2affe811d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:12.837343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfeb49-b809-48cf-b7ba-a4e2affe811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:12.861876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b8dfeb49-b809-48cf-b7ba-a4e2affe811d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:12.863086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b8dfeb49-b809-48cf-b7ba-a4e2affe811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:12.934301 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3499/6999] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:12.942164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdc73-2855-4b5d-8e67-6296d727e848 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:12.942458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdc73-2855-4b5d-8e67-6296d727e848 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:12.942738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdc73-2855-4b5d-8e67-6296d727e848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:12.964826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6d3cdc73-2855-4b5d-8e67-6296d727e848 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:12.966104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6d3cdc73-2855-4b5d-8e67-6296d727e848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:12.975173 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3501/7000] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:12.981035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfeb49-b809-48cf-b7ba-a4e2affe811d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:12.992298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfeb49-b809-48cf-b7ba-a4e2affe811d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:12.997118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:12.997734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:12.998056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:12.998331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:12.998986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:12.999291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:13.000987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:13.017278 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3500/7001] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:13.018439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cdc73-2855-4b5d-8e67-6296d727e848 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:13.028699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cdc73-2855-4b5d-8e67-6296d727e848 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:13.034677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:13.035424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:13.035709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:13.035944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:13.036586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:13.039434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:13.049467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:13.069708 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3502/7002] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:13 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:13.074398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:13.079662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:13.085439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ffecc-858d-4a76-a573-9c70ba218c15 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:13.148649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:13.149518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:13.149716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:13.150026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:13.150657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:13.152681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:13.174881 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3501/7003] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:13 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:13.179407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:13.190554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63281c38-53f9-40c2-b482-1e5e8102d2e0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:13.195563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:13.196170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:13.196456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:13.196732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:13.197377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:13.198999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:13.209377 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3503/7004] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:13.213267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:13.223557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde1ca-59c8-4387-95bb-1a217e02582b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:13.229180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c0950-4ffe-49a1-9e87-12206a97a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:13.229788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c0950-4ffe-49a1-9e87-12206a97a7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:13.230099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c0950-4ffe-49a1-9e87-12206a97a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:13.230423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c0950-4ffe-49a1-9e87-12206a97a7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:13.231323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c0950-4ffe-49a1-9e87-12206a97a7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:13.233003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c0950-4ffe-49a1-9e87-12206a97a7e1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:13.250792 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3502/7005] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:13.252334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:13.264252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa910f3a-1fdf-44d5-b029-a0994b693843 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:13.269177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:13.269868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:13.270159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:13.270429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] PATH_INFO: `/v3/projects/95c1c88d91b2435897e960b55ee21437/users/e997d1f139a34f40b7e809b476f3d3fb/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:13.291298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c1c88d91b2435897e960b55ee21437, user_id=e997d1f139a34f40b7e809b476f3d3fb, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:13.294785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:13.321597 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3504/7006] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:13 2022] PUT /identity/v3/projects/95c1c88d91b2435897e960b55ee21437/users/e997d1f139a34f40b7e809b476f3d3fb/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:13.323574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:13.323852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:13.324132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:13.356406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] MFA Rules not processed for user `e997d1f139a34f40b7e809b476f3d3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:13.358003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:13.452098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3503/7007] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:13.465728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:13.468223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:13.544423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba9f71-ce86-4d48-adb0-5621273890a0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:13.548505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:13.549224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:13.549891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:13.550250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:13.567014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:13.568691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:13.586648 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3505/7008] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:16.264526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:16.266960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:16.393368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a113a84-4ccc-46c7-9d29-18fe066d106d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:16.399642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:16.400697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:16.401097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:16.401423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] PATH_INFO: `/v3/users/d83b6ceb599746b394e379d52683b70c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:16.403434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d83b6ceb599746b394e379d52683b70c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:16.405680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:16.453885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:16.551834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd83b6ceb599746b394e379d52683b70c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:16.587749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd83b6ceb599746b394e379d52683b70c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:16.604021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd83b6ceb599746b394e379d52683b70c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:16.617393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd83b6ceb599746b394e379d52683b70c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:16.618657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bcb5aebc-e4e0-4f28-a91b-e132447bb6f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd83b6ceb599746b394e379d52683b70c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:16.633779 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3504/7009] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:16 2022] DELETE /identity/v3/users/d83b6ceb599746b394e379d52683b70c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:17.115543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:17.134718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49799bf5-fc06-4988-8914-f605d40102b6 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:17.182834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6761c0-6902-472c-83e0-908f7ec2b33b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:17.183187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6761c0-6902-472c-83e0-908f7ec2b33b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:17.183389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6761c0-6902-472c-83e0-908f7ec2b33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:17.221618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6c6761c0-6902-472c-83e0-908f7ec2b33b None None] MFA Rules not processed for user `93648515d8fb42a7aaf61e245db0e07d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:17.222869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6c6761c0-6902-472c-83e0-908f7ec2b33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:17.283560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:17.284672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:17.285358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:17.285358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] PATH_INFO: `/v3/projects/15a76a25754c42099c542fa13046f01b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:17.287135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a76a25754c42099c542fa13046f01b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:17.289533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:17.338597 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3505/7010] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:17.385385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a76a25754c42099c542fa13046f01b', 'request_id': 'req-c1fb8663-5956-4087-ac40-50b79d5cf12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:17.394623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a76a25754c42099c542fa13046f01b', 'request_id': 'req-c1fb8663-5956-4087-ac40-50b79d5cf12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:17.400379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:17.410784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a76a25754c42099c542fa13046f01b', 'request_id': 'req-c1fb8663-5956-4087-ac40-50b79d5cf12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:17.432529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] The token cache is being invalidate because project 15a76a25754c42099c542fa13046f01b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:17.432964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:17.434287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c1fb8663-5956-4087-ac40-50b79d5cf12b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:17.439034 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3506/7011] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:17 2022] DELETE /identity/v3/projects/15a76a25754c42099c542fa13046f01b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:17.443682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22ebbe-85de-4496-a75a-0f39a3899a4f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:17.443908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22ebbe-85de-4496-a75a-0f39a3899a4f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:17.445567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22ebbe-85de-4496-a75a-0f39a3899a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:17.478782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1d22ebbe-85de-4496-a75a-0f39a3899a4f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:17.480413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d22ebbe-85de-4496-a75a-0f39a3899a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:17.568347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3506/7012] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:17.582100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c3c19-ba62-47c8-aacd-d34c979e45f5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:17.582509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c3c19-ba62-47c8-aacd-d34c979e45f5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:17.582797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c3c19-ba62-47c8-aacd-d34c979e45f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:17.650017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-929c3c19-ba62-47c8-aacd-d34c979e45f5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:17.651653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-929c3c19-ba62-47c8-aacd-d34c979e45f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:17.689951 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3507/7013] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:17.695639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22ebbe-85de-4496-a75a-0f39a3899a4f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:17.709159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22ebbe-85de-4496-a75a-0f39a3899a4f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:17.736861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:17.737523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:17.738770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:17.739059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:17.739806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:17.740441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:17.742652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:17.781879 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3507/7014] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:17:17.784098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c3c19-ba62-47c8-aacd-d34c979e45f5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:17.795990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c3c19-ba62-47c8-aacd-d34c979e45f5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:17.803268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:17.804845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:17.805279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:17.805734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:17.806896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:17.809113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:17.890509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3508/7015] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:17 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:17.892888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:17.910399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98232bbd-4390-4bd0-9a01-f851bff4740f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:17.958686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:18.033618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:18.035240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:18.035817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:18.036358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:18.037226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:18.039806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:18.076729 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3508/7016] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:17 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:18.081468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:18.094194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c0708-8284-4b52-92ee-40eba4c1da39 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:18.099944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:18.099944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:18.100513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:18.101152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:18.109212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:18.112879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:18.133149 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3509/7017] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:18.133707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:18.144267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba86dab-ecc9-421d-ae86-f2fb5f696989 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:18.155672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7cb87-19c9-486c-9e1f-dc542904df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:18.156619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7cb87-19c9-486c-9e1f-dc542904df47 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:18.157065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7cb87-19c9-486c-9e1f-dc542904df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:18.157525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7cb87-19c9-486c-9e1f-dc542904df47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:18.158381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7cb87-19c9-486c-9e1f-dc542904df47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:18.160163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7cb87-19c9-486c-9e1f-dc542904df47 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:18.174504 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3509/7018] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:18.179902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:18.192839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38adb2-6da7-4330-8359-a363cedaa292 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:18.200089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:18.204424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:18.206533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:18.206906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] PATH_INFO: `/v3/projects/ea12a99438e74afa861bf5b72f95e77e/users/09c6a44a52c743bca8ff5cd46b8b2a1f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:18.235068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea12a99438e74afa861bf5b72f95e77e, user_id=09c6a44a52c743bca8ff5cd46b8b2a1f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:18.238378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:18.296802 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3510/7019] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:18 2022] PUT /identity/v3/projects/ea12a99438e74afa861bf5b72f95e77e/users/09c6a44a52c743bca8ff5cd46b8b2a1f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:18.299084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:18.299665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:18.300197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:18.361706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] MFA Rules not processed for user `09c6a44a52c743bca8ff5cd46b8b2a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:18.363656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:18.472905 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3510/7020] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:18.486527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:18.488958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:18.595437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7defb0-ccfe-4293-bf1b-45504d88f758 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:18.602433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65473bd2-2258-47c4-96fb-f59a3440f927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:18.603056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473bd2-2258-47c4-96fb-f59a3440f927 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:18.603337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473bd2-2258-47c4-96fb-f59a3440f927 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:18.603614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473bd2-2258-47c4-96fb-f59a3440f927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:18.620649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473bd2-2258-47c4-96fb-f59a3440f927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:18.622150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473bd2-2258-47c4-96fb-f59a3440f927 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:18.633372 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3511/7021] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:22.583214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:22.586377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:22.784719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffe90a-bc21-4c8d-acdd-943f70eb3baa None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:22.810073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:22.810073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:22.810073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:22.810073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] PATH_INFO: `/v3/users/e997d1f139a34f40b7e809b476f3d3fb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:22.810430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=e997d1f139a34f40b7e809b476f3d3fb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:22.810504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:22.875697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:22.908163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e997d1f139a34f40b7e809b476f3d3fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:22.938142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e997d1f139a34f40b7e809b476f3d3fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:22.954082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e997d1f139a34f40b7e809b476f3d3fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:22.988162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e997d1f139a34f40b7e809b476f3d3fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:22.989318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e997d1f139a34f40b7e809b476f3d3fb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:23.004417 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3511/7022] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:22 2022] DELETE /identity/v3/users/e997d1f139a34f40b7e809b476f3d3fb => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:23.010157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:23.012534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:23.013393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:23.064769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:23.066898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:23.169850 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3512/7023] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:23.182662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:23.185404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:23.287899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4f17-709c-4e43-9bcd-ef98628577ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:23.292248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2466caa-da2c-4cd2-9c0d-4c9f07b3d502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:23.292916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2466caa-da2c-4cd2-9c0d-4c9f07b3d502 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:23.293203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2466caa-da2c-4cd2-9c0d-4c9f07b3d502 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:23.293483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2466caa-da2c-4cd2-9c0d-4c9f07b3d502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:23.318081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2466caa-da2c-4cd2-9c0d-4c9f07b3d502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:23.320142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2466caa-da2c-4cd2-9c0d-4c9f07b3d502 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:23.333887 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3512/7024] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:23.728409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:23.735790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:23.759702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a25fa9-80e4-44eb-8f7a-6348513d8f98 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:23.764788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:23.765455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:23.765947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:23.766227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] PATH_INFO: `/v3/projects/95c1c88d91b2435897e960b55ee21437` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:23.767373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95c1c88d91b2435897e960b55ee21437)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:23.769448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:23.801513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:23.877848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95c1c88d91b2435897e960b55ee21437', 'request_id': 'req-0b997c71-8933-4d59-9b12-af50d1c72130'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:23.880485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95c1c88d91b2435897e960b55ee21437', 'request_id': 'req-0b997c71-8933-4d59-9b12-af50d1c72130'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:23.894041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95c1c88d91b2435897e960b55ee21437', 'request_id': 'req-0b997c71-8933-4d59-9b12-af50d1c72130'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:23.911495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] The token cache is being invalidate because project 95c1c88d91b2435897e960b55ee21437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:23.912410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:23.914379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b997c71-8933-4d59-9b12-af50d1c72130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:23.917729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3513/7025] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:23 2022] DELETE /identity/v3/projects/95c1c88d91b2435897e960b55ee21437 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:23.922688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65706f3f-7ab9-4728-a434-cd0db6826005 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:23.923695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65706f3f-7ab9-4728-a434-cd0db6826005 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:23.924399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65706f3f-7ab9-4728-a434-cd0db6826005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:23.950680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-65706f3f-7ab9-4728-a434-cd0db6826005 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:23.952440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65706f3f-7ab9-4728-a434-cd0db6826005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:24.036332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3513/7026] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:24.044845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc78c5b-f529-4698-885c-312b1d6c178d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.045439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc78c5b-f529-4698-885c-312b1d6c178d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.045439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc78c5b-f529-4698-885c-312b1d6c178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.071327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-efc78c5b-f529-4698-885c-312b1d6c178d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:24.073402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-efc78c5b-f529-4698-885c-312b1d6c178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:24.085515 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3514/7027] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:24.090030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65706f3f-7ab9-4728-a434-cd0db6826005 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:24.105565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65706f3f-7ab9-4728-a434-cd0db6826005 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:24.112557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:24.113432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.114901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.115379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.117455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:24.118691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:24.120636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:24.167261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3514/7028] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:24.179617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc78c5b-f529-4698-885c-312b1d6c178d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:24.190947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc78c5b-f529-4698-885c-312b1d6c178d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:24.197117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:24.197659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.197978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.198259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.198901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:24.200818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:24.262238 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3515/7029] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:24 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:24.266516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:24.281703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c55ec-97c6-468c-8f2f-9148271b01ce admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:24.342674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:24.389667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:24.390655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.390946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.391224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.391890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:24.393876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:24.425888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3515/7030] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:24 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:24.429577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:24.446955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e3c74-b2d6-4cd6-9625-490c40bcb63c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:24.456120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:24.456721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.457373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.457674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.458364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:24.460164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:24.504811 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3516/7031] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:24.510476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:24.524741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f4f5a-2448-4ab8-b009-528b7803fa94 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:24.529855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeea20-60f7-430b-a04b-200f371c483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:24.530638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeea20-60f7-430b-a04b-200f371c483a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.530928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeea20-60f7-430b-a04b-200f371c483a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.531208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeea20-60f7-430b-a04b-200f371c483a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.531861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeea20-60f7-430b-a04b-200f371c483a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:24.533461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeea20-60f7-430b-a04b-200f371c483a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:24.568860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:24.569664 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3516/7032] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:24.582454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cf924-24d8-4a9d-bfb9-2389e88dac11 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:24.589207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:24.589935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.590867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.591396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] PATH_INFO: `/v3/projects/19b2b86d48174d739afbbc8e8061161c/users/48a02bd3d7ad43a7b285ba5eb47bb15e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.619470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b2b86d48174d739afbbc8e8061161c, user_id=48a02bd3d7ad43a7b285ba5eb47bb15e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:24.622692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:24.653912 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3517/7033] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:24 2022] PUT /identity/v3/projects/19b2b86d48174d739afbbc8e8061161c/users/48a02bd3d7ad43a7b285ba5eb47bb15e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:24.661060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a94b16-c7ef-46e1-9393-24358a7855bf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.661357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a94b16-c7ef-46e1-9393-24358a7855bf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.661642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a94b16-c7ef-46e1-9393-24358a7855bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.697785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-91a94b16-c7ef-46e1-9393-24358a7855bf None None] MFA Rules not processed for user `48a02bd3d7ad43a7b285ba5eb47bb15e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:24.699286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91a94b16-c7ef-46e1-9393-24358a7855bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:24.775393 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3517/7034] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:24.785978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:24.787933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:24.858928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b032edb-05c6-4c92-a1a0-750c8c273534 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:24.863880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4631ab6-d364-4ead-b0cf-1b4fb1b60d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:24.864645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4631ab6-d364-4ead-b0cf-1b4fb1b60d6f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:24.865003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4631ab6-d364-4ead-b0cf-1b4fb1b60d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:24.865357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4631ab6-d364-4ead-b0cf-1b4fb1b60d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:24.877097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4631ab6-d364-4ead-b0cf-1b4fb1b60d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:24.878561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4631ab6-d364-4ead-b0cf-1b4fb1b60d6f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:24.890205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3518/7035] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:25.323062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a94b16-c7ef-46e1-9393-24358a7855bf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:25.401808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a94b16-c7ef-46e1-9393-24358a7855bf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:25.406570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:25.407203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:25.407479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:25.407760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] PATH_INFO: `/v3/users/48a02bd3d7ad43a7b285ba5eb47bb15e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:25.408875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48a02bd3d7ad43a7b285ba5eb47bb15e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:25.410819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:25.461142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48a02bd3d7ad43a7b285ba5eb47bb15e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:25.463186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:25.476944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48a02bd3d7ad43a7b285ba5eb47bb15e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:25.484968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48a02bd3d7ad43a7b285ba5eb47bb15e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:25.493392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48a02bd3d7ad43a7b285ba5eb47bb15e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:25.494172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48a02bd3d7ad43a7b285ba5eb47bb15e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:25.500751 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3518/7036] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:25 2022] DELETE /identity/v3/users/48a02bd3d7ad43a7b285ba5eb47bb15e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:25.505547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:25.506199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:25.506199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:25.527776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:25.529209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:25.592765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:25.619221 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3519/7037] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:25.629935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:25.632323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:25.724011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82ccce-c067-4a14-8807-cac67746e954 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:25.728930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba3bb7-5986-47f6-9be1-8b1f1604497f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:25.729662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba3bb7-5986-47f6-9be1-8b1f1604497f service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:25.730042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba3bb7-5986-47f6-9be1-8b1f1604497f service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:25.730390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba3bb7-5986-47f6-9be1-8b1f1604497f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:25.743454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba3bb7-5986-47f6-9be1-8b1f1604497f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:25.745325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba3bb7-5986-47f6-9be1-8b1f1604497f service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:25.755768 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3519/7038] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:26.171130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:26.172880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:26.192176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb2bdf-fe2c-4b71-aaa8-a1531ea008df None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:26.199542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:26.200557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.204512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.204512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] PATH_INFO: `/v3/projects/19b2b86d48174d739afbbc8e8061161c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.204512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b2b86d48174d739afbbc8e8061161c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:26.205553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:26.290914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b2b86d48174d739afbbc8e8061161c', 'request_id': 'req-fb62d641-1b8c-454a-a377-18b2016a8d79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:26.300898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b2b86d48174d739afbbc8e8061161c', 'request_id': 'req-fb62d641-1b8c-454a-a377-18b2016a8d79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:26.312270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b2b86d48174d739afbbc8e8061161c', 'request_id': 'req-fb62d641-1b8c-454a-a377-18b2016a8d79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:26.330695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] The token cache is being invalidate because project 19b2b86d48174d739afbbc8e8061161c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:26.334007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:26.336136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb62d641-1b8c-454a-a377-18b2016a8d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:26.339371 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3520/7039] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:26 2022] DELETE /identity/v3/projects/19b2b86d48174d739afbbc8e8061161c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:26.344659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0275390-5635-4faf-b1b5-1c8b32b9dc45 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.345724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0275390-5635-4faf-b1b5-1c8b32b9dc45 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.346329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0275390-5635-4faf-b1b5-1c8b32b9dc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.372714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f0275390-5635-4faf-b1b5-1c8b32b9dc45 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:26.373989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f0275390-5635-4faf-b1b5-1c8b32b9dc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:26.448752 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3520/7040] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:26.458362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca43d2-479d-4a2f-9f77-c7ae4b431c9d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.458793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca43d2-479d-4a2f-9f77-c7ae4b431c9d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.459217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca43d2-479d-4a2f-9f77-c7ae4b431c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.488787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-baca43d2-479d-4a2f-9f77-c7ae4b431c9d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:26.490179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-baca43d2-479d-4a2f-9f77-c7ae4b431c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:26.500307 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3521/7041] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:26.505047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0275390-5635-4faf-b1b5-1c8b32b9dc45 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:26.517725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0275390-5635-4faf-b1b5-1c8b32b9dc45 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:26.524594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:26.525155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.525522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.525776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.526558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:26.526849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:26.528623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:26.545702 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3521/7042] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:26.547233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca43d2-479d-4a2f-9f77-c7ae4b431c9d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:26.559125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca43d2-479d-4a2f-9f77-c7ae4b431c9d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:26.565378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:26.566043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.566339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.566614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.567631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:26.570742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:26.606743 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3522/7043] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:26 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:26.611144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:26.622827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352df9dc-1028-452b-b234-2da7e46054ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:26.677363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:26.686567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:26.687881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.688240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.688879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.689791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:26.691857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:26.723784 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3522/7044] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:26 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:26.724055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:26.738075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f0b96-eed6-4359-8bae-63cdce860685 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:26.744470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:26.745297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.745647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.746171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.746899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:26.748592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:26.759868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3523/7045] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:26.763933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:26.776287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f652a-561a-4588-b453-109c13193a54 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:26.781468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfc26a-dc82-4577-a8a4-4456cd12d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:26.786466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfc26a-dc82-4577-a8a4-4456cd12d48e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.786947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfc26a-dc82-4577-a8a4-4456cd12d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.787892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfc26a-dc82-4577-a8a4-4456cd12d48e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.789477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfc26a-dc82-4577-a8a4-4456cd12d48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:26.792255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfc26a-dc82-4577-a8a4-4456cd12d48e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:26.811408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3523/7046] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:26.815254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:26.827050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5ded8-f731-4db5-8742-97cf3919d220 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:26.834912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:26.835615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.835901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.836181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] PATH_INFO: `/v3/projects/dfc79269648b4ea4bb40ffaf5f9766ce/users/200b4be60e9345bb94273b6ece3f55ba/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:26.856286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc79269648b4ea4bb40ffaf5f9766ce, user_id=200b4be60e9345bb94273b6ece3f55ba, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:26.874183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:26.913632 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3524/7047] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:26 2022] PUT /identity/v3/projects/dfc79269648b4ea4bb40ffaf5f9766ce/users/200b4be60e9345bb94273b6ece3f55ba/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:26.923630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf39c-e838-42f7-9d21-93f71c3556d5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:26.924184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf39c-e838-42f7-9d21-93f71c3556d5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:26.925719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf39c-e838-42f7-9d21-93f71c3556d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.014936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f1ccf39c-e838-42f7-9d21-93f71c3556d5 None None] MFA Rules not processed for user `200b4be60e9345bb94273b6ece3f55ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:27.016518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f1ccf39c-e838-42f7-9d21-93f71c3556d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:27.020060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:27.022651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:27.125781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ba8eb-88cc-40bf-bbc6-a90378e30eef admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:27.134647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:27.135463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.136905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:27.137414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.139891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:27.153983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3524/7048] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:27.156435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf39c-e838-42f7-9d21-93f71c3556d5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:27.179929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:27.245065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:27.246633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:27.262529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf39c-e838-42f7-9d21-93f71c3556d5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:27.263412 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3525/7049] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:17:27.273706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:27.274479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.274816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:27.275027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.276756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:27.279247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:27.317115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3525/7050] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:27 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:27.318524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:27.347762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7395a-89db-4f26-8fae-6d7d278bdd32 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:27.513548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:27.514413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.514565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:27.514770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.515410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:27.517528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:27.546305 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3526/7051] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:27 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:27.550445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:27.563056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea9494-df3e-4887-962d-e23fd5307cde admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:27.570988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:27.571613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.571904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:27.572210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.572833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:27.607327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:27.647971 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3526/7052] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:27.651148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:27.666118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984095e-e276-4cae-96ae-680291e75296 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:27.672338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:27.673538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.674234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:27.674724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] PATH_INFO: `/v3/projects/35b60639201640eaaa4b0d440184d7e8/users/d8e350f51a364f2f985ef0d13d9ee9dd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.702292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b60639201640eaaa4b0d440184d7e8, user_id=d8e350f51a364f2f985ef0d13d9ee9dd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:27.712775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:27.746339 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3527/7053] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:27 2022] PUT /identity/v3/projects/35b60639201640eaaa4b0d440184d7e8/users/d8e350f51a364f2f985ef0d13d9ee9dd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:27.750513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:27.770614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ecb13-e2da-47c0-9e38-881e1e8287fa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:27.856221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:27.908700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b46c9-106d-4407-8989-9d112ea48e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:27.909640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b46c9-106d-4407-8989-9d112ea48e1f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.910046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b46c9-106d-4407-8989-9d112ea48e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:27.910432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b46c9-106d-4407-8989-9d112ea48e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.911359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b46c9-106d-4407-8989-9d112ea48e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:27.913895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b46c9-106d-4407-8989-9d112ea48e1f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:27.926949 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3527/7054] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:27.931613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:27.944135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5145f-1d33-4f40-b4d1-f96442da71d9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:27.951265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:27.952225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.952573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:27.952861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] PATH_INFO: `/v3/projects/35b60639201640eaaa4b0d440184d7e8/users/d8e350f51a364f2f985ef0d13d9ee9dd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:27.954923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b60639201640eaaa4b0d440184d7e8, user_id=d8e350f51a364f2f985ef0d13d9ee9dd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:27.968508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:27.988418 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3528/7055] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:27 2022] PUT /identity/v3/projects/35b60639201640eaaa4b0d440184d7e8/users/d8e350f51a364f2f985ef0d13d9ee9dd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:27.999907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dbe18-f4c1-4c73-9446-7bff031c6504 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:27.999907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dbe18-f4c1-4c73-9446-7bff031c6504 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:28.000211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dbe18-f4c1-4c73-9446-7bff031c6504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:28.042381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a36dbe18-f4c1-4c73-9446-7bff031c6504 None None] MFA Rules not processed for user `d8e350f51a364f2f985ef0d13d9ee9dd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:28.043603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a36dbe18-f4c1-4c73-9446-7bff031c6504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:28.132996 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3528/7056] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:28.148768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:28.150740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:28.223052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad194ba9-47b5-4615-a869-4732bb9797f0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:28.227816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:28.228437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:28.228717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:28.228986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:28.239561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:28.241767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:28.253534 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3529/7057] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:28.341923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dbe18-f4c1-4c73-9446-7bff031c6504 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:28.355332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dbe18-f4c1-4c73-9446-7bff031c6504 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:28.359485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d886da-2761-4d02-b129-c4ad2614878f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:28.360150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d886da-2761-4d02-b129-c4ad2614878f service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:28.360913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d886da-2761-4d02-b129-c4ad2614878f service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:28.360913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d886da-2761-4d02-b129-c4ad2614878f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:28.426376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d886da-2761-4d02-b129-c4ad2614878f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:28.428350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d886da-2761-4d02-b129-c4ad2614878f service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:28.441917 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3529/7058] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:28.581020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:28.582600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:28.608886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:28.680169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc4515-1a09-4708-8e4b-bfb64c9dcaa2 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:28.684928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:28.685567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:28.685927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:28.686234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] PATH_INFO: `/v3/users/09c6a44a52c743bca8ff5cd46b8b2a1f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:28.687369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c6a44a52c743bca8ff5cd46b8b2a1f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:28.689295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:28.738978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c6a44a52c743bca8ff5cd46b8b2a1f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:28.746450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c6a44a52c743bca8ff5cd46b8b2a1f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:28.747134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c6a44a52c743bca8ff5cd46b8b2a1f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:28.755022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c6a44a52c743bca8ff5cd46b8b2a1f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:28.755819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c6a44a52c743bca8ff5cd46b8b2a1f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:28.775577 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3530/7059] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:28 2022] DELETE /identity/v3/users/09c6a44a52c743bca8ff5cd46b8b2a1f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:28.780030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:28.780533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:28.780634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:28.800243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:28.801455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:28.856875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:28.866663 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3530/7060] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:28.876347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:28.878134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:28.955136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214c909-6b8e-4ea7-bf39-cbd3d95ae79a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:28.960648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6a39b-2080-4dbb-b504-8085f0cefbb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:28.961986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6a39b-2080-4dbb-b504-8085f0cefbb0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:28.961986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6a39b-2080-4dbb-b504-8085f0cefbb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:28.962196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6a39b-2080-4dbb-b504-8085f0cefbb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:28.981039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6a39b-2080-4dbb-b504-8085f0cefbb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:28.983527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6a39b-2080-4dbb-b504-8085f0cefbb0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:28.997203 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3531/7061] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:29.623117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:29.625202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:29.640775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0d7aa-d1c3-4387-bdc5-531accee7792 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:29.646684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:29.648515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:29.648807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:29.649090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] PATH_INFO: `/v3/projects/ea12a99438e74afa861bf5b72f95e77e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:29.651011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea12a99438e74afa861bf5b72f95e77e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:29.652583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:29.776732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea12a99438e74afa861bf5b72f95e77e', 'request_id': 'req-bc209e28-53c0-43a2-8417-8164c050ea3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:29.793139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea12a99438e74afa861bf5b72f95e77e', 'request_id': 'req-bc209e28-53c0-43a2-8417-8164c050ea3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:29.806089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea12a99438e74afa861bf5b72f95e77e', 'request_id': 'req-bc209e28-53c0-43a2-8417-8164c050ea3e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:29.835731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] The token cache is being invalidate because project ea12a99438e74afa861bf5b72f95e77e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:29.836085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:29.836763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bc209e28-53c0-43a2-8417-8164c050ea3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:29.838541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3531/7062] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:29 2022] DELETE /identity/v3/projects/ea12a99438e74afa861bf5b72f95e77e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:29.845029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9c5c0-8823-4fc7-a74c-f8ad6ba796d0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:29.845582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9c5c0-8823-4fc7-a74c-f8ad6ba796d0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:29.845582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9c5c0-8823-4fc7-a74c-f8ad6ba796d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:29.888616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bde9c5c0-8823-4fc7-a74c-f8ad6ba796d0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:29.890655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bde9c5c0-8823-4fc7-a74c-f8ad6ba796d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:30.022412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:30.176342 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3532/7063] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:30.186154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d109-3fb7-4267-9243-b7e3f92ea53e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.186439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d109-3fb7-4267-9243-b7e3f92ea53e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.186893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d109-3fb7-4267-9243-b7e3f92ea53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.243623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8b86d109-3fb7-4267-9243-b7e3f92ea53e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:30.248877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8b86d109-3fb7-4267-9243-b7e3f92ea53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:30.283080 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3532/7064] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:30.292282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9c5c0-8823-4fc7-a74c-f8ad6ba796d0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:30.309364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9c5c0-8823-4fc7-a74c-f8ad6ba796d0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:30.324864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:30.325546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.326977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.327947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.329770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:30.331332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:30.336211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:30.373372 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3533/7065] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:30.374835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d109-3fb7-4267-9243-b7e3f92ea53e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:30.390071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d109-3fb7-4267-9243-b7e3f92ea53e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:30.401276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:30.403314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.406001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.406493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.419492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:30.421614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:30.519810 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3533/7066] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:30 2022] POST /identity/v3/projects => generated 403 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:30.520968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:30.541748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7ba6a-845c-448e-9162-cf76abae2cdd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:30.678820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:30.678820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.678820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.679248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.680061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:30.682361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:30.716953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:30.725770 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3534/7067] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:30 2022] POST /identity/v3/users => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:30.730256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598ea8-363a-4447-983b-b38951fe7f8a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:30.736632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:30.737267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.737642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.737970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.738627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:30.740407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:30.754676 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3534/7068] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:30.758343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:30.771146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ab30d-cdd6-4bd6-a077-4d60ac25117e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:30.781077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bea5b2-cb9a-4eb9-9664-69e4b65855ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:30.782305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea5b2-cb9a-4eb9-9664-69e4b65855ae admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.782863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea5b2-cb9a-4eb9-9664-69e4b65855ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.783226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea5b2-cb9a-4eb9-9664-69e4b65855ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.784171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea5b2-cb9a-4eb9-9664-69e4b65855ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:30.786422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea5b2-cb9a-4eb9-9664-69e4b65855ae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:30.800833 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3535/7069] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:30.806828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:30.823516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132d47b-34bb-4065-891e-021e9fef4d58 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:30.829981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:30.830657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.831104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.831540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] PATH_INFO: `/v3/projects/65f7a2da8ed24d68bfcb8447d8977fac/users/d343abdf98014087b59a05d7ec28f017/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.852574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f7a2da8ed24d68bfcb8447d8977fac, user_id=d343abdf98014087b59a05d7ec28f017, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:30.855550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:30.877765 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3535/7070] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:30 2022] PUT /identity/v3/projects/65f7a2da8ed24d68bfcb8447d8977fac/users/d343abdf98014087b59a05d7ec28f017/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:30.888569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fb32-5aac-4f0a-b5d0-77279af8fd53 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:30.889072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fb32-5aac-4f0a-b5d0-77279af8fd53 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:30.889170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fb32-5aac-4f0a-b5d0-77279af8fd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:30.923921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-64b7fb32-5aac-4f0a-b5d0-77279af8fd53 None None] MFA Rules not processed for user `d343abdf98014087b59a05d7ec28f017`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:30.926468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64b7fb32-5aac-4f0a-b5d0-77279af8fd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:31.033877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3536/7071] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:31.036759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.107496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd670-4931-4250-b6da-735198ed8ebf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.113305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.113947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.114234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.114782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.115143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.117877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.146596 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3536/7072] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:31 2022] POST /identity/v3/projects => generated 401 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:31.150787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7fb32-5aac-4f0a-b5d0-77279af8fd53 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.161657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7fb32-5aac-4f0a-b5d0-77279af8fd53 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.193280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:31.223966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.224591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.224878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.225681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.226471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.228538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.252074 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3537/7073] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:31 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:31.255925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.263378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:31.267507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd065b-1572-4725-b37c-4adcc983873a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.274345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.275566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.275854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.276133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.276786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.278588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.300689 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3537/7074] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:31.302059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.315619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfa19c-393a-4510-aec5-6cc9082c5dc1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.321784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.323593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.323892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.324154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] PATH_INFO: `/v3/projects/2a25033f1683453588d724fc8aea7b2a/users/f7f4b13c42a846879c9a4c5abafa5347/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.343857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a25033f1683453588d724fc8aea7b2a, user_id=f7f4b13c42a846879c9a4c5abafa5347, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.346767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.363277 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3538/7075] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:31 2022] PUT /identity/v3/projects/2a25033f1683453588d724fc8aea7b2a/users/f7f4b13c42a846879c9a4c5abafa5347/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:31.367486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.377792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6cbee-92fc-427b-9a08-072a489362e6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.443987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743a452-1c7f-4e09-9ed7-a6d95c13b869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.444782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a452-1c7f-4e09-9ed7-a6d95c13b869 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.445133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a452-1c7f-4e09-9ed7-a6d95c13b869 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.445482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a452-1c7f-4e09-9ed7-a6d95c13b869 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.446237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743a452-1c7f-4e09-9ed7-a6d95c13b869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.447899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743a452-1c7f-4e09-9ed7-a6d95c13b869 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.462158 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3538/7076] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:31.463461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.475704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd214e-7a5a-42f9-bdf7-7f8db834c630 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.480687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.481602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.481920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.482200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] PATH_INFO: `/v3/projects/2a25033f1683453588d724fc8aea7b2a/users/f7f4b13c42a846879c9a4c5abafa5347/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.484116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a25033f1683453588d724fc8aea7b2a, user_id=f7f4b13c42a846879c9a4c5abafa5347, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.487170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.497834 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3539/7077] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:31 2022] PUT /identity/v3/projects/2a25033f1683453588d724fc8aea7b2a/users/f7f4b13c42a846879c9a4c5abafa5347/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:31.509873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4eb1d-8065-41cc-a622-d85047fe66a2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.512237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4eb1d-8065-41cc-a622-d85047fe66a2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.512237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4eb1d-8065-41cc-a622-d85047fe66a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.539971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-23b4eb1d-8065-41cc-a622-d85047fe66a2 None None] MFA Rules not processed for user `f7f4b13c42a846879c9a4c5abafa5347`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:31.541200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-23b4eb1d-8065-41cc-a622-d85047fe66a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:31.630549 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3539/7078] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:31 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:31.630830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.647779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d821b84-ef65-4a84-8368-004e7c460d3b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.657294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7f4b13c42a846879c9a4c5abafa5347', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a25033f1683453588d724fc8aea7b2a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.659638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.660252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.660687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.661540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.664135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.694763 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3540/7079] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:17:31 2022] POST /identity/v3/roles => generated 239 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:31.700106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4eb1d-8065-41cc-a622-d85047fe66a2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.710618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4eb1d-8065-41cc-a622-d85047fe66a2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:31.715877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf05b8e-4727-4d13-94a2-5292ffbdd9c0 tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7f4b13c42a846879c9a4c5abafa5347', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a25033f1683453588d724fc8aea7b2a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:31.716672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf05b8e-4727-4d13-94a2-5292ffbdd9c0 tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:31.717104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf05b8e-4727-4d13-94a2-5292ffbdd9c0 tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:31.717477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf05b8e-4727-4d13-94a2-5292ffbdd9c0 tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] PATH_INFO: `/v3/projects/65f7a2da8ed24d68bfcb8447d8977fac/users/d343abdf98014087b59a05d7ec28f017/roles/b971c73654ea4dc4999af8e3d23662f1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:31.720012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf05b8e-4727-4d13-94a2-5292ffbdd9c0 tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: Authorizing `identity:create_grant(project_id=65f7a2da8ed24d68bfcb8447d8977fac, user_id=d343abdf98014087b59a05d7ec28f017, role_id=b971c73654ea4dc4999af8e3d23662f1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:31.723856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf05b8e-4727-4d13-94a2-5292ffbdd9c0 tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:31.753493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3540/7080] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:31 2022] PUT /identity/v3/projects/65f7a2da8ed24d68bfcb8447d8977fac/users/d343abdf98014087b59a05d7ec28f017/roles/b971c73654ea4dc4999af8e3d23662f1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:31.754957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:31.917784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144ab57-c61f-472b-b2e3-b65de1bc398c tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:32.007637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd343abdf98014087b59a05d7ec28f017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65f7a2da8ed24d68bfcb8447d8977fac', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'tempest-test_role-836240458'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:32.008691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:32.009191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:32.009622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] PATH_INFO: `/v3/users/d343abdf98014087b59a05d7ec28f017/application_credentials` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:32.011036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] RBAC: Authorizing `identity:create_application_credential(user_id=d343abdf98014087b59a05d7ec28f017)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:32.013873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:32.056660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:32.197728 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3541/7081] 10.222.0.33 () {60 vars in 1315 bytes} [Sat Sep 10 21:17:31 2022] POST /identity/v3/users/d343abdf98014087b59a05d7ec28f017/application_credentials => generated 658 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:32.219833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262ea5-c1ca-4f4a-bbe3-485432f8e97c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:32.220371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262ea5-c1ca-4f4a-bbe3-485432f8e97c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:32.220371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262ea5-c1ca-4f4a-bbe3-485432f8e97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:32.262097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2d262ea5-c1ca-4f4a-bbe3-485432f8e97c None None] MFA Rules not processed for user `d343abdf98014087b59a05d7ec28f017`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:32.266686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2d262ea5-c1ca-4f4a-bbe3-485432f8e97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:32.346594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:32.441970 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3541/7082] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3395 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 10 21:17:32.445708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:32.457234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f977-02d3-4325-9a44-76827a83eae4 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:32.463824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd343abdf98014087b59a05d7ec28f017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65f7a2da8ed24d68bfcb8447d8977fac', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'tempest-test_role-836240458'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:32.464561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:32.464858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:32.465139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] PATH_INFO: `/v3/users/d343abdf98014087b59a05d7ec28f017/application_credentials/eeb93c01d016479686d8f4669052a701` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:32.466431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] RBAC: Authorizing `identity:delete_application_credential(user_id=d343abdf98014087b59a05d7ec28f017, application_credential_id=eeb93c01d016479686d8f4669052a701)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:32.470304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:32.503354 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3542/7083] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:17:32 2022] DELETE /identity/v3/users/d343abdf98014087b59a05d7ec28f017/application_credentials/eeb93c01d016479686d8f4669052a701 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:32.508374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262ea5-c1ca-4f4a-bbe3-485432f8e97c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:32.575351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262ea5-c1ca-4f4a-bbe3-485432f8e97c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:32.580231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7f4b13c42a846879c9a4c5abafa5347', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a25033f1683453588d724fc8aea7b2a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:32.580818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:32.581118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:32.581378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] PATH_INFO: `/v3/roles/b971c73654ea4dc4999af8e3d23662f1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:32.582975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b971c73654ea4dc4999af8e3d23662f1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:32.585469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:32.628021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Invalidating the token cache because role b971c73654ea4dc4999af8e3d23662f1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:32.628315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.629027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.630887 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3542/7084] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:32 2022] DELETE /identity/v3/roles/b971c73654ea4dc4999af8e3d23662f1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:32.637309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:32.639367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:32.715647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad05046-c232-4d80-8f8d-b610421a4c04 tempest-ApplicationCredentialsV3AdminTest-1456424399 tempest-ApplicationCredentialsV3AdminTest-1456424399-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:32.721067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:32.733201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:32.783386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:32.784043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:32.784303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:32.784575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] PATH_INFO: `/v3/users/d343abdf98014087b59a05d7ec28f017` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:32.785727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d343abdf98014087b59a05d7ec28f017)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:32.788394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:32.806121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a531689-5fdc-43e3-aedc-09bac534c0dd tempest-ApplicationCredentialsV3AdminTest-200639701 tempest-ApplicationCredentialsV3AdminTest-200639701-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:32.811232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:32.811505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:32.811793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:32.812072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] PATH_INFO: `/v3/users/3f03d688d0ee4d2b951828a3f496c914` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:32.813217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f03d688d0ee4d2b951828a3f496c914)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:32.815424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:32.820645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:32.835832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:32.838703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd343abdf98014087b59a05d7ec28f017'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.849538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd343abdf98014087b59a05d7ec28f017'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.854268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd343abdf98014087b59a05d7ec28f017'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.862537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd343abdf98014087b59a05d7ec28f017'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.863381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0b28f9b2-e987-429d-9d50-934199b91a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd343abdf98014087b59a05d7ec28f017'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.872241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f03d688d0ee4d2b951828a3f496c914'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.882333 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3543/7085] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:32 2022] DELETE /identity/v3/users/d343abdf98014087b59a05d7ec28f017 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:32.885346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f03d688d0ee4d2b951828a3f496c914'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.887052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f731c6-644f-440f-a580-e742adaa103c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:32.887339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f731c6-644f-440f-a580-e742adaa103c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:32.887612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f731c6-644f-440f-a580-e742adaa103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:32.892385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f03d688d0ee4d2b951828a3f496c914'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.901460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f03d688d0ee4d2b951828a3f496c914'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.902141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f03d688d0ee4d2b951828a3f496c914'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:32.907644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-30f731c6-644f-440f-a580-e742adaa103c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:32.908774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3543/7086] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:32 2022] DELETE /identity/v3/users/3f03d688d0ee4d2b951828a3f496c914 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:32.908891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30f731c6-644f-440f-a580-e742adaa103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:32.977767 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3544/7087] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:32.992219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:32.994022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:33.066302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6193c-7e42-49f5-af2c-d554124fdb29 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:33.070805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06270c5b-c971-4cb1-972e-c6d9a8c860d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:33.071429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06270c5b-c971-4cb1-972e-c6d9a8c860d7 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.071721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06270c5b-c971-4cb1-972e-c6d9a8c860d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.071991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06270c5b-c971-4cb1-972e-c6d9a8c860d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.082733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06270c5b-c971-4cb1-972e-c6d9a8c860d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:33.085004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06270c5b-c971-4cb1-972e-c6d9a8c860d7 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:33.099060 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3544/7088] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:33.113153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f731c6-644f-440f-a580-e742adaa103c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:33.114591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30f731c6-644f-440f-a580-e742adaa103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:33.153994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f731c6-644f-440f-a580-e742adaa103c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:33.165809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:33.166821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.167104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.168193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] PATH_INFO: `/v3/projects/2d055111e2e44ed7afe3b335094df69a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.169486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d055111e2e44ed7afe3b335094df69a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:33.172051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:33.245982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d055111e2e44ed7afe3b335094df69a', 'request_id': 'req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.253334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d055111e2e44ed7afe3b335094df69a', 'request_id': 'req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.267186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d055111e2e44ed7afe3b335094df69a', 'request_id': 'req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.293395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] The token cache is being invalidate because project 2d055111e2e44ed7afe3b335094df69a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:33.293801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.294796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d1c3fb61-1ab2-41ea-9604-24f83d26ea85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.296805 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3545/7089] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:33 2022] DELETE /identity/v3/projects/2d055111e2e44ed7afe3b335094df69a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:33.306590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.306590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.307133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.361035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:33.362385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:33.455312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db71f3-b4ab-4f24-92ce-7f4d3c15df87 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.455688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db71f3-b4ab-4f24-92ce-7f4d3c15df87 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.456019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db71f3-b4ab-4f24-92ce-7f4d3c15df87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.461257 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3545/7090] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:33.479090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:33.483046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:33.483757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-88db71f3-b4ab-4f24-92ce-7f4d3c15df87 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:33.486984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88db71f3-b4ab-4f24-92ce-7f4d3c15df87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:33.498729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7d52a-02c1-4f16-bfe4-5d3cce3515c0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:33.505040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:33.505971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.506839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.507120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] PATH_INFO: `/v3/projects/65f7a2da8ed24d68bfcb8447d8977fac` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.508347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f7a2da8ed24d68bfcb8447d8977fac)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:33.510723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:33.516185 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3546/7091] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:33.520838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db71f3-b4ab-4f24-92ce-7f4d3c15df87 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:33.542052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db71f3-b4ab-4f24-92ce-7f4d3c15df87 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:33.545306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:33.548512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:33.549931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.550642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.550987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.551806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:33.552484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:33.557067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:33.578044 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3547/7092] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:33.579867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:33.591429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4cfac-b5a9-4b6b-8737-2e7e7b69e237 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:33.591870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f7a2da8ed24d68bfcb8447d8977fac', 'request_id': 'req-3993c31b-86b6-4313-a1f1-addd118967fd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.599338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f7a2da8ed24d68bfcb8447d8977fac', 'request_id': 'req-3993c31b-86b6-4313-a1f1-addd118967fd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.609250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f7a2da8ed24d68bfcb8447d8977fac', 'request_id': 'req-3993c31b-86b6-4313-a1f1-addd118967fd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.623539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:33.637435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] The token cache is being invalidate because project 65f7a2da8ed24d68bfcb8447d8977fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:33.637744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.638446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.640078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3546/7093] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:33 2022] DELETE /identity/v3/projects/65f7a2da8ed24d68bfcb8447d8977fac => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:33.644478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:33.646378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:33.713425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:33.717178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.717473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.717816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.718493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:33.720431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:33.754281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993c31b-86b6-4313-a1f1-addd118967fd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:33.767705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:33.768277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.771164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.771468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] PATH_INFO: `/v3/users/f7f4b13c42a846879c9a4c5abafa5347` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.775981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f4b13c42a846879c9a4c5abafa5347)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:33.780131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:33.784482 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3548/7094] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:33 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:33.793307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:33.796988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:33.869861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f4b13c42a846879c9a4c5abafa5347'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.887231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f4b13c42a846879c9a4c5abafa5347'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.899885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f4b13c42a846879c9a4c5abafa5347'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.906383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60a11f-1e23-46b8-b4ea-d2bec60486a0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:33.910700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f4b13c42a846879c9a4c5abafa5347'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.911773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f4b13c42a846879c9a4c5abafa5347'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:33.914157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:33.914787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:33.915232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:33.915476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:33.916116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:33.918329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:33.923891 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3547/7095] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:33 2022] DELETE /identity/v3/users/f7f4b13c42a846879c9a4c5abafa5347 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:33.956780 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3549/7096] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:33 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:33.958245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:33.967265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:34.023353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:34.073528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91feef-04a8-49ce-ba2f-cc757425f9f7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.079344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.079972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.080287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.080567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.081214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.082965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.094795 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3548/7097] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:34.098711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.110316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f3fa-be5e-42c2-bd49-1bc6679f266b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.119437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f867d8-e7e4-4696-97e0-f996b2087831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.120176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f867d8-e7e4-4696-97e0-f996b2087831 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.120505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f867d8-e7e4-4696-97e0-f996b2087831 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.120943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f867d8-e7e4-4696-97e0-f996b2087831 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.121783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f867d8-e7e4-4696-97e0-f996b2087831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.123583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f867d8-e7e4-4696-97e0-f996b2087831 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.134823 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3550/7098] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:34.139328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.150036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d876a7-7ed0-4922-8a37-c8176b761c07 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.155739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.156415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.156700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.157006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] PATH_INFO: `/v3/projects/70e1660220ea432f992abf725502fa6d/users/335e978e0c3241fcb9661c595f7d2cea/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.174333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e1660220ea432f992abf725502fa6d, user_id=335e978e0c3241fcb9661c595f7d2cea, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.177613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.192145 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3549/7099] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:34 2022] PUT /identity/v3/projects/70e1660220ea432f992abf725502fa6d/users/335e978e0c3241fcb9661c595f7d2cea/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:34.203288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76a248-e62a-4834-8604-47d3a1a36216 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.203608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76a248-e62a-4834-8604-47d3a1a36216 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.203882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76a248-e62a-4834-8604-47d3a1a36216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.233537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9d76a248-e62a-4834-8604-47d3a1a36216 None None] MFA Rules not processed for user `335e978e0c3241fcb9661c595f7d2cea`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:34.235013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9d76a248-e62a-4834-8604-47d3a1a36216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:34.260541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.262480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:34.309461 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3551/7100] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:34.314122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76a248-e62a-4834-8604-47d3a1a36216 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.335033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:34.349818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c82480-3eaa-431d-ba11-5fd9098a073f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.356353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.356994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.357279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.357556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] PATH_INFO: `/v3/projects/2a25033f1683453588d724fc8aea7b2a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.358852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a25033f1683453588d724fc8aea7b2a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.360753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.391091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76a248-e62a-4834-8604-47d3a1a36216 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.396393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.397063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.397394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.397717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.398326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.400243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.411956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a25033f1683453588d724fc8aea7b2a', 'request_id': 'req-5c158af6-bea4-46fd-81ad-b21439596eea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:34.417941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a25033f1683453588d724fc8aea7b2a', 'request_id': 'req-5c158af6-bea4-46fd-81ad-b21439596eea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:34.424835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a25033f1683453588d724fc8aea7b2a', 'request_id': 'req-5c158af6-bea4-46fd-81ad-b21439596eea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:34.431756 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3552/7101] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:34.435436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.438713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] The token cache is being invalidate because project 2a25033f1683453588d724fc8aea7b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:34.439013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:34.440353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5c158af6-bea4-46fd-81ad-b21439596eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:34.441919 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3550/7102] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:34 2022] DELETE /identity/v3/projects/2a25033f1683453588d724fc8aea7b2a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:34.448878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4454a5-d437-4f76-b487-850ccadb87ca None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.448878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4454a5-d437-4f76-b487-850ccadb87ca None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.448878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4454a5-d437-4f76-b487-850ccadb87ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.471958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9e4454a5-d437-4f76-b487-850ccadb87ca None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:34.473172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9e4454a5-d437-4f76-b487-850ccadb87ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:34.502297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:34.503217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a204-ca69-44ed-9ebd-e2d9527daeaf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.508523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.509160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.509449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.509724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.510515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.512468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.533949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3553/7103] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:34.537982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.539679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:34.542099 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3551/7104] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:34.551109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a22575-bad6-4dd9-8aea-e7991fda59fc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.551531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a22575-bad6-4dd9-8aea-e7991fda59fc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.551709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a22575-bad6-4dd9-8aea-e7991fda59fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.552631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ef7eb-fab8-42d9-ba87-e84fc58c044d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.558333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.559050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.559402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.559773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.560478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.562483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.573812 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3554/7105] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:34.577352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f0a22575-bad6-4dd9-8aea-e7991fda59fc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:34.578244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.578662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f0a22575-bad6-4dd9-8aea-e7991fda59fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:34.587094 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3552/7106] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:34.589547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0db8-dc93-454e-b6ac-4f762f9cf195 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.591294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a22575-bad6-4dd9-8aea-e7991fda59fc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.594714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.595367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.595650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.595925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] PATH_INFO: `/v3/projects/e38a51846ab44195842b61de62db074d/users/01f68128de4245e99ce6b3c658490493/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.600634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a22575-bad6-4dd9-8aea-e7991fda59fc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.605301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.606187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.606476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.606756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.607403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:34.607710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.609573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.612086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38a51846ab44195842b61de62db074d, user_id=01f68128de4245e99ce6b3c658490493, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.614850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.622597 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3553/7107] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:17:34.627349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.628791 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3555/7108] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:34 2022] PUT /identity/v3/projects/e38a51846ab44195842b61de62db074d/users/01f68128de4245e99ce6b3c658490493/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:34.633394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.639222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b57e20-8b0a-46f9-ba93-b2f9cf3c31c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.695504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:34.698430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.699040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.699348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.699571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68b82c-9ad7-45f3-85c7-4066fbcc7434 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.699680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.700244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.702307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.705727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.706388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.706675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.706952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.707598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.709255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.719987 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3556/7109] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:34.724412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.734524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85105b59-a7d2-43ec-9286-6cd080f68982 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.736629 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3554/7110] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:34.737483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.792309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d492885-46a8-46e4-9216-94f9897d8d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.792992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d492885-46a8-46e4-9216-94f9897d8d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.793270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d492885-46a8-46e4-9216-94f9897d8d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.793543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d492885-46a8-46e4-9216-94f9897d8d2e admin admin] PATH_INFO: `/v3/projects/e38a51846ab44195842b61de62db074d/users/01f68128de4245e99ce6b3c658490493/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.795469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d492885-46a8-46e4-9216-94f9897d8d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38a51846ab44195842b61de62db074d, user_id=01f68128de4245e99ce6b3c658490493, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.798734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d492885-46a8-46e4-9216-94f9897d8d2e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.799814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80723c-fa9c-4255-a5ca-d98c8d2038f6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.804599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.805206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.805492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.805783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.806460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.808407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.809198 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3557/7111] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:34 2022] PUT /identity/v3/projects/e38a51846ab44195842b61de62db074d/users/01f68128de4245e99ce6b3c658490493/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:34.821133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386cd30-0837-46f4-bbe1-4b6797a7ff4f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.821422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386cd30-0837-46f4-bbe1-4b6797a7ff4f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.821685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386cd30-0837-46f4-bbe1-4b6797a7ff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.845580 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3555/7112] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:34.848649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.860989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a69bc-6c7e-48cf-9bb4-c879a405effb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.867337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d386cd30-0837-46f4-bbe1-4b6797a7ff4f None None] MFA Rules not processed for user `01f68128de4245e99ce6b3c658490493`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:34.868636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d386cd30-0837-46f4-bbe1-4b6797a7ff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:34.895341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:34.934933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.935540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.935818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.936091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.936727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.938465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.949132 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3556/7113] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:34.952084 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3558/7114] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:34.956632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386cd30-0837-46f4-bbe1-4b6797a7ff4f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.956770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:34.968727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055fa17-e2fa-48c8-93f7-b07b420f3281 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.969927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386cd30-0837-46f4-bbe1-4b6797a7ff4f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:34.976531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.977262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.977773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.978395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.979063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.981208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:34.982665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:34.983295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:34.983579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:34.984245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:34.984527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:34.987246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.001223 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3557/7115] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:17:34 2022] POST /identity/v3/domains => generated 246 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:35.002064 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3559/7116] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:35.005272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.007584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.021148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58e0cc-01e8-4e13-9a5c-6ec023ab7214 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.148739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.149434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.149547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.149813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.150496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.152404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.157188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3eff-c9eb-4b12-84ae-136090693126 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.159313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:35.164151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c550ed2-6e7a-4c52-bd87-23fabaa752c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.165466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c550ed2-6e7a-4c52-bd87-23fabaa752c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.165871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c550ed2-6e7a-4c52-bd87-23fabaa752c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.166312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c550ed2-6e7a-4c52-bd87-23fabaa752c0 admin admin] PATH_INFO: `/v3/projects/d5e9807ab6b44595a6a800dfffd6e580/users/7e226bde935543849b0bf3ea52d38439/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.193353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c550ed2-6e7a-4c52-bd87-23fabaa752c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e9807ab6b44595a6a800dfffd6e580, user_id=7e226bde935543849b0bf3ea52d38439, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.196971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c550ed2-6e7a-4c52-bd87-23fabaa752c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.218286 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3558/7117] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:35 2022] PUT /identity/v3/projects/d5e9807ab6b44595a6a800dfffd6e580/users/7e226bde935543849b0bf3ea52d38439/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:35.230815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba8b5-fd8e-4810-a8ea-c49a0998b49f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.231407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba8b5-fd8e-4810-a8ea-c49a0998b49f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.231681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba8b5-fd8e-4810-a8ea-c49a0998b49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.246259 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3560/7118] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:35 2022] POST /identity/v3/projects => generated 410 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:35.250199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.272464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3a2ba8b5-fd8e-4810-a8ea-c49a0998b49f None None] MFA Rules not processed for user `7e226bde935543849b0bf3ea52d38439`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:35.274021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3a2ba8b5-fd8e-4810-a8ea-c49a0998b49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:35.338179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050fa76-6bbd-474e-b751-c2f897feb2cd tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.356822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.357456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.357743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.358057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.358707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.361229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.372567 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3559/7119] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:35.377567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ba8b5-fd8e-4810-a8ea-c49a0998b49f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.404192 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3561/7120] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:35 2022] POST /identity/v3/users => generated 347 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:35.405239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.413815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:35.422764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85141004-0dbf-41f7-8e74-7d254c0587a1 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.439709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.440895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.441403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.441968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.442870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.445634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.462942 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3562/7121] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:35.463933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.474309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ba8b5-fd8e-4810-a8ea-c49a0998b49f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.475768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee53e5-1ecf-417d-95fc-bfa947ca4495 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.480116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.481165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.481451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.481565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b8a2d-eb3e-4ba6-a0a1-2a704f5d4016 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.481793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.482340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b8a2d-eb3e-4ba6-a0a1-2a704f5d4016 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.482434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.482519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b8a2d-eb3e-4ba6-a0a1-2a704f5d4016 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.482792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b8a2d-eb3e-4ba6-a0a1-2a704f5d4016 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/projects/b212884d48f14fd8a2db864d941dc5cf/users/4ad4bb7d45a74fc0aa86c184e72c6c45/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.484746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.502507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b8a2d-eb3e-4ba6-a0a1-2a704f5d4016 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b212884d48f14fd8a2db864d941dc5cf, user_id=4ad4bb7d45a74fc0aa86c184e72c6c45, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.505426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b8a2d-eb3e-4ba6-a0a1-2a704f5d4016 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.523819 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3560/7122] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:35 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:35.531138 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3563/7123] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:35 2022] PUT /identity/v3/projects/b212884d48f14fd8a2db864d941dc5cf/users/4ad4bb7d45a74fc0aa86c184e72c6c45/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:35.534446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.541044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab794-308f-4972-90ec-a09ba615df78 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.541659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab794-308f-4972-90ec-a09ba615df78 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.542241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab794-308f-4972-90ec-a09ba615df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.646129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2888f-a768-4c8b-841d-a7c24e1b4d3c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.655790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.656509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.656856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.657142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.667755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.670554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.699391 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3561/7124] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:35 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:35.703263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.718036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99caae7-2019-4608-8877-fb5c4ffd931f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.721086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-416ab794-308f-4972-90ec-a09ba615df78 None None] MFA Rules not processed for user `4ad4bb7d45a74fc0aa86c184e72c6c45`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:35.722466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-416ab794-308f-4972-90ec-a09ba615df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:35.723612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.724235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.724522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.725351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.726050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.727669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.755946 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3562/7126] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:35.755946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.760078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617c72-4a23-4a5f-94fc-46cdae907e38 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.760398 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3564/7126] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:35.769870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f5be9-eaff-4624-841c-4b86adeb617c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.770206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f5be9-eaff-4624-841c-4b86adeb617c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.771026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f5be9-eaff-4624-841c-4b86adeb617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.783121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.784020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.784317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.784629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] PATH_INFO: `/v3/projects/1a211bfcb2ed4677993994c8176794b8/users/361588b7cbc343f49bed86a5936235ea/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.786418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:35.807224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-440f5be9-eaff-4624-841c-4b86adeb617c None None] MFA Rules not processed for user `4ad4bb7d45a74fc0aa86c184e72c6c45`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:35.807826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a211bfcb2ed4677993994c8176794b8, user_id=361588b7cbc343f49bed86a5936235ea, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.808722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-440f5be9-eaff-4624-841c-4b86adeb617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:35.810586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.822935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3565/7127] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:35.829070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f5be9-eaff-4624-841c-4b86adeb617c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.829728 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3563/7128] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:35 2022] PUT /identity/v3/projects/1a211bfcb2ed4677993994c8176794b8/users/361588b7cbc343f49bed86a5936235ea/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:35.833950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.842527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f5be9-eaff-4624-841c-4b86adeb617c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.912546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ceca7-b249-47bb-93ec-e17ea845c5b2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.917892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.918617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.918969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.919307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.921021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.922925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.923946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ad4bb7d45a74fc0aa86c184e72c6c45', 'user_domain_id': 'a3d92bbf59274ee39d716463fe72aa16', 'system_scope': None, 'project_id': 'b212884d48f14fd8a2db864d941dc5cf', 'project_domain_id': 'a3d92bbf59274ee39d716463fe72aa16', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.924783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.925284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.925726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.926167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:35.941857 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3564/7129] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:35.942846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.943407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.946525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.956254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151458d9-fbd9-43b8-b56e-17f54eca8611 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:35.960363 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3566/7130] 10.222.0.33 () {56 vars in 1284 bytes} [Sat Sep 10 21:17:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:35.968531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:35.977147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1989b-795b-4300-8a2a-0ec287af509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:35.977984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1989b-795b-4300-8a2a-0ec287af509a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:35.978269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1989b-795b-4300-8a2a-0ec287af509a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:35.979032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1989b-795b-4300-8a2a-0ec287af509a admin admin] PATH_INFO: `/v3/projects/1a211bfcb2ed4677993994c8176794b8/users/361588b7cbc343f49bed86a5936235ea/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:35.981778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1989b-795b-4300-8a2a-0ec287af509a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a211bfcb2ed4677993994c8176794b8, user_id=361588b7cbc343f49bed86a5936235ea, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:35.984519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1989b-795b-4300-8a2a-0ec287af509a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:35.997761 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3565/7131] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:35 2022] PUT /identity/v3/projects/1a211bfcb2ed4677993994c8176794b8/users/361588b7cbc343f49bed86a5936235ea/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:36.074893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe55df-11a9-44a9-9333-e213ab33c84b tempest-test_project-1771823587 tempest-user-154766561] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.098593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f3ac4-803b-4b28-bc6b-b7584ebbce02 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.098891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f3ac4-803b-4b28-bc6b-b7584ebbce02 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.099205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f3ac4-803b-4b28-bc6b-b7584ebbce02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.169723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d11f3ac4-803b-4b28-bc6b-b7584ebbce02 None None] MFA Rules not processed for user `361588b7cbc343f49bed86a5936235ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:36.170990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d11f3ac4-803b-4b28-bc6b-b7584ebbce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:36.302300 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3566/7132] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:36.312069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f3ac4-803b-4b28-bc6b-b7584ebbce02 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.324116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f3ac4-803b-4b28-bc6b-b7584ebbce02 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.332033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.332757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.333272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.333503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.335212 ncdv-master-1 devstack@keystone.service[93527]: WARNING oslo_log.versionutils [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 10 21:17:36.335802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.337138 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:36.337138 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:17:36.337138 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:36.338293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.380795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.381853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.382502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.383067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/users/4ad4bb7d45a74fc0aa86c184e72c6c45` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.384845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:delete_user(user_id=4ad4bb7d45a74fc0aa86c184e72c6c45)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.389183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.399404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3567/7133] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:17:36 2022] POST /identity/v3/policies => generated 230 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:36.411653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.427228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82012b6-39d4-4979-90bd-41eec4a33bec tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.436127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.436127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.436127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.436905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies/e018208733eb477fb30120a791d4e032` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.438073 ncdv-master-1 devstack@keystone.service[93527]: WARNING oslo_log.versionutils [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 10 21:17:36.438649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=e018208733eb477fb30120a791d4e032)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.441240 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:36.441240 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:17:36.441240 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:36.442905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.469568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:36.482067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad4bb7d45a74fc0aa86c184e72c6c45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.502872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad4bb7d45a74fc0aa86c184e72c6c45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.503658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad4bb7d45a74fc0aa86c184e72c6c45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.510074 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3568/7134] 10.222.0.33 () {60 vars in 1252 bytes} [Sat Sep 10 21:17:36 2022] PATCH /identity/v3/policies/e018208733eb477fb30120a791d4e032 => generated 236 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:36.515098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.534100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad4bb7d45a74fc0aa86c184e72c6c45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.536254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad4bb7d45a74fc0aa86c184e72c6c45'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.537135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f38e6-e0fb-4fe5-a8c1-764318c5c386 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.546325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.546962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.547257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.547537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies/e018208733eb477fb30120a791d4e032` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.548328 ncdv-master-1 devstack@keystone.service[93527]: WARNING oslo_log.versionutils [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 10 21:17:36.548894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=e018208733eb477fb30120a791d4e032)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.550159 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:36.550159 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:17:36.550159 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:36.551057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.564427 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3569/7135] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:17:36 2022] GET /identity/v3/policies/e018208733eb477fb30120a791d4e032 => generated 236 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:36.570145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.573310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:36.582838 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3567/7136] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:35 2022] DELETE /identity/v3/users/4ad4bb7d45a74fc0aa86c184e72c6c45 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:36.583955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.586928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:36.603574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07313e7-8762-4316-a2ef-30bf076a83ce tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.694007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.694661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.694661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.695212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies/e018208733eb477fb30120a791d4e032` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.695983 ncdv-master-1 devstack@keystone.service[93527]: WARNING oslo_log.versionutils [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 10 21:17:36.696608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=e018208733eb477fb30120a791d4e032)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.712072 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:36.712072 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:17:36.712072 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:36.716680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.721214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72de476-0db0-4ac0-905b-6680be2b4c91 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.730116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.732770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.733258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.733779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/projects/b212884d48f14fd8a2db864d941dc5cf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.735732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b212884d48f14fd8a2db864d941dc5cf)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.737872 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3570/7137] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:36 2022] DELETE /identity/v3/policies/e018208733eb477fb30120a791d4e032 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:36.738028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.743890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.758871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91f7c2-89c5-4747-968f-566268e17968 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.767160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.768454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.768839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.769137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.773903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.776288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.802767 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3571/7138] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:17:36 2022] POST /identity/v3/policies => generated 229 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:36.804038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.804639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b212884d48f14fd8a2db864d941dc5cf', 'request_id': 'req-995dcc37-2aa8-4b63-b689-93b05725e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.814664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b212884d48f14fd8a2db864d941dc5cf', 'request_id': 'req-995dcc37-2aa8-4b63-b689-93b05725e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.815349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363ee39-49c0-48e2-b339-2be261bfa8e0 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.824493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:36.826342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b212884d48f14fd8a2db864d941dc5cf', 'request_id': 'req-995dcc37-2aa8-4b63-b689-93b05725e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.843196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] The token cache is being invalidate because project b212884d48f14fd8a2db864d941dc5cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:36.843564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.845419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:36.847083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3568/7139] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:36 2022] DELETE /identity/v3/projects/b212884d48f14fd8a2db864d941dc5cf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:36.851822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.853640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:36.901134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.902551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.902551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.902551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.904178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.906620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.935915 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3572/7140] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:17:36 2022] POST /identity/v3/policies => generated 228 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:36.939855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:36.942388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:36.947362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:36.961552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaf3a3-d070-4574-b9f6-4a6f88beec0c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:36.971796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:36.972445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:36.974263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:36.974263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:36.976781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:36.979714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:36.989980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dcc37-2aa8-4b63-b689-93b05725e2b7 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.003485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.005427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.005643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.005942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/domains/a3d92bbf59274ee39d716463fe72aa16` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.011186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a3d92bbf59274ee39d716463fe72aa16)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.013946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.017414 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3573/7141] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:17:36 2022] POST /identity/v3/policies => generated 230 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:37.021655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.035693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ae61-3122-48ef-930e-3e575b239553 tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.045198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.046636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.047288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.047785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.049145 ncdv-master-1 devstack@keystone.service[93527]: WARNING oslo_log.versionutils [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 10 21:17:37.050057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.050866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.051739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.052474 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:37.052474 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:17:37.052474 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:37.055090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.073286 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3574/7142] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:17:37 2022] GET /identity/v3/policies => generated 763 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:37.074743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.077147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.095404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.098227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.108007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268031a6-99be-48f7-a478-c926179c339e tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.108874 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3569/7143] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:17:36 2022] PATCH /identity/v3/domains/a3d92bbf59274ee39d716463fe72aa16 => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:37.109944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.111621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.231188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14415f4c-e6fb-423a-b87f-1098a17f67c8 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.236469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.239914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.241666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.242154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies/98e0acd5ea40484a9f59505115ec95d0` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.243231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01f68128de4245e99ce6b3c658490493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e38a51846ab44195842b61de62db074d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.243882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.244162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.244438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] PATH_INFO: `/v3/domains/a3d92bbf59274ee39d716463fe72aa16` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.244620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=98e0acd5ea40484a9f59505115ec95d0)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.247080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.260225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a3d92bbf59274ee39d716463fe72aa16)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.263732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.275387 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3575/7144] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:37 2022] DELETE /identity/v3/policies/98e0acd5ea40484a9f59505115ec95d0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:37.279572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.284051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.312838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79ce64-966c-4669-8b6c-b55b0a953bfe tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.316673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.323314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.323965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.324248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.324527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies/6cd0561c51504caca8146bb51ebe180a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.325861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=6cd0561c51504caca8146bb51ebe180a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.326423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.330599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.365066 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3576/7145] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:37 2022] DELETE /identity/v3/policies/6cd0561c51504caca8146bb51ebe180a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:37.369398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.380484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9776a51-f511-4b33-bda1-5d3e1b7be92b tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.396456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16', 'request_id': 'req-884e014b-ad41-459d-9481-889754cc8e14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.408624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16', 'request_id': 'req-884e014b-ad41-459d-9481-889754cc8e14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.422287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16', 'request_id': 'req-884e014b-ad41-459d-9481-889754cc8e14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.443269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] The token cache is being invalidate because project a3d92bbf59274ee39d716463fe72aa16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:37.443864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.444990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.452380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:UPDATE=1,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:37.454610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16', 'request_id': 'req-884e014b-ad41-459d-9481-889754cc8e14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.455897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16', 'request_id': 'req-884e014b-ad41-459d-9481-889754cc8e14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.457077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '361588b7cbc343f49bed86a5936235ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a211bfcb2ed4677993994c8176794b8', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.457862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a3d92bbf59274ee39d716463fe72aa16', 'request_id': 'req-884e014b-ad41-459d-9481-889754cc8e14'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.458553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.459028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.460182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] PATH_INFO: `/v3/policies/9bdc6bc8076146b0ba849c4ab8ade031` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.461425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=9bdc6bc8076146b0ba849c4ab8ade031)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.463490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.465918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:37.471785 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3570/7146] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:17:37 2022] DELETE /identity/v3/domains/a3d92bbf59274ee39d716463fe72aa16 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:37.479926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.482381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.489762 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3577/7147] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:37 2022] DELETE /identity/v3/policies/9bdc6bc8076146b0ba849c4ab8ade031 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:37.494160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.496177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.561898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e014b-ad41-459d-9481-889754cc8e14 tempest-TestDefaultProjectId-866114651 tempest-TestDefaultProjectId-866114651-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.567257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.567887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.568161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.568434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] PATH_INFO: `/v3/users/335e978e0c3241fcb9661c595f7d2cea` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.569776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=335e978e0c3241fcb9661c595f7d2cea)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.571719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.580336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308704e-0e1a-4a71-ab43-604f8beb920c tempest-PoliciesTestJSON-378012873 tempest-PoliciesTestJSON-378012873-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.587155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.587935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.588220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.588492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] PATH_INFO: `/v3/users/7e226bde935543849b0bf3ea52d38439` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.589705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e226bde935543849b0bf3ea52d38439)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.591672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.642411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335e978e0c3241fcb9661c595f7d2cea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.651977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335e978e0c3241fcb9661c595f7d2cea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.652994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335e978e0c3241fcb9661c595f7d2cea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.668065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335e978e0c3241fcb9661c595f7d2cea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.668709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1da33bc5-dc40-4bb3-8dd7-99f3c2deb269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335e978e0c3241fcb9661c595f7d2cea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.687108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e226bde935543849b0bf3ea52d38439'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.690684 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3571/7148] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:37 2022] DELETE /identity/v3/users/335e978e0c3241fcb9661c595f7d2cea => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:37.695876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.696377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.696978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.713432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e226bde935543849b0bf3ea52d38439'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.722666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e226bde935543849b0bf3ea52d38439'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.727287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:37.728519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.740259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e226bde935543849b0bf3ea52d38439'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.744675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8893ce9b-3790-481d-9644-92899e71d405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e226bde935543849b0bf3ea52d38439'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:37.778453 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3578/7149] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:37 2022] DELETE /identity/v3/users/7e226bde935543849b0bf3ea52d38439 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:37.784360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.784639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.784917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.810782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:37.812269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.817686 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3572/7150] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:37.828833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.830733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.846927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:37.900652 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3579/7151] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:37.900652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:37.901220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:37.917229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b581b-dcde-43cc-bd99-baf16df041af None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.918082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549b905-cbf4-4c8c-9510-c16da9f03e16 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:37.925170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.926075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.926471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.926855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] PATH_INFO: `/v3/users/200b4be60e9345bb94273b6ece3f55ba` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.929535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=200b4be60e9345bb94273b6ece3f55ba)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:37.931663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:37.939069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:37.981994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:37.982879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:37.983300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:37.983711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:37.998497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:38.000161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:38.012733 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3573/7152] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:38.013700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:38.014897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200b4be60e9345bb94273b6ece3f55ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.030047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084fa1f-7124-45b8-b269-10a8c096165f service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:38.036368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:38.037333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:38.037906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:38.038533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:38.040395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200b4be60e9345bb94273b6ece3f55ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.052237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200b4be60e9345bb94273b6ece3f55ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.056816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:38.059250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:38.062399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200b4be60e9345bb94273b6ece3f55ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.063465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1aca792c-880e-4dcf-b147-f63e6365cf1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200b4be60e9345bb94273b6ece3f55ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.072885 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3580/7153] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:37 2022] DELETE /identity/v3/users/200b4be60e9345bb94273b6ece3f55ba => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:38.073626 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3574/7154] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:38.077010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:38.077482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:38.077973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:38.103710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:38.106327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:38.188919 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3581/7155] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:38.220350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:38.234221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a007f-caf2-4e23-b7a6-4eb6165ae82d service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:38.296899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:38.297749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:38.298349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:38.298691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:38.311567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:38.313120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:38.326954 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3575/7156] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:38.638379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:38.640685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:38.664486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9a463-46c8-482a-a933-c5ad68af6124 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:38.670736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:38.672444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:38.672944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:38.673246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] PATH_INFO: `/v3/projects/70e1660220ea432f992abf725502fa6d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:38.675378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e1660220ea432f992abf725502fa6d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:38.679062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:38.768915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e1660220ea432f992abf725502fa6d', 'request_id': 'req-917ba6d7-8b97-45de-87e6-b91f66b34118'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.778773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e1660220ea432f992abf725502fa6d', 'request_id': 'req-917ba6d7-8b97-45de-87e6-b91f66b34118'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.788596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e1660220ea432f992abf725502fa6d', 'request_id': 'req-917ba6d7-8b97-45de-87e6-b91f66b34118'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.803416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:38.805295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:38.809610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] The token cache is being invalidate because project 70e1660220ea432f992abf725502fa6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:38.810156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.811048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:38.821563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0bfc3-d90a-443d-8079-0fd1b6938755 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:38.822334 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3582/7157] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:38 2022] DELETE /identity/v3/projects/70e1660220ea432f992abf725502fa6d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:38.838854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:38.841492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:38.859179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:38.944525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:38.957627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:38.958571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:38.958947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:38.962239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] PATH_INFO: `/v3/projects/d5e9807ab6b44595a6a800dfffd6e580` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:38.965041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e9807ab6b44595a6a800dfffd6e580)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:38.968455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:38.997820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ba6d7-8b97-45de-87e6-b91f66b34118 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:39.008553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:39.009694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:39.009694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:39.018178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] PATH_INFO: `/v3/users/01f68128de4245e99ce6b3c658490493` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:39.022412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f68128de4245e99ce6b3c658490493)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:39.024883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:39.058222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e9807ab6b44595a6a800dfffd6e580', 'request_id': 'req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.065679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e9807ab6b44595a6a800dfffd6e580', 'request_id': 'req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.076445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e9807ab6b44595a6a800dfffd6e580', 'request_id': 'req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.091327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] The token cache is being invalidate because project d5e9807ab6b44595a6a800dfffd6e580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:39.091668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.092432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.094127 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3576/7158] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:38 2022] DELETE /identity/v3/projects/d5e9807ab6b44595a6a800dfffd6e580 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:39.095299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:39.098172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:39.098641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f68128de4245e99ce6b3c658490493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.117320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f68128de4245e99ce6b3c658490493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.127933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f68128de4245e99ce6b3c658490493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.144153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f68128de4245e99ce6b3c658490493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.144839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f68128de4245e99ce6b3c658490493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.154804 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3583/7159] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:38 2022] DELETE /identity/v3/users/01f68128de4245e99ce6b3c658490493 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:39.156915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:39.173683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:39.196066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144bb9c-2b53-4834-afb9-99e8d0d3fa72 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:39.294166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:39.295416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:39.296151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:39.296612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] PATH_INFO: `/v3/projects/dfc79269648b4ea4bb40ffaf5f9766ce` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:39.298310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc79269648b4ea4bb40ffaf5f9766ce)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:39.301224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:39.312309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b82fa4-143b-42ac-8d09-330b59bfa4bd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:39.330158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:39.330824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:39.331112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:39.331396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] PATH_INFO: `/v3/users/361588b7cbc343f49bed86a5936235ea` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:39.332609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361588b7cbc343f49bed86a5936235ea)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:39.335176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:39.373333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:39.398200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:39.403321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc79269648b4ea4bb40ffaf5f9766ce', 'request_id': 'req-269d7ac3-46d3-48c6-a348-d3822828e178'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.432207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361588b7cbc343f49bed86a5936235ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.432568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc79269648b4ea4bb40ffaf5f9766ce', 'request_id': 'req-269d7ac3-46d3-48c6-a348-d3822828e178'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.446839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc79269648b4ea4bb40ffaf5f9766ce', 'request_id': 'req-269d7ac3-46d3-48c6-a348-d3822828e178'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.463903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361588b7cbc343f49bed86a5936235ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.482246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] The token cache is being invalidate because project dfc79269648b4ea4bb40ffaf5f9766ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:39.483115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.485426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.488305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361588b7cbc343f49bed86a5936235ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.489719 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3577/7160] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:39 2022] DELETE /identity/v3/projects/dfc79269648b4ea4bb40ffaf5f9766ce => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:39.495386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:39.497083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:39.509247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361588b7cbc343f49bed86a5936235ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.509400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361588b7cbc343f49bed86a5936235ea'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.520926 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3584/7161] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:39 2022] DELETE /identity/v3/users/361588b7cbc343f49bed86a5936235ea => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:39.617887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d7ac3-46d3-48c6-a348-d3822828e178 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:39.745631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:39.746524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:39.746811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:39.747085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] PATH_INFO: `/v3/users/d8e350f51a364f2f985ef0d13d9ee9dd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:39.749641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e350f51a364f2f985ef0d13d9ee9dd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:39.751999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:39.870331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e350f51a364f2f985ef0d13d9ee9dd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.890624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e350f51a364f2f985ef0d13d9ee9dd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.891486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e350f51a364f2f985ef0d13d9ee9dd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.908484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e350f51a364f2f985ef0d13d9ee9dd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.909320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e350f51a364f2f985ef0d13d9ee9dd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:39.922708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:39.950475 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3578/7162] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:39 2022] DELETE /identity/v3/users/d8e350f51a364f2f985ef0d13d9ee9dd => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:39.956622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:39.958661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:39.996556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf723f0e-20c4-4a14-8f63-77133e2d71d9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:40.088425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:40.089740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.184638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:40.185625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.186202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.186674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] PATH_INFO: `/v3/projects/e38a51846ab44195842b61de62db074d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.188194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38a51846ab44195842b61de62db074d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:40.191202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:40.230272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa782b4-6f76-446d-86d6-47a09d6ccfe5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:40.235794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:40.236432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.236715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.237106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] PATH_INFO: `/v3/projects/1a211bfcb2ed4677993994c8176794b8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.238175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a211bfcb2ed4677993994c8176794b8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:40.240581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:40.257050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38a51846ab44195842b61de62db074d', 'request_id': 'req-99c9f34d-c2cb-4fd7-9842-3002fc39e383'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.266052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38a51846ab44195842b61de62db074d', 'request_id': 'req-99c9f34d-c2cb-4fd7-9842-3002fc39e383'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.280523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38a51846ab44195842b61de62db074d', 'request_id': 'req-99c9f34d-c2cb-4fd7-9842-3002fc39e383'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.305507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] The token cache is being invalidate because project e38a51846ab44195842b61de62db074d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:40.306432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.307924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-99c9f34d-c2cb-4fd7-9842-3002fc39e383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.309531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a211bfcb2ed4677993994c8176794b8', 'request_id': 'req-87e88d90-4a74-4b5f-af02-6bd165a0dd21'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.314397 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3585/7163] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:39 2022] DELETE /identity/v3/projects/e38a51846ab44195842b61de62db074d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:40.320051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c12c06-c24e-4992-ba25-591f3cbc7ed3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.321099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c12c06-c24e-4992-ba25-591f3cbc7ed3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.321394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c12c06-c24e-4992-ba25-591f3cbc7ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.324105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a211bfcb2ed4677993994c8176794b8', 'request_id': 'req-87e88d90-4a74-4b5f-af02-6bd165a0dd21'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.344967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a211bfcb2ed4677993994c8176794b8', 'request_id': 'req-87e88d90-4a74-4b5f-af02-6bd165a0dd21'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.358117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-87c12c06-c24e-4992-ba25-591f3cbc7ed3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:40.359541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87c12c06-c24e-4992-ba25-591f3cbc7ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.364229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] The token cache is being invalidate because project 1a211bfcb2ed4677993994c8176794b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:40.364948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.366974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-87e88d90-4a74-4b5f-af02-6bd165a0dd21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.370194 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3579/7164] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:40 2022] DELETE /identity/v3/projects/1a211bfcb2ed4677993994c8176794b8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:40.377382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d71a0-207f-4310-8877-9eb0c52fe20e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.378299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d71a0-207f-4310-8877-9eb0c52fe20e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.379887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d71a0-207f-4310-8877-9eb0c52fe20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.411494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:40.413052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c98d71a0-207f-4310-8877-9eb0c52fe20e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:40.415377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c98d71a0-207f-4310-8877-9eb0c52fe20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.483310 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3586/7165] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:40.492430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903bd8-f53b-4858-946c-73790ca44bcc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.492730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903bd8-f53b-4858-946c-73790ca44bcc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.493003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5903bd8-f53b-4858-946c-73790ca44bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.517524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e5903bd8-f53b-4858-946c-73790ca44bcc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:40.519188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5903bd8-f53b-4858-946c-73790ca44bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.528385 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3587/7166] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:40.533599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5903bd8-f53b-4858-946c-73790ca44bcc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:40.550670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5903bd8-f53b-4858-946c-73790ca44bcc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:40.562133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:40.564145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.565667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.566065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.566836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:40.567330 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3580/7167] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:40.567510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:40.570041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:40.576437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.577078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.577078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.611860 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3588/7168] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:40.614053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:40.617627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:40.618935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.628623 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3581/7169] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:40.629478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:40.633745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.645598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5724d9-992c-4f0e-b1bd-43512c467cf2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:40.649198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b08551-ea8d-43c3-b6d0-d567eee88f04 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:40.655281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:40.656253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.656718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.657170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] PATH_INFO: `/v3/projects/35b60639201640eaaa4b0d440184d7e8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.658519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b60639201640eaaa4b0d440184d7e8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:40.660668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:40.664986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:40.665613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.665921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.666198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.666972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:40.669010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:40.675851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:40.733603 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3589/7170] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:40 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:40.735234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:40.755192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b60639201640eaaa4b0d440184d7e8', 'request_id': 'req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.770914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b60639201640eaaa4b0d440184d7e8', 'request_id': 'req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.790131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b60639201640eaaa4b0d440184d7e8', 'request_id': 'req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.807310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] The token cache is being invalidate because project 35b60639201640eaaa4b0d440184d7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:40.807930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.810232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:40.813562 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3582/7171] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:40 2022] DELETE /identity/v3/projects/35b60639201640eaaa4b0d440184d7e8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:40.814388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:40.818782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.882162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0905-4d80-4173-b2dd-afb9baff58d9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:40.893675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f043e0-d0d6-4fb7-9867-e81bd899c326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:40.894198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f043e0-d0d6-4fb7-9867-e81bd899c326 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.894515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f043e0-d0d6-4fb7-9867-e81bd899c326 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.894791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f043e0-d0d6-4fb7-9867-e81bd899c326 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.896061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f043e0-d0d6-4fb7-9867-e81bd899c326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:40.896331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f043e0-d0d6-4fb7-9867-e81bd899c326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:40.898574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f043e0-d0d6-4fb7-9867-e81bd899c326 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:40.934362 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3590/7172] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:40.936601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.936874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.938634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.971288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677fcd-f910-4b6e-abd5-e6b3bc5f5565 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:40.978499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:40.983018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:40.988588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3f3a9-8a81-4a2e-8905-3b2a663f98b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:40.989446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f3a9-8a81-4a2e-8905-3b2a663f98b8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:40.989741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f3a9-8a81-4a2e-8905-3b2a663f98b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:40.990283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f3a9-8a81-4a2e-8905-3b2a663f98b8 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:40.991378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3f3a9-8a81-4a2e-8905-3b2a663f98b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:40.994922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3f3a9-8a81-4a2e-8905-3b2a663f98b8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.002238 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3591/7173] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:41.003650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.004999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:41.039869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7aea2e-daf7-4515-a8b5-301ecca99c02 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.054599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.055499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.056904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.057491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.058746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.061205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3583/7174] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:40 2022] POST /identity/v3/users => generated 285 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:41.064604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.080133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74fab-15dd-48d8-9660-78dd2be0df3b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.081359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74fab-15dd-48d8-9660-78dd2be0df3b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.085000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e74fab-15dd-48d8-9660-78dd2be0df3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.133438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-17e74fab-15dd-48d8-9660-78dd2be0df3b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:41.134954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-17e74fab-15dd-48d8-9660-78dd2be0df3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:41.138384 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3592/7175] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:41 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:41.143091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.289444 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3584/7176] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:41.290200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74fab-15dd-48d8-9660-78dd2be0df3b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.313174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c944a3-122e-4619-be07-4aa651bed8da admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.314662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e74fab-15dd-48d8-9660-78dd2be0df3b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.320329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.321608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.322571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.323311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.324843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.325688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.326430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.327012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.329057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.329901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.331714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.335159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.336993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:41.386185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3593/7177] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:41.387598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.418636 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3585/7178] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:41 2022] POST /identity/v3/users => generated 285 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:41.421056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.423524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba692a-8c80-4873-905d-3b3fad85c935 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.429801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.430573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.430928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.431281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.432299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:41.432696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.435331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.442770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5314ea-3e56-47df-9f9d-a539992fc440 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.459021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.460844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.461617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.462517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.463433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.466326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.468944 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3594/7179] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:41.474170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.476844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:41.486537 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3586/7180] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:41.487697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.492063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66b392-e071-4760-ba97-961c06d44edd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.501210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.502240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.502469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ba4be-bf42-4314-95d7-d3014f712a4c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.503775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.505319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.513538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.518943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.519580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.519866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.520141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.520788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.523368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.523817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.546178 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3595/7181] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:41.547737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.564397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0caa8-4258-4e3a-832f-d5c301d97b1d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.572957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.574065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.574662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.575409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] PATH_INFO: `/v3/projects/008348cf6bbc4776853ab733ac2927e0/users/ea03554527fc4a3bb49e9d78f0849a03/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.594821 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3587/7182] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:41 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:41.594821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.603746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=008348cf6bbc4776853ab733ac2927e0, user_id=ea03554527fc4a3bb49e9d78f0849a03, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.607458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.647855 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3596/7183] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:41 2022] PUT /identity/v3/projects/008348cf6bbc4776853ab733ac2927e0/users/ea03554527fc4a3bb49e9d78f0849a03/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:41.648887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.728331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0a84d-ee35-4488-9adf-1913229a3ea0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.792559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a2dd0-e052-4eba-8e3d-9b64e38c61ee admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.802085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.804680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.805288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.805793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.806846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.811305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.849330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eda39-ccdb-4269-b6b0-07d01a253ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.851046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eda39-ccdb-4269-b6b0-07d01a253ccf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.851316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eda39-ccdb-4269-b6b0-07d01a253ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.851685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eda39-ccdb-4269-b6b0-07d01a253ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.852354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80eda39-ccdb-4269-b6b0-07d01a253ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:41.858021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80eda39-ccdb-4269-b6b0-07d01a253ccf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:41.889561 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3588/7184] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:41.894701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58bf84-4fc2-4bf0-8214-32b98d84202c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.894701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58bf84-4fc2-4bf0-8214-32b98d84202c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.894701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58bf84-4fc2-4bf0-8214-32b98d84202c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:41.895049 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3597/7185] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:41 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:41.901204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:41.956308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e853f-acb2-479f-b8c5-80a07e2966b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:41.979293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ce58bf84-4fc2-4bf0-8214-32b98d84202c None None] MFA Rules not processed for user `ea03554527fc4a3bb49e9d78f0849a03`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:41.983627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ce58bf84-4fc2-4bf0-8214-32b98d84202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:41.986877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:41.988215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:41.988215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:41.988215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] PATH_INFO: `/v3/projects/7ce320298cc24ce2824406f42ac30e3e/users/0c3b56a6d5664fa89f1e50d649294f79/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.050167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce320298cc24ce2824406f42ac30e3e, user_id=0c3b56a6d5664fa89f1e50d649294f79, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.053923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.097295 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3598/7186] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:41 2022] PUT /identity/v3/projects/7ce320298cc24ce2824406f42ac30e3e/users/0c3b56a6d5664fa89f1e50d649294f79/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:42.097295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.180616 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3589/7187] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:42.181852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b838865-2446-4120-b00b-6134443489d9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.183374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b838865-2446-4120-b00b-6134443489d9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.183757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b838865-2446-4120-b00b-6134443489d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.219993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c169d-0956-4a72-9d37-7b257952ce50 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.226932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.227567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.227850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.228120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.228826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.231446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.248347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3599/7188] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:42.248347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.259720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:42.261820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f445752-6013-4872-aed3-acded88d5815 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.268217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.268843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.269122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.269396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.270063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.272760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.279746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:42.305231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2b838865-2446-4120-b00b-6134443489d9 None None] MFA Rules not processed for user `0c3b56a6d5664fa89f1e50d649294f79`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:42.307081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b838865-2446-4120-b00b-6134443489d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:42.335471 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3600/7189] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:42 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:42.335471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.411318 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3590/7190] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:42.412412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b838865-2446-4120-b00b-6134443489d9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.430420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d98b6-523c-459b-a519-d45dc4446b16 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.432969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b838865-2446-4120-b00b-6134443489d9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.438623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.439252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.439518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.439789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.440419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.442642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.447700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.448528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.449190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.449913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.450861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.453009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.466190 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3601/7191] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:42.467307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.477856 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3591/7192] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:42 2022] POST /identity/v3/users => generated 292 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:42.477856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.498728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41ff9-c5d2-4bb5-8bda-b0a6f584d2b4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.507324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.507999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.508281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.508562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] PATH_INFO: `/v3/projects/d5373fd3d1734191ab3dd5886026cb02/users/15eab44adcdf4bbe808a45784393d828/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.534619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5373fd3d1734191ab3dd5886026cb02, user_id=15eab44adcdf4bbe808a45784393d828, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.538305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.564738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc767a3-3c3c-4bee-8456-d62c4b620917 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.565532 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3592/7193] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:42 2022] PUT /identity/v3/projects/d5373fd3d1734191ab3dd5886026cb02/users/15eab44adcdf4bbe808a45784393d828/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:42.566573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.672211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.672898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.673208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.673606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.674308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:create_application_credential(user_id=0c3b56a6d5664fa89f1e50d649294f79)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.675517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.716660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e467c-8901-4d56-94f2-458664956c49 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.729449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139031d6-9c1d-42da-8b34-f2a0c474dd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.730284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139031d6-9c1d-42da-8b34-f2a0c474dd1b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.730612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139031d6-9c1d-42da-8b34-f2a0c474dd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.731148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139031d6-9c1d-42da-8b34-f2a0c474dd1b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.731812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139031d6-9c1d-42da-8b34-f2a0c474dd1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.733450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139031d6-9c1d-42da-8b34-f2a0c474dd1b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.745928 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3593/7194] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:42.747435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9eb9-3ac2-4b4d-99ad-1ec443518a58 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.747731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9eb9-3ac2-4b4d-99ad-1ec443518a58 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.748011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9eb9-3ac2-4b4d-99ad-1ec443518a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.761794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:42.790008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d35d9eb9-3ac2-4b4d-99ad-1ec443518a58 None None] MFA Rules not processed for user `15eab44adcdf4bbe808a45784393d828`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:42.791418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d35d9eb9-3ac2-4b4d-99ad-1ec443518a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:42.807796 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3602/7195] 10.222.0.33 () {60 vars in 1315 bytes} [Sat Sep 10 21:17:42 2022] POST /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials => generated 920 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:42.808558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.821487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84ff4d-4c85-4eb5-b8d5-1b2fd23313f0 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.826478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.827214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.827563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.827904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] PATH_INFO: `/v3/projects/3c4e94f65f10447a89daa284e4aa4350/users/2aa88731c70049db87f16e2325b02643/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.849203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:42.860692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4e94f65f10447a89daa284e4aa4350, user_id=2aa88731c70049db87f16e2325b02643, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.864038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.884016 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3594/7196] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:42.884016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d9eb9-3ac2-4b4d-99ad-1ec443518a58 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.884300 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3603/7197] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:42 2022] PUT /identity/v3/projects/3c4e94f65f10447a89daa284e4aa4350/users/2aa88731c70049db87f16e2325b02643/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:42.885433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:42.894580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d9eb9-3ac2-4b4d-99ad-1ec443518a58 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.974128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33100fd4-7de9-4e19-a1de-58dd86ae86b2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:42.977966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.978853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.979197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.979540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules/344caa667a9a4887aece79a796d1b476` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.981175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:delete_access_rule(user_id=0c3b56a6d5664fa89f1e50d649294f79, access_rule_id=344caa667a9a4887aece79a796d1b476)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.982542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:42.982778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:42.983221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:42.983580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:42.983863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:42.984584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:42.986525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.002160 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3604/7198] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:43.003924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.016067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3bf54-b643-44a3-90be-43b92aa79d2c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.021161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.021815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.022166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.022450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.023061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.025051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application session.delete(ref) Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application yield resource Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application next(self.gen) Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 21:17:43.047970 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise self.value Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application session.commit() Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.session.flush() Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 21:17:43.049176 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application compat.raise_( Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._synchronize( Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise AssertionError( Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Sep 10 21:17:43.050282 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:17:43.051293 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3595/7199] 10.222.0.33 () {58 vars in 1362 bytes} [Sat Sep 10 21:17:42 2022] DELETE /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules/344caa667a9a4887aece79a796d1b476 => generated 89 bytes in 170 msecs (HTTP/1.1 403) 5 headers in 183 bytes (1 switches on core 0) Sep 10 21:17:43.054494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.066433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2aa7e-c519-4161-bba2-a53282456592 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.067812 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3605/7200] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:43 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:43.068919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.143275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.144114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.144447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.144845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] PATH_INFO: `/v3/projects/3c4e94f65f10447a89daa284e4aa4350/users/2aa88731c70049db87f16e2325b02643/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.145159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86835e21-bd01-4c95-900a-81a625277ea6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.147109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4e94f65f10447a89daa284e4aa4350, user_id=2aa88731c70049db87f16e2325b02643, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.150446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.150881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.151558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.151820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.152101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials/a33108164b28406a8d4c37a0c7325f85` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.166068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:delete_application_credential(user_id=0c3b56a6d5664fa89f1e50d649294f79, application_credential_id=a33108164b28406a8d4c37a0c7325f85)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.169720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.173915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3596/7201] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:43 2022] PUT /identity/v3/projects/3c4e94f65f10447a89daa284e4aa4350/users/2aa88731c70049db87f16e2325b02643/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:43.175098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.191663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:43.206261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3606/7202] 10.222.0.33 () {58 vars in 1395 bytes} [Sat Sep 10 21:17:43 2022] DELETE /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials/a33108164b28406a8d4c37a0c7325f85 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:43.214059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.236656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa3356-6efe-42c1-8759-38cb2540e22b tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.279856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05fa8-7f31-4041-90a4-8bac55e74768 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.285360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.286348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.286831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.287242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.287981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.290100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.323429 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3597/7203] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:43 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:43.323429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.333241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:43.334789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd93f3-1a4a-49ff-b22c-bbd50ff601c1 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.336878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd93f3-1a4a-49ff-b22c-bbd50ff601c1 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.337088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd93f3-1a4a-49ff-b22c-bbd50ff601c1 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.337352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd93f3-1a4a-49ff-b22c-bbd50ff601c1 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.343037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd93f3-1a4a-49ff-b22c-bbd50ff601c1 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:list_access_rules(user_id=0c3b56a6d5664fa89f1e50d649294f79)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.343037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd93f3-1a4a-49ff-b22c-bbd50ff601c1 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.351348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:43.353889 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3607/7204] 10.222.0.33 () {58 vars in 1260 bytes} [Sat Sep 10 21:17:43 2022] GET /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules => generated 369 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:43.359639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.359931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.360211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.399490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] MFA Rules not processed for user `2aa88731c70049db87f16e2325b02643`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:43.400711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:43.426588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aeab3-8ef6-4e10-99ad-995f2a457632 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.432310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.432921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.433193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.433466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.435260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:43.449727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.451349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.464301 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3598/7205] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:43.465103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.474108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3608/7206] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:43.474975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.475591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6e865-eb9d-4a27-85ed-5677415a1856 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.476640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:43.480595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.481445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.481729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.482034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.482670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.484250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.494001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13baa1-1657-4647-9fa2-1e1480482589 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.498494 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3599/7207] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:43.498494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.501981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.502619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.502901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.503172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] PATH_INFO: `/v3/users/eac0a8d66da042e096cab5c715f40a27` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.504325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac0a8d66da042e096cab5c715f40a27)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.506884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.508853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058d10e-d809-44c6-945c-a9665f7b2dbe admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.512966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.513597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.513904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.514179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules/344caa667a9a4887aece79a796d1b476` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.515302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:delete_access_rule(user_id=0c3b56a6d5664fa89f1e50d649294f79, access_rule_id=344caa667a9a4887aece79a796d1b476)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.516654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.536812 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3600/7208] 10.222.0.33 () {58 vars in 1362 bytes} [Sat Sep 10 21:17:43 2022] DELETE /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules/344caa667a9a4887aece79a796d1b476 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:43.538278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.551273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59696f1d-634f-4afa-94d3-efba5895c251 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.556352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.556956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.557244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.557525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.558224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.559257 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:43.559257 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:17:43.559257 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:43.560150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.562016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac0a8d66da042e096cab5c715f40a27'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:43.577062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac0a8d66da042e096cab5c715f40a27'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:43.578238 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3601/7209] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:43 2022] POST /identity/v3/services => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:43.578980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.585297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac0a8d66da042e096cab5c715f40a27'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:43.593492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac0a8d66da042e096cab5c715f40a27'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:43.594383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac0a8d66da042e096cab5c715f40a27'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:43.596010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:43.602419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c6d9-b35b-43a4-9bbe-09df8bb6b5eb tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.603258 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3609/7210] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:43 2022] DELETE /identity/v3/users/eac0a8d66da042e096cab5c715f40a27 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:43.604091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.606490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:43.689953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.689953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.689953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.689953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] PATH_INFO: `/v3/projects/f22854352a7b46d495aac3802c21a47b/users/2f83cde752df4d989657d416504353c4/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.702861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22854352a7b46d495aac3802c21a47b, user_id=2f83cde752df4d989657d416504353c4, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.706226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.707197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961abcee-b200-4047-9609-3853e42a1a70 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.731871 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3602/7211] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:43 2022] PUT /identity/v3/projects/f22854352a7b46d495aac3802c21a47b/users/2f83cde752df4d989657d416504353c4/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:43.732707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.738154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.738718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.738991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.739352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.740694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:43.743715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:list_access_rules(user_id=0c3b56a6d5664fa89f1e50d649294f79)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.746414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.756739 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3610/7212] 10.222.0.33 () {58 vars in 1260 bytes} [Sat Sep 10 21:17:43 2022] GET /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules => generated 158 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:43.760260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.763215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:43.782011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefad16-e9dd-432a-b890-511723a3e9ca tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.791578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:43.848542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46474215-b35b-4e25-9cbc-a2947df889b2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:43.854665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.855305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.855579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.856282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.856956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.861848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.884918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:43.885571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:43.885868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:43.886130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:43.886937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:43.888325 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1333218139. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:43.888325 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:43.888325 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:43.888609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:43.903830 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3611/7213] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:43.903830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:43.956150 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3603/7214] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:17:43 2022] POST /identity/v3/endpoints => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:43.958153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.003585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0f712-9a0d-43e4-ba41-7a3404532fae tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.017175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.018318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.018616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.018895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] PATH_INFO: `/v3/projects/f22854352a7b46d495aac3802c21a47b/users/2f83cde752df4d989657d416504353c4/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.021288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22854352a7b46d495aac3802c21a47b, user_id=2f83cde752df4d989657d416504353c4, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.024438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.027831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde3dc2-bc78-44a0-81b9-8a40ad3eabd4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.043482 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3604/7215] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:43 2022] PUT /identity/v3/projects/f22854352a7b46d495aac3802c21a47b/users/2f83cde752df4d989657d416504353c4/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:44.048629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.056106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.056879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.057178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.057681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.058588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:create_application_credential(user_id=0c3b56a6d5664fa89f1e50d649294f79)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.060758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.100890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:44.179080 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3612/7216] 10.222.0.33 () {60 vars in 1315 bytes} [Sat Sep 10 21:17:43 2022] POST /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials => generated 920 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:44.180239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.182344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.191585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95445c7f-f219-4501-8136-67a8049ac4c1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.197434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48f3a-87a5-48a2-9b15-7ed11f8a374e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.198525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48f3a-87a5-48a2-9b15-7ed11f8a374e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.198972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48f3a-87a5-48a2-9b15-7ed11f8a374e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.199256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48f3a-87a5-48a2-9b15-7ed11f8a374e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1333218139` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.199840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d48f3a-87a5-48a2-9b15-7ed11f8a374e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1333218139)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.200921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d48f3a-87a5-48a2-9b15-7ed11f8a374e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.202758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c75a95-771c-4c6d-b3a1-e49c28467f3a tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.206125 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3605/7217] 10.222.0.33 () {58 vars in 1206 bytes} [Sat Sep 10 21:17:44 2022] GET /identity/v3/regions/tempest-region-1333218139 => generated 180 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:44.207431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118135c-6596-4995-bcf3-f652eaac02ff None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.207769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118135c-6596-4995-bcf3-f652eaac02ff None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.208380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118135c-6596-4995-bcf3-f652eaac02ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.253539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8118135c-6596-4995-bcf3-f652eaac02ff None None] MFA Rules not processed for user `2f83cde752df4d989657d416504353c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:44.254815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8118135c-6596-4995-bcf3-f652eaac02ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.296081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.296778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.297885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.297885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] PATH_INFO: `/v3/projects/3d9f98a3f93d409a94ee87eec97206c1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.298843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9f98a3f93d409a94ee87eec97206c1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.300823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.314848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:44.348397 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3606/7218] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3647 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:44.348397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118135c-6596-4995-bcf3-f652eaac02ff None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.388616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9f98a3f93d409a94ee87eec97206c1', 'request_id': 'req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.395006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9f98a3f93d409a94ee87eec97206c1', 'request_id': 'req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.404627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9f98a3f93d409a94ee87eec97206c1', 'request_id': 'req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.419974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] The token cache is being invalidate because project 3d9f98a3f93d409a94ee87eec97206c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:44.422292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.422987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.424729 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3613/7219] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:44 2022] DELETE /identity/v3/projects/3d9f98a3f93d409a94ee87eec97206c1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:44.425498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.428445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.450458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118135c-6596-4995-bcf3-f652eaac02ff None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.514439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:44.519785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.520726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.521012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.521922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.523509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:list_access_rules(user_id=0c3b56a6d5664fa89f1e50d649294f79)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.525223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.535033 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3607/7220] 10.222.0.33 () {58 vars in 1260 bytes} [Sat Sep 10 21:17:44 2022] GET /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules => generated 369 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:44.535033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.535335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.541198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02feedd-1f37-4f0a-9ac8-6cbec46a773f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.547328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e87162-a522-4545-be67-eeb8bceb8d72 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.548224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e87162-a522-4545-be67-eeb8bceb8d72 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.548686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e87162-a522-4545-be67-eeb8bceb8d72 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.548907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e87162-a522-4545-be67-eeb8bceb8d72 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.549633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e87162-a522-4545-be67-eeb8bceb8d72 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.552605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e87162-a522-4545-be67-eeb8bceb8d72 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.575156 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3614/7221] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:44 2022] POST /identity/v3/services => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:44.575156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.575156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.575156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.595725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:44.597053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.625587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29984fe4-0395-48e4-b31c-249fc288a7e2 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.641610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.642248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.643120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.643404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.645182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.676397 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3615/7222] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3688 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:44.677180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.679215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.696425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:44.703392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0cf5-0a5f-461b-b7b1-88430d08622f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.708073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf604d-ff63-4351-88b0-a5c2b9d88336 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.708756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf604d-ff63-4351-88b0-a5c2b9d88336 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.709096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf604d-ff63-4351-88b0-a5c2b9d88336 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.709440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf604d-ff63-4351-88b0-a5c2b9d88336 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.710137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf604d-ff63-4351-88b0-a5c2b9d88336 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:create_application_credential(user_id=0c3b56a6d5664fa89f1e50d649294f79)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.711322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf604d-ff63-4351-88b0-a5c2b9d88336 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.722902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.724336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.745394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3608/7223] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:44 2022] GET /identity/v3/auth/tokens => generated 3780 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:44.748008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.769458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315eeb0-7ce3-4ee2-8870-9875a9c69d93 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.775863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.776483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.776806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.777092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.777770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.779419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.801131 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1186730759. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:44.801131 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:44.801131 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:44.806008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3616/7224] 10.222.0.33 () {60 vars in 1315 bytes} [Sat Sep 10 21:17:44 2022] POST /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/application_credentials => generated 921 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:44.807863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.808304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.808594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.830731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:44.832191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.834949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3609/7225] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:17:44 2022] POST /identity/v3/endpoints => generated 363 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:44.835761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.852405 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3617/7226] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:44 2022] POST /identity/v3/auth/tokens => generated 3873 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:44.853256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.854795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.860046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2a4bd-032a-4ab3-a9c8-25d520286ad8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.864315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c3b56a6d5664fa89f1e50d649294f79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ce320298cc24ce2824406f42ac30e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.864948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.865307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.865586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules/d1fb3c39098f4c8abcf87fa37cc5f42e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.866759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorizing `identity:get_access_rule(user_id=0c3b56a6d5664fa89f1e50d649294f79, access_rule_id=d1fb3c39098f4c8abcf87fa37cc5f42e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.867310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7c945-5471-4f79-a731-2aba4eafde60 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.868262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.872211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.872827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.873113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.873392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] PATH_INFO: `/v3/users/15eab44adcdf4bbe808a45784393d828` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.874447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3610/7227] 10.222.0.33 () {58 vars in 1359 bytes} [Sat Sep 10 21:17:44 2022] GET /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79/access_rules/d1fb3c39098f4c8abcf87fa37cc5f42e => generated 229 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:44.874625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=15eab44adcdf4bbe808a45784393d828)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.875249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.876730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.897122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94436da6-a6ba-46df-a80d-1d8b322bb932 tempest-AccessRulesV3Test-408593066 tempest-AccessRulesV3Test-408593066-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.902584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.903314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.903607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.903908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1186730759` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.904543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1186730759)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.905625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.907735 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3611/7228] 10.222.0.33 () {58 vars in 1206 bytes} [Sat Sep 10 21:17:44 2022] GET /identity/v3/regions/tempest-region-1186730759 => generated 180 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:44.908530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.918555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:44.921486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbdc6c-61c4-4b32-bdc4-c0529637f4c7 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:44.927064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:44.927708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:44.927991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:44.928273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:44.928917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:44.929225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:44.930849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:44.935710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15eab44adcdf4bbe808a45784393d828'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.952579 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3612/7229] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:44.952778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15eab44adcdf4bbe808a45784393d828'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.952880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.954519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.958597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15eab44adcdf4bbe808a45784393d828'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.970416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15eab44adcdf4bbe808a45784393d828'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.971060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15eab44adcdf4bbe808a45784393d828'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:44.981263 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3618/7230] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:44 2022] DELETE /identity/v3/users/15eab44adcdf4bbe808a45784393d828 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:44.982180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:44.983578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:44.988893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767b808-95f4-48ec-9a8a-5f8d7310b16a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.065297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.066891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.067243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.067506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] PATH_INFO: `/v3/users/0c3b56a6d5664fa89f1e50d649294f79` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.069157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c3b56a6d5664fa89f1e50d649294f79)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.071707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.075330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78282ed-4de6-48c5-8b2d-63d298d827ab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.081120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.081733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.082593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.082881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.083548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.086220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.114155 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1890051280. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:45.114155 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:45.114155 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:45.123417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:45.135574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c3b56a6d5664fa89f1e50d649294f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:45.143886 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3619/7231] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:17:44 2022] POST /identity/v3/endpoints => generated 360 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:45.143886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.143886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.146480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c3b56a6d5664fa89f1e50d649294f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:45.147695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c3b56a6d5664fa89f1e50d649294f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:45.155959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c3b56a6d5664fa89f1e50d649294f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:45.156724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c3b56a6d5664fa89f1e50d649294f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:45.170647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e094-a956-4dbc-98ac-12b529d7a086 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.175826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c741d-33fa-45be-ba0d-746d1ba2153a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.176476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c741d-33fa-45be-ba0d-746d1ba2153a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.176782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c741d-33fa-45be-ba0d-746d1ba2153a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.177059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c741d-33fa-45be-ba0d-746d1ba2153a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.177742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c741d-33fa-45be-ba0d-746d1ba2153a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.179821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c741d-33fa-45be-ba0d-746d1ba2153a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.206741 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3620/7232] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:45 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:45.208152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.208394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3613/7233] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:44 2022] DELETE /identity/v3/users/0c3b56a6d5664fa89f1e50d649294f79 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:45.211954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.211954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.213073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.214456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.232551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:45.233762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.295806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:45.296067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3d19f-0a25-4ff5-997a-d0d4d6b1e36b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.319403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688d5d3-d7c5-4cf2-aef0-0c215197ad97 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.320126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d5d3-d7c5-4cf2-aef0-0c215197ad97 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.320410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d5d3-d7c5-4cf2-aef0-0c215197ad97 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.320687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d5d3-d7c5-4cf2-aef0-0c215197ad97 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1890051280` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.321326 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3621/7234] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:45 2022] POST /identity/v3/auth/tokens => generated 4057 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:45.321547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d5d3-d7c5-4cf2-aef0-0c215197ad97 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1890051280)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.322651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.322763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d5d3-d7c5-4cf2-aef0-0c215197ad97 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.325478 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3614/7235] 10.222.0.33 () {58 vars in 1206 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/regions/tempest-region-1890051280 => generated 180 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:45.326725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8f785-1a13-483c-ac4a-729002f6d0fa None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.328418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8f785-1a13-483c-ac4a-729002f6d0fa None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.328701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8f785-1a13-483c-ac4a-729002f6d0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.331942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.346462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2b71d-29e7-44d8-89de-c21b5cc4d9d7 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.349589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-eeb8f785-1a13-483c-ac4a-729002f6d0fa None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:45.351221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eeb8f785-1a13-483c-ac4a-729002f6d0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.351818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.352651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.353359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.353570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.354501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.356839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.363394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3615/7236] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:45 2022] POST /identity/v3/auth/tokens => generated 4057 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:45.364471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8f785-1a13-483c-ac4a-729002f6d0fa None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.380543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8f785-1a13-483c-ac4a-729002f6d0fa None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.385023 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3622/7237] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:45 2022] POST /identity/v3/users => generated 287 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:45.386446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.386768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.387058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.387355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.387642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.388522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.389371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.389554 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:45.389554 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:17:45.389554 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:45.390490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.402121 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3616/7238] 10.222.0.33 () {58 vars in 1134 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/endpoints => generated 4398 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:45.402121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.402562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.470941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9b9cd-9587-4af4-b843-37f5fccbc750 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.471697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926f2a4-03f2-4ee9-8f6a-be7b49a8a275 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.476170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.476380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.476905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.477276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.477394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.477600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.477717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.478048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.488806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.488980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.490662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.490781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.501738 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3623/7239] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/auth/tokens => generated 4057 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:45.502609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.515613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b27f9-6199-4710-8130-00a94f2cc927 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.518029 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3617/7240] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/auth/tokens => generated 4057 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:45.522552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.523978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.524735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.524925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.525187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.526017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.527421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.538681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2184e5-20a2-4ba8-883d-524c61f90b55 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.539480 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3624/7241] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:45.543429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.546543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.548040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.549250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.549973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints/1ed341b6d3504b1a82dd7527cdc057f5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.555644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:45.559604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=1ed341b6d3504b1a82dd7527cdc057f5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.562195 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:45.562195 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:17:45.562195 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:45.563973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.566693 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3618/7242] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/endpoints/1ed341b6d3504b1a82dd7527cdc057f5 => generated 360 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:45.571324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5a5f-30cf-4ad2-b96f-af2edb985c03 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.584534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.585731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.586867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.587442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.588658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.589369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.591914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.602575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c819b-7042-4c18-a54f-c0dcce0f1eec tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.614621 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3625/7243] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:45.616133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.616448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.617422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.618175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.618583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints/1ed341b6d3504b1a82dd7527cdc057f5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.620029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=1ed341b6d3504b1a82dd7527cdc057f5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.622833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.628712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7c903-d864-4661-ae2d-233c58b56d0f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.636313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '1ed341b6d3504b1a82dd7527cdc057f5', 'request_id': 'req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:45.637128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d2561-0d8a-4397-80b8-5dab221df2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.641143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2561-0d8a-4397-80b8-5dab221df2be admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.641577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2561-0d8a-4397-80b8-5dab221df2be admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.641891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2561-0d8a-4397-80b8-5dab221df2be admin admin] PATH_INFO: `/v3/projects/dd7dc6344de94253979adb877c91e102/users/d986a5c0854d4ba298481dd3c3b524c4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.654947 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3619/7244] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:17:45 2022] DELETE /identity/v3/endpoints/1ed341b6d3504b1a82dd7527cdc057f5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:45.656838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.664616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d2561-0d8a-4397-80b8-5dab221df2be admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7dc6344de94253979adb877c91e102, user_id=d986a5c0854d4ba298481dd3c3b524c4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.668407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d2561-0d8a-4397-80b8-5dab221df2be admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.689410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2f3e7-eaca-4bf1-a41e-c146ff77022a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.701021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3626/7245] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:45 2022] PUT /identity/v3/projects/dd7dc6344de94253979adb877c91e102/users/d986a5c0854d4ba298481dd3c3b524c4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:45.703043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da76f-936a-4cf2-a655-44240f128664 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.703340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da76f-936a-4cf2-a655-44240f128664 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.703619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59da76f-936a-4cf2-a655-44240f128664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.755231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b59da76f-936a-4cf2-a655-44240f128664 None None] MFA Rules not processed for user `d986a5c0854d4ba298481dd3c3b524c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:45.759735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b59da76f-936a-4cf2-a655-44240f128664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.808585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.809260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.809556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.810033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.810719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.813273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.833100 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3620/7246] 10.222.0.33 () {58 vars in 1134 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/endpoints => generated 4051 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:45.834999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.847410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99396d0-a9b9-42f6-9722-a02ecc4c6f61 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.854232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.855666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.856269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.856555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints/1ed341b6d3504b1a82dd7527cdc057f5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.862249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:17:45.864069 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3627/7247] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3884 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Could not find endpoint: 1ed341b6d3504b1a82dd7527cdc057f5.: keystone.exception.EndpointNotFound: Could not find endpoint: 1ed341b6d3504b1a82dd7527cdc057f5. Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._iter().one() Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._only_one_row( Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:17:45.870397 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 1ed341b6d3504b1a82dd7527cdc057f5. Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:17:45.871678 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:17:45.872683 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 1ed341b6d3504b1a82dd7527cdc057f5. Sep 10 21:17:45.873695 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:45.877583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.877938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.878232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.883090 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3621/7248] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:17:45 2022] DELETE /identity/v3/endpoints/1ed341b6d3504b1a82dd7527cdc057f5 => generated 114 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:17:45.887244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.904860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf24598-0f0f-4952-b5c5-95d80a0ae32a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:45.918112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:45.919559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:45.919810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:45.920048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] MFA Rules not processed for user `d986a5c0854d4ba298481dd3c3b524c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:45.920276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1890051280` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:45.921167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1890051280)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:45.921376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.923125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:45.931078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3628/7249] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3884 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:45.943121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:45.945203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:45.945782 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1890051280. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:45.945782 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:45.945782 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:45.952642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:45.967494 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3622/7250] 10.222.0.33 () {58 vars in 1209 bytes} [Sat Sep 10 21:17:45 2022] DELETE /identity/v3/regions/tempest-region-1890051280 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:45.968830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.021722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f227b8-77e4-4219-a920-ead847cf5c23 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.033507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.034132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.034496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.034702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.035324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.037733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.051367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3623/7251] 10.222.0.33 () {58 vars in 1134 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/endpoints => generated 4051 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:46.051367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.051367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.144112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f16b1f-773d-4289-bbc2-bb552e4e39bd tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.149565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.150231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.150510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.150784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] PATH_INFO: `/v3/projects/d5373fd3d1734191ab3dd5886026cb02` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.151914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5373fd3d1734191ab3dd5886026cb02)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.153865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.159444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b57c38-9c6c-4f52-ac18-f9251c569630 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.170433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:46.176376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.177107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.177484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.177904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.186815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.188481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.205228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5373fd3d1734191ab3dd5886026cb02', 'request_id': 'req-4df295f8-7396-4d5d-a306-af6b925cf02d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.209913 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3629/7252] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:45 2022] GET /identity/v3/auth/tokens => generated 3884 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:46.210741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.213409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5373fd3d1734191ab3dd5886026cb02', 'request_id': 'req-4df295f8-7396-4d5d-a306-af6b925cf02d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.221429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982928e-d154-4a43-866f-8afe260578a9 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.224338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5373fd3d1734191ab3dd5886026cb02', 'request_id': 'req-4df295f8-7396-4d5d-a306-af6b925cf02d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.240951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] The token cache is being invalidate because project d5373fd3d1734191ab3dd5886026cb02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:46.240951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.241340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.245920 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3624/7253] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:46 2022] DELETE /identity/v3/projects/d5373fd3d1734191ab3dd5886026cb02 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:46.247030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.250177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.295525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.296365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.296669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.297307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.298496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Adding query filter params (service_id=eafe049b285b4015991e5c14b311a5b2) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:46.299034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.300135 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:46.300135 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:17:46.300135 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:46.301146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.309583 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3630/7254] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:17:46 2022] GET /identity/v3/endpoints?service_id=eafe049b285b4015991e5c14b311a5b2 => generated 503 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:46.310579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.312338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.332706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df295f8-7396-4d5d-a306-af6b925cf02d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.338734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.339636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.339989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.340747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] PATH_INFO: `/v3/projects/7ce320298cc24ce2824406f42ac30e3e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.342016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce320298cc24ce2824406f42ac30e3e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.344393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.380426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:46.398720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429c917-2ae4-41df-96a1-727edd13ecaf tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.401874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce320298cc24ce2824406f42ac30e3e', 'request_id': 'req-69e4fd3b-2429-4462-9ce8-9792f3838e92'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.408604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce320298cc24ce2824406f42ac30e3e', 'request_id': 'req-69e4fd3b-2429-4462-9ce8-9792f3838e92'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.420378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce320298cc24ce2824406f42ac30e3e', 'request_id': 'req-69e4fd3b-2429-4462-9ce8-9792f3838e92'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.435973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] The token cache is being invalidate because project 7ce320298cc24ce2824406f42ac30e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:46.436566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.437486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.441132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3625/7255] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:46 2022] DELETE /identity/v3/projects/7ce320298cc24ce2824406f42ac30e3e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:46.442344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.444685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.476617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.477282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.477565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.477850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] PATH_INFO: `/v3/users/2f83cde752df4d989657d416504353c4` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.479040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f83cde752df4d989657d416504353c4)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.481266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.492333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:46.540623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f83cde752df4d989657d416504353c4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.542275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4fd3b-2429-4462-9ce8-9792f3838e92 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.550078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f97216-4c3e-448f-b578-412d42f7c4ca tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.550697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f97216-4c3e-448f-b578-412d42f7c4ca tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.551165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f97216-4c3e-448f-b578-412d42f7c4ca tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.551447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f97216-4c3e-448f-b578-412d42f7c4ca tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.552439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f97216-4c3e-448f-b578-412d42f7c4ca tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Adding query filter params (service_id=62f99372cebd42378badf4a4532b8c36) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:46.553485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f97216-4c3e-448f-b578-412d42f7c4ca tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.557332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f83cde752df4d989657d416504353c4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.557464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f97216-4c3e-448f-b578-412d42f7c4ca tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.568383 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3626/7256] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:17:46 2022] GET /identity/v3/endpoints?service_id=62f99372cebd42378badf4a4532b8c36 => generated 505 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:46.568383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.568383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.568383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.571847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f83cde752df4d989657d416504353c4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.583311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f83cde752df4d989657d416504353c4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.586043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1befeb3-460c-4386-ba55-6337dbee08cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f83cde752df4d989657d416504353c4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:46.594309 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3631/7257] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:46 2022] DELETE /identity/v3/users/2f83cde752df4d989657d416504353c4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:46.598541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ae7c-79a8-4a04-baea-1e0274e7434c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.598836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ae7c-79a8-4a04-baea-1e0274e7434c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.599195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ae7c-79a8-4a04-baea-1e0274e7434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.604719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:46.607047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.620943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2d43ae7c-79a8-4a04-baea-1e0274e7434c None None] MFA Rules not processed for user `d986a5c0854d4ba298481dd3c3b524c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:46.622943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2d43ae7c-79a8-4a04-baea-1e0274e7434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.701866 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3627/7258] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3873 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:46.701866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.701866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.703113 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3632/7259] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3884 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:46.706766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012842-03f8-48c2-ba9d-407a76269aae None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.721086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012842-03f8-48c2-ba9d-407a76269aae None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.721393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012842-03f8-48c2-ba9d-407a76269aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.768234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-17012842-03f8-48c2-ba9d-407a76269aae None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:46.769519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-17012842-03f8-48c2-ba9d-407a76269aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:46.806553 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3633/7260] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:46.810536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012842-03f8-48c2-ba9d-407a76269aae None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.830217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962c847-2cec-4b86-9e8e-50c456923ed6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.837033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.838300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.838601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.839442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.840046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:46.840499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.843926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.854095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012842-03f8-48c2-ba9d-407a76269aae None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.869669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.870777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.872505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.874456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.875959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:46.876791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.881804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.890143 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3628/7261] 10.222.0.33 () {58 vars in 1167 bytes} [Sat Sep 10 21:17:46 2022] GET /identity/v3/endpoints?interface=public => generated 3406 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:46.891961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.905552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd345a6f-84d0-4ba0-9c8f-cd46d2c1847a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.907270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:46.912580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.913327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.913564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.913870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.914789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:46.914911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3634/7262] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:17:46.918568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.919870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.921805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.930197 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3629/7263] 10.222.0.33 () {58 vars in 1171 bytes} [Sat Sep 10 21:17:46 2022] GET /identity/v3/endpoints?interface=internal => generated 480 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:46.936253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:46.947054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7216d-00e6-441e-b5b5-21e3690bed35 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.951632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17f08b-984d-4818-b184-b0d92d06036d tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:46.957659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.958011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:46.958535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.959024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.959147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:46.959658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:46.960093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.960202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:46.960954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.961473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:46.963725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:46.965632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.012413 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3630/7264] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:46 2022] POST /identity/v3/services => generated 288 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:47.025531 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3635/7265] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:46 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:47.030112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.031393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.067329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda8bb2-6d1e-40ad-a566-8bec65a25c0a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.151899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:47.221788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.224108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.226716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32df73-0bd6-4709-a391-154695573072 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.227348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.227910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.228850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.231336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.247610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.249117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.249400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.250336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.251444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.253802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.255547 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1882443546. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:47.255547 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:47.255547 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:47.289302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3636/7266] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:47 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:47.289302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.290068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:47.306281 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3631/7267] 10.222.0.33 () {60 vars in 1156 bytes} [Sat Sep 10 21:17:47 2022] POST /identity/v3/endpoints => generated 360 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:47.308730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba1e27-a163-4a8b-aac4-b281f9249bfe admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.310876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.333731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.334927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.335233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571ccb3-16a2-4234-a747-75fd9dfe59cc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.335366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.335768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] PATH_INFO: `/v3/projects/f22854352a7b46d495aac3802c21a47b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.337103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22854352a7b46d495aac3802c21a47b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.339137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.342091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.342706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.342981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.343245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.343882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.345922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.386694 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3632/7268] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:47.387756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.421339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c50d89-82e0-43fd-a5f2-89ae13a9ce73 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.426020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22854352a7b46d495aac3802c21a47b', 'request_id': 'req-c8e4279f-bc97-41ca-9324-faca21c12960'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:47.434689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22854352a7b46d495aac3802c21a47b', 'request_id': 'req-c8e4279f-bc97-41ca-9324-faca21c12960'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:47.442683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22854352a7b46d495aac3802c21a47b', 'request_id': 'req-c8e4279f-bc97-41ca-9324-faca21c12960'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:47.462217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] The token cache is being invalidate because project f22854352a7b46d495aac3802c21a47b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:47.462808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:47.463668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:47.465918 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3637/7269] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:47 2022] DELETE /identity/v3/projects/f22854352a7b46d495aac3802c21a47b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:47.467326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.469406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:47.499588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3745906-3c7d-4cea-b62d-7cc9f03b32b3 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.500326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3745906-3c7d-4cea-b62d-7cc9f03b32b3 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.500609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3745906-3c7d-4cea-b62d-7cc9f03b32b3 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.500890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3745906-3c7d-4cea-b62d-7cc9f03b32b3 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1882443546` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.501504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3745906-3c7d-4cea-b62d-7cc9f03b32b3 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1882443546)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.503040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3745906-3c7d-4cea-b62d-7cc9f03b32b3 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.510494 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3633/7270] 10.222.0.33 () {58 vars in 1206 bytes} [Sat Sep 10 21:17:47 2022] GET /identity/v3/regions/tempest-region-1882443546 => generated 180 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:47.510494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18866106-0630-497d-95ca-6bed2e634366 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.510494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18866106-0630-497d-95ca-6bed2e634366 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.510494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18866106-0630-497d-95ca-6bed2e634366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.525305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:47.532070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-18866106-0630-497d-95ca-6bed2e634366 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:47.533914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-18866106-0630-497d-95ca-6bed2e634366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:47.580871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4279f-bc97-41ca-9324-faca21c12960 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.589124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517efbd5-c59d-4f6b-9b9b-2bf389e8c5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.589750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517efbd5-c59d-4f6b-9b9b-2bf389e8c5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.590074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517efbd5-c59d-4f6b-9b9b-2bf389e8c5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.590346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517efbd5-c59d-4f6b-9b9b-2bf389e8c5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.590982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517efbd5-c59d-4f6b-9b9b-2bf389e8c5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.592808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517efbd5-c59d-4f6b-9b9b-2bf389e8c5c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.611857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3634/7271] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:47 2022] POST /identity/v3/auth/tokens => generated 4190 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:47.612920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18866106-0630-497d-95ca-6bed2e634366 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.614914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-18866106-0630-497d-95ca-6bed2e634366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:47.618513 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3638/7272] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:47.620690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d7250-35c5-4732-844e-891174574b60 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.620690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d7250-35c5-4732-844e-891174574b60 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.620690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d7250-35c5-4732-844e-891174574b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.634191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18866106-0630-497d-95ca-6bed2e634366 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.641054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.641802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.642127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.642664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints/5e981d1af9f0464bae515b65e2e60160` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.646202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b47d7250-35c5-4732-844e-891174574b60 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:47.648486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b47d7250-35c5-4732-844e-891174574b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:47.650813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=5e981d1af9f0464bae515b65e2e60160)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.652836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.662595 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3639/7273] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:47 2022] POST /identity/v3/auth/tokens => generated 4190 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:47.662595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d7250-35c5-4732-844e-891174574b60 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.673480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d7250-35c5-4732-844e-891174574b60 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.680253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.681015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.681416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.681634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] PATH_INFO: `/v3/projects/1af33396df314edf8546375a33aab2c3/users/f17aca39815a4cb4b6756d91f270af2b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.682749 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-682081884. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:47.682749 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:47.682749 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:47.699894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af33396df314edf8546375a33aab2c3, user_id=f17aca39815a4cb4b6756d91f270af2b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.701421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:47.702690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.722709 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3640/7274] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:47 2022] PUT /identity/v3/projects/1af33396df314edf8546375a33aab2c3/users/f17aca39815a4cb4b6756d91f270af2b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:47.723584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.723771 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3635/7275] 10.222.0.33 () {60 vars in 1256 bytes} [Sat Sep 10 21:17:47 2022] PATCH /identity/v3/endpoints/5e981d1af9f0464bae515b65e2e60160 => generated 362 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:47.734380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.762904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b3c72-2ce5-49db-851f-b35903eeaafa tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.811203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463da02-166d-4421-bfb0-a1612b52022a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.828098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dca44-6c5b-4d1a-869c-cd84e195d954 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.828379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.828880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dca44-6c5b-4d1a-869c-cd84e195d954 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.829025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.829172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dca44-6c5b-4d1a-869c-cd84e195d954 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.829288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.829476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.830068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dca44-6c5b-4d1a-869c-cd84e195d954 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-682081884` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.830180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:47.830451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.831150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dca44-6c5b-4d1a-869c-cd84e195d954 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-682081884)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.832047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.832778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dca44-6c5b-4d1a-869c-cd84e195d954 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.836473 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3636/7276] 10.222.0.33 () {58 vars in 1203 bytes} [Sat Sep 10 21:17:47 2022] GET /identity/v3/regions/tempest-region-682081884 => generated 178 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:17:47.837737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfb5a9-89bf-484d-aad5-1ff8a66449e4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.838051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfb5a9-89bf-484d-aad5-1ff8a66449e4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.838389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfb5a9-89bf-484d-aad5-1ff8a66449e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.852849 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3641/7277] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:47.852849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.862404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf7817-f0f2-42e3-b47b-b723e5c9e9f0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.868580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.869247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.869408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-37cfb5a9-89bf-484d-aad5-1ff8a66449e4 None None] MFA Rules not processed for user `f17aca39815a4cb4b6756d91f270af2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:47.869647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.869803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints/5e981d1af9f0464bae515b65e2e60160` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.870909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-37cfb5a9-89bf-484d-aad5-1ff8a66449e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:47.875543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=5e981d1af9f0464bae515b65e2e60160)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.877709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.887110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '5e981d1af9f0464bae515b65e2e60160', 'request_id': 'req-845cc7e4-e30a-4d28-aba3-3e06954dec5a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:47.902113 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3642/7278] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:17:47 2022] DELETE /identity/v3/endpoints/5e981d1af9f0464bae515b65e2e60160 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:47.902113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.922706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cc7e4-e30a-4d28-aba3-3e06954dec5a tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.928275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:47.928912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:47.929191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:47.929472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:47.930137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:47.932039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:47.950068 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3637/7279] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:47 2022] POST /identity/v3/auth/tokens => generated 4023 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:47.950997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfb5a9-89bf-484d-aad5-1ff8a66449e4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.965491 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3643/7280] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:47 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:47.965491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:47.988255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfb5a9-89bf-484d-aad5-1ff8a66449e4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:47.998235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:48.047942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e6bb0-6ac7-41d3-8d2a-90bca68ad5eb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.054431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.055095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.055379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.055735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.056423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.058552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.060156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.060908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.061158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.061407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-682081884` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.061986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-682081884)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.063658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.075299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:48.092355 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3638/7281] 10.222.0.33 () {58 vars in 1206 bytes} [Sat Sep 10 21:17:47 2022] DELETE /identity/v3/regions/tempest-region-682081884 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:48.093164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.097098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3644/7282] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:47 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:48.097098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.119466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a8b91-24c4-4608-9513-0a483541cb94 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.197278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.198140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.198430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.198708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.199352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.201417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.203733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204febb6-5c44-4c46-97bb-f8177835409d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.223600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.223761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.223998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.224276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1882443546` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.225769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1882443546)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.227752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.230334 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3639/7283] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:48 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:48.231577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.243884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab53a4-2ad4-4867-8db7-290f28fd395f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.248261 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1882443546. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:48.248261 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:48.248261 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:48.250224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.251334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.251461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.251881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.252734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.254786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.260888 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3645/7284] 10.222.0.33 () {58 vars in 1209 bytes} [Sat Sep 10 21:17:48 2022] DELETE /identity/v3/regions/tempest-region-1882443546 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:48.262268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.291129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3640/7285] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:48 2022] POST /identity/v3/users => generated 291 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:48.292612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.299582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf247304-5116-49a9-9c1f-212529fe1fdc tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.308701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.310302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.310595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.311190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.311880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.313553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.319744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4db1e4-9619-4acb-a3d8-21de4fc79b4d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.348939 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3646/7286] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:48.350847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.351558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.351900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.352057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.352302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/services/545ea848a9d04c80bc091d0611f42e68` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.353130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_service(service_id=545ea848a9d04c80bc091d0611f42e68)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.355325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.364156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb326c2-df52-45d0-97b3-806de07d3466 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.372988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.373608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.373948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.374584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.375237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.376856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.393946 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3647/7287] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:48.395524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.408067 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3641/7288] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:48 2022] DELETE /identity/v3/services/545ea848a9d04c80bc091d0611f42e68 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:48.408067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.414439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ac064-77f0-4f11-97af-2aa8cfd41f05 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.434693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.435394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.435696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.435883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.436493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.438189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.441116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36659c0e-2381-4fb6-894f-661cb143282e tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.450315 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3648/7289] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:48.452315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.452927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.453703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.454202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.454379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.455285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.457217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.473135 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3642/7290] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:48.475435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.480673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec0158-8b56-4527-8030-9f9ab78e70b6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.488718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209ab32-213e-42dd-a459-6c59406837b9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.489104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03926ede-64eb-45f9-ae1e-d2c67187b33b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.489414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209ab32-213e-42dd-a459-6c59406837b9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.490567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209ab32-213e-42dd-a459-6c59406837b9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.491336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209ab32-213e-42dd-a459-6c59406837b9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints/b2ddf07c8b124522a2d8bd7811b17059` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.494866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.495698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.496150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.496452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] PATH_INFO: `/v3/projects/f6d0bab81f114005b52e30b106e2b714/users/ef78b6464d324628985286e45b69d926/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.502491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209ab32-213e-42dd-a459-6c59406837b9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=b2ddf07c8b124522a2d8bd7811b17059)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.506901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209ab32-213e-42dd-a459-6c59406837b9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.514032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d0bab81f114005b52e30b106e2b714, user_id=ef78b6464d324628985286e45b69d926, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.517208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.528334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9209ab32-213e-42dd-a459-6c59406837b9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'b2ddf07c8b124522a2d8bd7811b17059', 'request_id': 'req-9209ab32-213e-42dd-a459-6c59406837b9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:48.532453 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3643/7291] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:48 2022] PUT /identity/v3/projects/f6d0bab81f114005b52e30b106e2b714/users/ef78b6464d324628985286e45b69d926/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:48.533103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.548514 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3649/7292] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:17:48 2022] DELETE /identity/v3/endpoints/b2ddf07c8b124522a2d8bd7811b17059 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:48.549764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c00a79-390c-42bd-b49b-5a13a1ac05ff None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.550115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c00a79-390c-42bd-b49b-5a13a1ac05ff None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.550388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c00a79-390c-42bd-b49b-5a13a1ac05ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.565316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:48.581987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-87c00a79-390c-42bd-b49b-5a13a1ac05ff None None] MFA Rules not processed for user `ef78b6464d324628985286e45b69d926`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:48.584227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-87c00a79-390c-42bd-b49b-5a13a1ac05ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:48.593550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=3,keystone:INSERT=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:48.620672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f3771-a2c2-4ca3-8714-6a51663bb6d6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.637207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.638041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.638329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.638662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] PATH_INFO: `/v3/projects/7e30a27a30874ecd963ba7c9e427a8c8/users/cee2ce91e7ed49f9a6427b0680a80ca6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.654889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e30a27a30874ecd963ba7c9e427a8c8, user_id=cee2ce91e7ed49f9a6427b0680a80ca6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.657565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.669230 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3650/7293] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3719 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:48.671407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c00a79-390c-42bd-b49b-5a13a1ac05ff None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.699049 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3644/7294] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:48 2022] PUT /identity/v3/projects/7e30a27a30874ecd963ba7c9e427a8c8/users/cee2ce91e7ed49f9a6427b0680a80ca6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:48.700102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.794384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c00a79-390c-42bd-b49b-5a13a1ac05ff None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.800590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269abc7-2d4d-4e05-9d79-c39cb752a043 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.813209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bfc4c-a8dd-43f9-82af-8783ada013a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.813973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bfc4c-a8dd-43f9-82af-8783ada013a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.814317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bfc4c-a8dd-43f9-82af-8783ada013a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.814788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bfc4c-a8dd-43f9-82af-8783ada013a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.815491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bfc4c-a8dd-43f9-82af-8783ada013a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.820231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bfc4c-a8dd-43f9-82af-8783ada013a6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.853987 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3645/7295] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:48 2022] POST /identity/v3/projects => generated 391 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:48.855201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fe293-467e-429a-ae7c-27cf5c2f90e3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.855489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fe293-467e-429a-ae7c-27cf5c2f90e3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.855763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fe293-467e-429a-ae7c-27cf5c2f90e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.872410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.873122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.873413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:48.873697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1186730759` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:48.874335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1186730759)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:48.876000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:48.886581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fc2fe293-467e-429a-ae7c-27cf5c2f90e3 None None] MFA Rules not processed for user `cee2ce91e7ed49f9a6427b0680a80ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:48.887852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fc2fe293-467e-429a-ae7c-27cf5c2f90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:48.890344 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1186730759. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:48.890344 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:48.890344 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:48.899706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3651/7296] 10.222.0.33 () {58 vars in 1209 bytes} [Sat Sep 10 21:17:48 2022] DELETE /identity/v3/regions/tempest-region-1186730759 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:48.902989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.923052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565823-e2b0-4e08-a58e-2b330e4752c9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:48.932581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:48.967977 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3646/7297] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3707 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:48.972787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fe293-467e-429a-ae7c-27cf5c2f90e3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:48.998901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:48.999524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:48.999810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.000142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.000784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.003348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.011134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:49.030900 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3652/7298] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:48 2022] POST /identity/v3/users => generated 302 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:49.032172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.064567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fe293-467e-429a-ae7c-27cf5c2f90e3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.072217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.073464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.073857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.074063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/services/62f99372cebd42378badf4a4532b8c36` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.074648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_service(service_id=62f99372cebd42378badf4a4532b8c36)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.076288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.101518 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3647/7299] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:48 2022] DELETE /identity/v3/services/62f99372cebd42378badf4a4532b8c36 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:49.102329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.120231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649759-38d0-4a47-8189-702126dec0c3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.125732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d98c9-cf34-4868-a46b-14c36e9b23d9 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.132984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.133728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.134160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.134570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.135445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.137184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.138598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17aca39815a4cb4b6756d91f270af2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1af33396df314edf8546375a33aab2c3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.139452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.139747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.140037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] PATH_INFO: `/v3/users/f17aca39815a4cb4b6756d91f270af2b/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.141533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] RBAC: Authorizing `identity:list_user_projects(user_id=f17aca39815a4cb4b6756d91f270af2b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.143527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.154015 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3648/7300] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:49.155133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.190936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630ebb0-7fcc-4200-abca-ea5d13be33c9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.197707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7f67b-3530-4833-86cf-dd6c59f6299f tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.199060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7f67b-3530-4833-86cf-dd6c59f6299f tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.199348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7f67b-3530-4833-86cf-dd6c59f6299f tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.203885 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3653/7301] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:17:49 2022] GET /identity/v3/users/f17aca39815a4cb4b6756d91f270af2b/projects => generated 515 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:49.204102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7f67b-3530-4833-86cf-dd6c59f6299f tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/endpoints/3f50926c38804eaa83bd8c7e68540cca` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.204258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.213024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7f67b-3530-4833-86cf-dd6c59f6299f tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=3f50926c38804eaa83bd8c7e68540cca)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.215744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7f67b-3530-4833-86cf-dd6c59f6299f tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.217351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6520d-824d-486a-8c30-15c1975ec8f5 tempest-IdentityV3ProjectsTest-961086225 tempest-IdentityV3ProjectsTest-961086225-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.222826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.223682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.224068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.224399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] PATH_INFO: `/v3/projects/3a91bf698a2c49bb8a4b43267e9fba5c/users/04e9e877f53b41f7abe46ab0e492a438/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.228573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-75d7f67b-3530-4833-86cf-dd6c59f6299f tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '3f50926c38804eaa83bd8c7e68540cca', 'request_id': 'req-75d7f67b-3530-4833-86cf-dd6c59f6299f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.243091 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3649/7302] 10.222.0.33 () {58 vars in 1236 bytes} [Sat Sep 10 21:17:49 2022] DELETE /identity/v3/endpoints/3f50926c38804eaa83bd8c7e68540cca => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:49.244498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f54bf-9660-4a7e-8bfe-c7c63974bfd6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.245119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f54bf-9660-4a7e-8bfe-c7c63974bfd6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.245356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f54bf-9660-4a7e-8bfe-c7c63974bfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.250023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a91bf698a2c49bb8a4b43267e9fba5c, user_id=04e9e877f53b41f7abe46ab0e492a438, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.256613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.277080 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3654/7303] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:49 2022] PUT /identity/v3/projects/3a91bf698a2c49bb8a4b43267e9fba5c/users/04e9e877f53b41f7abe46ab0e492a438/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:49.278065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.282845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-911f54bf-9660-4a7e-8bfe-c7c63974bfd6 None None] MFA Rules not processed for user `f17aca39815a4cb4b6756d91f270af2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:49.284507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-911f54bf-9660-4a7e-8bfe-c7c63974bfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:49.317634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:49.362558 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3650/7304] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3389 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:49.362558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f54bf-9660-4a7e-8bfe-c7c63974bfd6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.362822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea1794-1a09-49c9-8b70-27c0398a3940 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.379268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf571304-3c7d-42c3-b97e-8af4591d81a8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.379905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf571304-3c7d-42c3-b97e-8af4591d81a8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.380187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf571304-3c7d-42c3-b97e-8af4591d81a8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.380465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf571304-3c7d-42c3-b97e-8af4591d81a8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/regions/tempest-region-1333218139` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.382548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf571304-3c7d-42c3-b97e-8af4591d81a8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1333218139)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.384160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf571304-3c7d-42c3-b97e-8af4591d81a8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.397245 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-cf571304-3c7d-42c3-b97e-8af4591d81a8 tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1333218139. To ensure interoperability, identifiers should be a valid uuid. Sep 10 21:17:49.397245 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 10 21:17:49.397245 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:17:49.405303 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3655/7305] 10.222.0.33 () {58 vars in 1209 bytes} [Sat Sep 10 21:17:49 2022] DELETE /identity/v3/regions/tempest-region-1333218139 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:49.409591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.409981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.410266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.412372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:49.431390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] MFA Rules not processed for user `f17aca39815a4cb4b6756d91f270af2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:49.431999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:49.444118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f54bf-9660-4a7e-8bfe-c7c63974bfd6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.459369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.460056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.460334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.460605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.461236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.462895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.472755 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3651/7306] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:49.473714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.478586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.models.token_model [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] User f17aca39815a4cb4b6756d91f270af2b has no access to project 7e30a27a30874ecd963ba7c9e427a8c8 {{(pid=93527) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Sep 10 21:17:49.482678 ncdv-master-1 devstack@keystone.service[93527]: WARNING keystone.server.flask.application [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 10 21:17:49.484195 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3656/7307] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:49 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 78 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 10 21:17:49.484846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.496317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b890cf-7463-4012-8fa0-729cf6f0e5cc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.499760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e90a6c-05c2-4088-af4c-31e6ebf9c658 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.501301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa88731c70049db87f16e2325b02643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c4e94f65f10447a89daa284e4aa4350', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.502038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.502320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.502722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] PATH_INFO: `/v3/services/eafe049b285b4015991e5c14b311a5b2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.503297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorizing `identity:delete_service(service_id=eafe049b285b4015991e5c14b311a5b2)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.504404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.504882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.505045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.505327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.505726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] PATH_INFO: `/v3/projects/3a91bf698a2c49bb8a4b43267e9fba5c/users/04e9e877f53b41f7abe46ab0e492a438/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.508372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a91bf698a2c49bb8a4b43267e9fba5c, user_id=04e9e877f53b41f7abe46ab0e492a438, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.510997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.522782 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3657/7308] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:49 2022] PUT /identity/v3/projects/3a91bf698a2c49bb8a4b43267e9fba5c/users/04e9e877f53b41f7abe46ab0e492a438/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:49.532876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.532992 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3652/7309] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:49 2022] DELETE /identity/v3/services/eafe049b285b4015991e5c14b311a5b2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:49.532992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.533817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:49.558776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53677ba8-e422-4179-863e-d642c8f8d86c tempest-EndPointsTestJSON-1406060628 tempest-EndPointsTestJSON-1406060628-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.605428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8af93-3c82-4a31-985f-daff019f8fa1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:49.610990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.611745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.612065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.612349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] PATH_INFO: `/v3/users/f17aca39815a4cb4b6756d91f270af2b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.613536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f17aca39815a4cb4b6756d91f270af2b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.615586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.641692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:49.642499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.643080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.643794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] PATH_INFO: `/v3/users/ea03554527fc4a3bb49e9d78f0849a03` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.652873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea03554527fc4a3bb49e9d78f0849a03)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:49.655071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:49.688760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f17aca39815a4cb4b6756d91f270af2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.690112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:49.709912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f17aca39815a4cb4b6756d91f270af2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.718781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f17aca39815a4cb4b6756d91f270af2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.727832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f17aca39815a4cb4b6756d91f270af2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.728567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-035ad4a7-3462-49eb-a031-9ff308074102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f17aca39815a4cb4b6756d91f270af2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.736455 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3658/7310] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:49 2022] DELETE /identity/v3/users/f17aca39815a4cb4b6756d91f270af2b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:49.738093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972c70-fe0d-4efe-b523-96a4d3ccf4f4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.738385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972c70-fe0d-4efe-b523-96a4d3ccf4f4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.738653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972c70-fe0d-4efe-b523-96a4d3ccf4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.746604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea03554527fc4a3bb49e9d78f0849a03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.757547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea03554527fc4a3bb49e9d78f0849a03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.758770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea03554527fc4a3bb49e9d78f0849a03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.773578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-27972c70-fe0d-4efe-b523-96a4d3ccf4f4 None None] MFA Rules not processed for user `04e9e877f53b41f7abe46ab0e492a438`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:49.773999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea03554527fc4a3bb49e9d78f0849a03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.774966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-27972c70-fe0d-4efe-b523-96a4d3ccf4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:49.775438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4723bb7a-1472-4696-a43b-96342bd6f1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea03554527fc4a3bb49e9d78f0849a03'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:49.794460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:49.800457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3653/7311] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:49 2022] DELETE /identity/v3/users/ea03554527fc4a3bb49e9d78f0849a03 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:49.802377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.802905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.803594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.832953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:49.835084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:49.877520 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3659/7312] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:49.877520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:49.877520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:49.877520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:49.897563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:49.898816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:49.922737 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3654/7313] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:49.923678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.925141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:49.979761 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3660/7314] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:49.983352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:49.983352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.022499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf2737-07b9-408a-a9b8-6a7d4351d6af None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.042118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.042118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.042118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.042118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.072802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:50.083380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05928f-0800-42f4-9be7-dbb9d8386782 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.090981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.092474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.092795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.093081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.121555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.123291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.134802 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3661/7315] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:50.135731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:50.137243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.140108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.141971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.155002 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3655/7316] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:50.157490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e35c-5732-48ac-8a97-e18ac127ef5e service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.162372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.163247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.166601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.166601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.179980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.182657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.198397 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3662/7317] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:50.214016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:50.215449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.232252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6d3f0-6a84-4f1d-9625-ed18af17b33e service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.237965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.239491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.239785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.240662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.242746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.316169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:50.354342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.356410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.393545 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3656/7318] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:50.512672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:50.514921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.551600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440bcd9-5cbb-4bea-8378-6388ca49a376 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.560925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.561248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.561703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.562580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] PATH_INFO: `/v3/projects/1af33396df314edf8546375a33aab2c3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.563892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af33396df314edf8546375a33aab2c3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.566278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.601368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:50.603828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.626939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16594c57-bbcb-4fb0-88b0-bcdd0954cc6c service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.644285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.645448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.645866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.646576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] PATH_INFO: `/v3/projects/008348cf6bbc4776853ab733ac2927e0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.648661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=008348cf6bbc4776853ab733ac2927e0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.652836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.664089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af33396df314edf8546375a33aab2c3', 'request_id': 'req-a4d929ba-2373-4711-9bca-4ad388507dd5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.675605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af33396df314edf8546375a33aab2c3', 'request_id': 'req-a4d929ba-2373-4711-9bca-4ad388507dd5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.684971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af33396df314edf8546375a33aab2c3', 'request_id': 'req-a4d929ba-2373-4711-9bca-4ad388507dd5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.707262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] The token cache is being invalidate because project 1af33396df314edf8546375a33aab2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:50.707778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.708833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.710670 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3663/7319] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:50 2022] DELETE /identity/v3/projects/1af33396df314edf8546375a33aab2c3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:50.715116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:50.717361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.723400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008348cf6bbc4776853ab733ac2927e0', 'request_id': 'req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.733157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008348cf6bbc4776853ab733ac2927e0', 'request_id': 'req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.745003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008348cf6bbc4776853ab733ac2927e0', 'request_id': 'req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.764286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] The token cache is being invalidate because project 008348cf6bbc4776853ab733ac2927e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:50.764772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.765584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:50.767358 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3657/7320] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:50 2022] DELETE /identity/v3/projects/008348cf6bbc4776853ab733ac2927e0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:50.772296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:50.775535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:50.849799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d929ba-2373-4711-9bca-4ad388507dd5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.921796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e033b74-1b5a-4512-aa9f-5a9ce48e3085 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:50.941365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.942713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.943381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.944111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] PATH_INFO: `/v3/users/2aa88731c70049db87f16e2325b02643` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.947169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa88731c70049db87f16e2325b02643)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.949477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.952311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:50.953402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:50.953710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:50.954380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] PATH_INFO: `/v3/users/cee2ce91e7ed49f9a6427b0680a80ca6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:50.955620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee2ce91e7ed49f9a6427b0680a80ca6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:50.957799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:50.968648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:51.048735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa88731c70049db87f16e2325b02643'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.062050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa88731c70049db87f16e2325b02643'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.063441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa88731c70049db87f16e2325b02643'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.065547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee2ce91e7ed49f9a6427b0680a80ca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.066132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:51.076115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa88731c70049db87f16e2325b02643'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.077541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa88731c70049db87f16e2325b02643'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.098287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee2ce91e7ed49f9a6427b0680a80ca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.108684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee2ce91e7ed49f9a6427b0680a80ca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.113733 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3658/7321] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:50 2022] DELETE /identity/v3/users/2aa88731c70049db87f16e2325b02643 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:51.130095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee2ce91e7ed49f9a6427b0680a80ca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.132428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee2ce91e7ed49f9a6427b0680a80ca6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.149163 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3664/7322] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:50 2022] DELETE /identity/v3/users/cee2ce91e7ed49f9a6427b0680a80ca6 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:51.552043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:51.554932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:51.602016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:51.603649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:51.697736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a32a5-0c38-4058-9837-b5a999667873 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:51.720995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2511515-7a9d-47dd-aa44-b8cdd38c93bd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:51.727387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:51.728208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:51.728499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:51.729219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] PATH_INFO: `/v3/projects/7e30a27a30874ecd963ba7c9e427a8c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:51.730840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e30a27a30874ecd963ba7c9e427a8c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:51.734506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:51.774785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:51.775433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:51.775779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:51.776062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] PATH_INFO: `/v3/projects/3c4e94f65f10447a89daa284e4aa4350` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:51.777218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4e94f65f10447a89daa284e4aa4350)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:51.779171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:51.794812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e30a27a30874ecd963ba7c9e427a8c8', 'request_id': 'req-f6557eff-61ed-44d2-ab04-7da65314d8f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.800847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e30a27a30874ecd963ba7c9e427a8c8', 'request_id': 'req-f6557eff-61ed-44d2-ab04-7da65314d8f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.808894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e30a27a30874ecd963ba7c9e427a8c8', 'request_id': 'req-f6557eff-61ed-44d2-ab04-7da65314d8f8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.822313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] The token cache is being invalidate because project 7e30a27a30874ecd963ba7c9e427a8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:51.822737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.823383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f6557eff-61ed-44d2-ab04-7da65314d8f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.824909 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3665/7323] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:51 2022] DELETE /identity/v3/projects/7e30a27a30874ecd963ba7c9e427a8c8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:51.829448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0e7a-588c-44db-9120-0954655fbf80 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:51.829738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0e7a-588c-44db-9120-0954655fbf80 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:51.830056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb0e7a-588c-44db-9120-0954655fbf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:51.831458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4e94f65f10447a89daa284e4aa4350', 'request_id': 'req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.839257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4e94f65f10447a89daa284e4aa4350', 'request_id': 'req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.847220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4e94f65f10447a89daa284e4aa4350', 'request_id': 'req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.849572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-33cb0e7a-588c-44db-9120-0954655fbf80 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:51.850939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-33cb0e7a-588c-44db-9120-0954655fbf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:51.856559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:51.859120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] The token cache is being invalidate because project 3c4e94f65f10447a89daa284e4aa4350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:51.859417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.860080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3e91dde8-a8b9-4c0a-85d2-16ecb75d547b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:51.862394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3659/7324] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:51 2022] DELETE /identity/v3/projects/3c4e94f65f10447a89daa284e4aa4350 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:51.868648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a26a55-be2a-4b9f-8e61-5bc4fbb7d67c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:51.868930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a26a55-be2a-4b9f-8e61-5bc4fbb7d67c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:51.869242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a26a55-be2a-4b9f-8e61-5bc4fbb7d67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:51.892655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-77a26a55-be2a-4b9f-8e61-5bc4fbb7d67c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:51.894514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-77a26a55-be2a-4b9f-8e61-5bc4fbb7d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:51.900885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:51.917931 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3666/7325] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:51.926762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871dc3c-85ad-417d-ad5f-2d26367603d2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:51.927103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871dc3c-85ad-417d-ad5f-2d26367603d2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:51.927389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871dc3c-85ad-417d-ad5f-2d26367603d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:51.959010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8871dc3c-85ad-417d-ad5f-2d26367603d2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:51.960245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8871dc3c-85ad-417d-ad5f-2d26367603d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:51.978327 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3667/7326] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:51.983126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871dc3c-85ad-417d-ad5f-2d26367603d2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:51.994919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871dc3c-85ad-417d-ad5f-2d26367603d2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.000790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.000968 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3660/7327] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:52.001406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.001696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.002141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.003324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:52.003324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.005061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.010505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb54a5-afab-41be-8647-c462ded26e0e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.010857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb54a5-afab-41be-8647-c462ded26e0e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.011227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb54a5-afab-41be-8647-c462ded26e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.018788 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3668/7328] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:52.023061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.034712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f475-cbe5-42f1-8624-94930d2b96a5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.038526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-50bb54a5-afab-41be-8647-c462ded26e0e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:52.040634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-50bb54a5-afab-41be-8647-c462ded26e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:52.042317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.042848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.043065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.043330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.043989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.046458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.057128 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3661/7329] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:52.058350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb54a5-afab-41be-8647-c462ded26e0e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.070550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb54a5-afab-41be-8647-c462ded26e0e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.077879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.078989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.079531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.079889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.080732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:52.081177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.082968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.093048 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3669/7330] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:52.103077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.108389 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3662/7331] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:52.110208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.123127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24317249-258f-449e-a944-37f0815ddb1f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.176045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ae5f8-e25b-461d-b68f-11402b1148b7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.183084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.183746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.184033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.184313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.184952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.187008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.191038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.191862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.192335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.192678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.193575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.195840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.211808 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3670/7332] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:52.215581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.232967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e8a3-8f99-4160-8cee-4a7e3f68638a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.238029 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3663/7333] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:52.241400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.242940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.243696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.244177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.244568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.245463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.247436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.261620 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3671/7334] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:52.262874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.278836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4b2ee-3b1b-433b-88e6-4a07f298484d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.345756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb89b03-b2e8-4ba0-b3b2-3d9b08ecaaca admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.351780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.352398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.352681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.353020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.353673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.355969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.357491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:52.359253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.359979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.360607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.361007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.362401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.364203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.381869 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3672/7335] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:52.382072 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3664/7336] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:52.383179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.384508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.396932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599a156-8968-46c4-8f90-abe9645c3017 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.398279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f4f8c-86c7-4384-859f-29cdbd134be3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.402110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0347daf-9f13-4d52-99ca-525ae1227f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.403150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0347daf-9f13-4d52-99ca-525ae1227f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.403522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0347daf-9f13-4d52-99ca-525ae1227f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.403870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0347daf-9f13-4d52-99ca-525ae1227f36 admin admin] PATH_INFO: `/v3/projects/3cfd38709d244f18a36b3e7d02f0fc6a/users/f96c6c82386e4cf28bfba40a124bf956/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.405699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.406373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.406626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.406867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.407527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.409295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.411062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:52.426254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0347daf-9f13-4d52-99ca-525ae1227f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfd38709d244f18a36b3e7d02f0fc6a, user_id=f96c6c82386e4cf28bfba40a124bf956, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.426525 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3665/7337] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:52.430953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.432072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0347daf-9f13-4d52-99ca-525ae1227f36 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.441993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc80db-2816-46f1-ac62-f4b77374d73f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.446680 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3673/7338] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:52 2022] PUT /identity/v3/projects/3cfd38709d244f18a36b3e7d02f0fc6a/users/f96c6c82386e4cf28bfba40a124bf956/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:52.463598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257382c-2797-489b-ad7b-a0a58e983f2f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.463965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257382c-2797-489b-ad7b-a0a58e983f2f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.464470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257382c-2797-489b-ad7b-a0a58e983f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.503609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5257382c-2797-489b-ad7b-a0a58e983f2f None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:52.505902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5257382c-2797-489b-ad7b-a0a58e983f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:52.547883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.548509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.548795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.549074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.549722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.552158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.565632 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3666/7339] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:52.571926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.594700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d057a04-63e3-4900-88ce-dcdf46a55ee4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.604633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdd135-47fb-4049-9004-b6a7deeb7d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.605428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdd135-47fb-4049-9004-b6a7deeb7d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.605645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdd135-47fb-4049-9004-b6a7deeb7d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.605956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdd135-47fb-4049-9004-b6a7deeb7d7b admin admin] PATH_INFO: `/v3/projects/7ede7ae8765244e4aa532a7f8dff0941/users/51b977acabe34218b96055251a3f2196/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.616562 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3674/7340] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:52.622629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257382c-2797-489b-ad7b-a0a58e983f2f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.634866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdd135-47fb-4049-9004-b6a7deeb7d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ede7ae8765244e4aa532a7f8dff0941, user_id=51b977acabe34218b96055251a3f2196, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.636318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257382c-2797-489b-ad7b-a0a58e983f2f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.639029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdd135-47fb-4049-9004-b6a7deeb7d7b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.642704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813f36-d71b-4090-bf79-a4d40d75f9da tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.643355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813f36-d71b-4090-bf79-a4d40d75f9da tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.643871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813f36-d71b-4090-bf79-a4d40d75f9da tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.644041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84813f36-d71b-4090-bf79-a4d40d75f9da tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.667696 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3667/7341] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:52 2022] PUT /identity/v3/projects/7ede7ae8765244e4aa532a7f8dff0941/users/51b977acabe34218b96055251a3f2196/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:52.679978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9e8d-9e85-4a1d-b79a-5f9b23aecc16 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.680272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9e8d-9e85-4a1d-b79a-5f9b23aecc16 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.680566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb9e8d-9e85-4a1d-b79a-5f9b23aecc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.705369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-84813f36-d71b-4090-bf79-a4d40d75f9da tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:52.718772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1aeb9e8d-9e85-4a1d-b79a-5f9b23aecc16 None None] MFA Rules not processed for user `51b977acabe34218b96055251a3f2196`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:52.720181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1aeb9e8d-9e85-4a1d-b79a-5f9b23aecc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:52.721047 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3675/7342] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:52.826140 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3668/7343] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:52.828475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813f36-d71b-4090-bf79-a4d40d75f9da tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.842744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84813f36-d71b-4090-bf79-a4d40d75f9da tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:52.916849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:52.917042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:52.917513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:52.917816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:52.918846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:52.921163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:52.949879 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3676/7344] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:52.953967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb9e8d-9e85-4a1d-b79a-5f9b23aecc16 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:52.967415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb9e8d-9e85-4a1d-b79a-5f9b23aecc16 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.041357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:53.050581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.051205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.051489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.051761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.052916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.054950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.078605 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3669/7345] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:52 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:53.082903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.091854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:53.099188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c3a2-1fde-4904-9e1c-e058b73bc204 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.108053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.108864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.109411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.110213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.111263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.114273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.132345 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3677/7346] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:53.136345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.148260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7a1ee-2504-4831-892d-9047f37fa16a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.154479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.156188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.156511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.156841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] PATH_INFO: `/v3/projects/c41cebf054f841f784bd54c652d99792/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.176577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41cebf054f841f784bd54c652d99792, user_id=cf395756ee424d1e89c23dd476d43a38, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.179828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.197867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3670/7347] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:53 2022] PUT /identity/v3/projects/c41cebf054f841f784bd54c652d99792/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:53.202074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.215881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a5515-9d55-49bd-acff-e36661e4ffc9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.280863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee8ccc-c777-414e-8e9b-7eb476b79ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.281495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee8ccc-c777-414e-8e9b-7eb476b79ebf admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.281779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee8ccc-c777-414e-8e9b-7eb476b79ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.282320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee8ccc-c777-414e-8e9b-7eb476b79ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.283085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee8ccc-c777-414e-8e9b-7eb476b79ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.284936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee8ccc-c777-414e-8e9b-7eb476b79ebf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.297098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3678/7348] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:53.301752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.313611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b624d-3dcf-4ed6-9f39-6261c0630be0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.319151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.319879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.320226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.320564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] PATH_INFO: `/v3/projects/c41cebf054f841f784bd54c652d99792/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.322614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41cebf054f841f784bd54c652d99792, user_id=cf395756ee424d1e89c23dd476d43a38, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.325385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.344950 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3671/7349] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:53 2022] PUT /identity/v3/projects/c41cebf054f841f784bd54c652d99792/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:53.349362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b020ff-48cb-4b99-bc9f-6cfe198cd4c1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.349717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b020ff-48cb-4b99-bc9f-6cfe198cd4c1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.350113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b020ff-48cb-4b99-bc9f-6cfe198cd4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.382133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a9b020ff-48cb-4b99-bc9f-6cfe198cd4c1 None None] MFA Rules not processed for user `cf395756ee424d1e89c23dd476d43a38`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:53.383108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a9b020ff-48cb-4b99-bc9f-6cfe198cd4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:53.463155 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3679/7350] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:53.467831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.540809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b50ee8-306f-4bdc-86c9-3b3e94bb34a9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.545594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.546257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.546505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.549424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.550213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.552123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.583780 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3672/7351] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:53 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:53.585144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b020ff-48cb-4b99-bc9f-6cfe198cd4c1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.595823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b020ff-48cb-4b99-bc9f-6cfe198cd4c1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.655454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.656162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.656514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.656857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.657604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.659638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.685378 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3680/7352] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:53 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:53.689904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.702111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9e188-2ba7-4d6e-bac9-d33b7deec4b9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.707332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f22e-1055-460c-9d43-ab56b73400bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.713913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f22e-1055-460c-9d43-ab56b73400bf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.713913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f22e-1055-460c-9d43-ab56b73400bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.713913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f22e-1055-460c-9d43-ab56b73400bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.713913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688f22e-1055-460c-9d43-ab56b73400bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.713913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688f22e-1055-460c-9d43-ab56b73400bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.723046 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3673/7353] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:53.733138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6207ca5-235b-40b6-a726-ada36f8deceb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.733433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6207ca5-235b-40b6-a726-ada36f8deceb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.733711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6207ca5-235b-40b6-a726-ada36f8deceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.739642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.752230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:53.762370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd94e9-a706-4282-9c09-2a5353d73934 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.765179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e6207ca5-235b-40b6-a726-ada36f8deceb None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:53.766818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e6207ca5-235b-40b6-a726-ada36f8deceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:53.768046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.768938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.769234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.769508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.770353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.771174 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3674/7354] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:53 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:17:53.776085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.776553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b9ba-05ca-46d6-9615-085dc02b289d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.776907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b9ba-05ca-46d6-9615-085dc02b289d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.777338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b9ba-05ca-46d6-9615-085dc02b289d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.794528 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3681/7355] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:53.795536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.806284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-98b9b9ba-05ca-46d6-9615-085dc02b289d None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:53.807923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-98b9b9ba-05ca-46d6-9615-085dc02b289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:53.823068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbb2d9-0d9c-43ad-84a0-06d7b99e60c9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.834878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fcca8-efdc-4eb7-b315-48272fdde715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.835583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fcca8-efdc-4eb7-b315-48272fdde715 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.835872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fcca8-efdc-4eb7-b315-48272fdde715 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.836149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fcca8-efdc-4eb7-b315-48272fdde715 admin admin] PATH_INFO: `/v3/projects/e32a18ea720342d583553dcb59c7753a/users/504de572cc524b1e86ed53f26d4d9e04/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.863816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fcca8-efdc-4eb7-b315-48272fdde715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32a18ea720342d583553dcb59c7753a, user_id=504de572cc524b1e86ed53f26d4d9e04, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:53.864035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fcca8-efdc-4eb7-b315-48272fdde715 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:53.894420 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3682/7356] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:53 2022] PUT /identity/v3/projects/e32a18ea720342d583553dcb59c7753a/users/504de572cc524b1e86ed53f26d4d9e04/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:53.897905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f2eb0-2596-45e5-9646-b3249027415d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.898187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f2eb0-2596-45e5-9646-b3249027415d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.898465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f2eb0-2596-45e5-9646-b3249027415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:53.909810 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3675/7357] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:53.910860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b9ba-05ca-46d6-9615-085dc02b289d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:53.920021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b9ba-05ca-46d6-9615-085dc02b289d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:53.934813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b59f2eb0-2596-45e5-9646-b3249027415d None None] MFA Rules not processed for user `504de572cc524b1e86ed53f26d4d9e04`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:53.936157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b59f2eb0-2596-45e5-9646-b3249027415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:53.941201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:53.989335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea464-d680-420d-86b3-d7b081e0112c tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:53.990836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea464-d680-420d-86b3-d7b081e0112c tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:53.991305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea464-d680-420d-86b3-d7b081e0112c tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:53.991734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea464-d680-420d-86b3-d7b081e0112c tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.020540 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3683/7358] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:54.026859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f2eb0-2596-45e5-9646-b3249027415d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-820ea464-d680-420d-86b3-d7b081e0112c tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:17:54.062394 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 10 21:17:54.063596 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:54.070229 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3676/7359] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:17:53 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 188 bytes in 160 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 10 21:17:54.071259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea464-d680-420d-86b3-d7b081e0112c tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.099567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea464-d680-420d-86b3-d7b081e0112c tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.109485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16644af8-285a-4e92-bab0-5f197f90af9b tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.110405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16644af8-285a-4e92-bab0-5f197f90af9b tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.111071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16644af8-285a-4e92-bab0-5f197f90af9b tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.111310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16644af8-285a-4e92-bab0-5f197f90af9b tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.142316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f2eb0-2596-45e5-9646-b3249027415d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.149390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.150332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.150826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.151253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.152067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.155234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-16644af8-285a-4e92-bab0-5f197f90af9b tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:17:54.169122 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 10 21:17:54.170408 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:17:54.173775 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3677/7360] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 188 bytes in 103 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 10 21:17:54.174629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16644af8-285a-4e92-bab0-5f197f90af9b tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.183582 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3684/7361] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/domains => generated 276 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:54.188335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.191074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16644af8-285a-4e92-bab0-5f197f90af9b tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.278791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:54.286302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b581c-1d50-4319-bda4-a3b4b1efd72a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.287638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b581c-1d50-4319-bda4-a3b4b1efd72a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.287927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b581c-1d50-4319-bda4-a3b4b1efd72a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.288213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b581c-1d50-4319-bda4-a3b4b1efd72a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.298571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c466e-b61a-44c5-adf5-583a4f8b9d41 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.319390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.320023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.320733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.320924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.321602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.326186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.354759 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3685/7362] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/roles => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:54.356277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.358184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:54.360269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4a6b581c-1d50-4319-bda4-a3b4b1efd72a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:54.373553 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3678/7363] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:54.374343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b581c-1d50-4319-bda4-a3b4b1efd72a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.403136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b581c-1d50-4319-bda4-a3b4b1efd72a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.408753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.409594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.409911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.410189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/d189ee72ffab4faeb54eac6d88be3d48/users/504de572cc524b1e86ed53f26d4d9e04/roles/1277ff8b8cc841a3a5c07a0de0ccf380` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.425217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d189ee72ffab4faeb54eac6d88be3d48, user_id=504de572cc524b1e86ed53f26d4d9e04, role_id=1277ff8b8cc841a3a5c07a0de0ccf380)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.428057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.436958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c01c7-7013-4bd6-9e72-8622a644225b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.441477 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3679/7364] 10.222.0.33 () {60 vars in 1480 bytes} [Sat Sep 10 21:17:54 2022] PUT /identity/v3/domains/d189ee72ffab4faeb54eac6d88be3d48/users/504de572cc524b1e86ed53f26d4d9e04/roles/1277ff8b8cc841a3a5c07a0de0ccf380 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:54.446296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.500422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:54.507273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.508007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.508286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.508561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] PATH_INFO: `/v3/users/ef78b6464d324628985286e45b69d926` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.510114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef78b6464d324628985286e45b69d926)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.512232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.524274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a424ca-b2a1-40a7-b2fb-7b82ec812f39 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.529191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.529808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.530121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.530401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.531058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.532651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.557695 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3680/7365] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/domains => generated 276 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:54.562066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.578367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef78b6464d324628985286e45b69d926'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:54.593510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef78b6464d324628985286e45b69d926'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:54.601880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef78b6464d324628985286e45b69d926'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:54.611510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef78b6464d324628985286e45b69d926'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:54.612612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a1fcb746-8ecd-45b0-b2eb-c11d318e530f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef78b6464d324628985286e45b69d926'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:54.620595 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3686/7366] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:54 2022] DELETE /identity/v3/users/ef78b6464d324628985286e45b69d926 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:54.625320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.625611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.626127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.633676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f727a6-87ee-4b33-ac6b-508d283ef2bc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.648635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:54.650126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:54.656686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:54.704582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.705429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.706096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.706457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.707194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.709413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.727780 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3681/7367] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/roles => generated 239 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:54.731633 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3687/7368] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:54.732305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.735905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:54.743408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.745939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:54.749461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74651404-ec3d-4ace-8f27-7f34913f48e2 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.754412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.755125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.755501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.756152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.757051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.759070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.789842 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3682/7369] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/groups => generated 326 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:54.790951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.800739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5f5a-564b-4494-9553-d56d26363e3b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.805913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.806549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.806835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.807111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/users/504de572cc524b1e86ed53f26d4d9e04` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.812563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=7279da4ca7bd4c2c9ec202e0ad1fe916, user_id=504de572cc524b1e86ed53f26d4d9e04)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.813541 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:17:54.813541 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:17:54.813541 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:17:54.814776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.816661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968e72-9b72-430a-b449-f381600fa36e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.821133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.821738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.822055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.822330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.831891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.833443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.839533 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3683/7370] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:17:54 2022] PUT /identity/v3/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/users/504de572cc524b1e86ed53f26d4d9e04 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:54.843857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.844033 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3688/7371] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:17:54.878573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:54.915326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52307a6b-ad96-4861-b9e4-e8b8f50007cd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:54.921339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:54.922182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:54.922573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:54.923111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/7d0ad7e21560473ba0e509be9d374e5c/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/roles/cba5cdff14504901ace62120fd85a293` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:54.927299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=7d0ad7e21560473ba0e509be9d374e5c, group_id=7279da4ca7bd4c2c9ec202e0ad1fe916, role_id=cba5cdff14504901ace62120fd85a293)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:54.931015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:54.946723 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3684/7372] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:54 2022] PUT /identity/v3/domains/7d0ad7e21560473ba0e509be9d374e5c/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/roles/cba5cdff14504901ace62120fd85a293 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:54.951201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:54.968217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3036010-67ca-454c-bb23-d1227660f5cd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.034133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.034757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.035039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.035312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.036072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.038863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.074209 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3689/7373] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:17:54 2022] POST /identity/v3/domains => generated 275 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:55.076535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.087750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ff3a-8c1d-4b39-9158-d4ee0705d1c3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.154540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.155158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.155882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.155882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.157168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.159162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.176414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3685/7374] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:17:55 2022] POST /identity/v3/roles => generated 240 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:55.181812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.192559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a565c-2004-44d0-a5a9-95c055d276b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.197579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.198601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.198888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.198989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.199180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/941de5908b9d499e87b02790386a94bf/users/504de572cc524b1e86ed53f26d4d9e04/roles/645dc72beb294d498e26c5241621b6c2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.200822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:55.213235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=941de5908b9d499e87b02790386a94bf, user_id=504de572cc524b1e86ed53f26d4d9e04, role_id=645dc72beb294d498e26c5241621b6c2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.216728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.231970 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3690/7375] 10.222.0.33 () {60 vars in 1480 bytes} [Sat Sep 10 21:17:55 2022] PUT /identity/v3/domains/941de5908b9d499e87b02790386a94bf/users/504de572cc524b1e86ed53f26d4d9e04/roles/645dc72beb294d498e26c5241621b6c2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:55.236180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.269616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:55.276337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad4d1-f973-4383-820d-cfa20139c638 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.306353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0726-ef71-4f7c-b3fa-ed4e46a7d070 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.311272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.311879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.312165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.312594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.313124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.314879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.335164 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3691/7376] 10.222.0.33 () {60 vars in 1149 bytes} [Sat Sep 10 21:17:55 2022] POST /identity/v3/domains => generated 275 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:55.336170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.337690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.338364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.338648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.338921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] PATH_INFO: `/v3/projects/f6d0bab81f114005b52e30b106e2b714` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.340086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6d0bab81f114005b52e30b106e2b714)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.342815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.355172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:55.404563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6d0bab81f114005b52e30b106e2b714', 'request_id': 'req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.409356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394f933-a38b-4246-94c6-1c215308329d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.411498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6d0bab81f114005b52e30b106e2b714', 'request_id': 'req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.422923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6d0bab81f114005b52e30b106e2b714', 'request_id': 'req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.434862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] The token cache is being invalidate because project f6d0bab81f114005b52e30b106e2b714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:55.435295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.436045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fed0164d-fce6-44b1-8e6e-ca1dc0e40b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.438137 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3686/7377] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:55 2022] DELETE /identity/v3/projects/f6d0bab81f114005b52e30b106e2b714 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:55.439225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.439606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.439979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.470125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:55.471531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:55.473163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e96f7c-da1a-4187-9071-190c3cbb5a83 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.473780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96f7c-da1a-4187-9071-190c3cbb5a83 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.474100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96f7c-da1a-4187-9071-190c3cbb5a83 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.475092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96f7c-da1a-4187-9071-190c3cbb5a83 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.475651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e96f7c-da1a-4187-9071-190c3cbb5a83 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.477716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e96f7c-da1a-4187-9071-190c3cbb5a83 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.479688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3687/7378] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:55 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:17:55.484447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.486686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:55.498749 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3692/7379] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:17:55 2022] POST /identity/v3/roles => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:55.500018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.500333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.500581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.526846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:55.528119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:55.594882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40b8be-e757-48c8-81af-39c349c464d0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.604815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.606453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.606895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.607332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] PATH_INFO: `/v3/users/04e9e877f53b41f7abe46ab0e492a438` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.608652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e9e877f53b41f7abe46ab0e492a438)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.610623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.617871 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3693/7380] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:55.617871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.619752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:55.634940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dedd4b-bd8b-4e63-b0aa-5baf5c98cc0f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.640207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.641272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.641563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.642029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.642629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.644651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.650778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:55.673003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e9e877f53b41f7abe46ab0e492a438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.683173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e9e877f53b41f7abe46ab0e492a438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.684057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e9e877f53b41f7abe46ab0e492a438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.685988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3694/7381] 10.222.0.33 () {60 vars in 1147 bytes} [Sat Sep 10 21:17:55 2022] POST /identity/v3/groups => generated 325 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:55.690557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.696198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e9e877f53b41f7abe46ab0e492a438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.697201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e9e877f53b41f7abe46ab0e492a438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.704760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bcb72-96ab-4adc-885e-60c48d0b39ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.712364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20f33b-f067-4c21-8c07-e4ef96ce198f tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.713181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f33b-f067-4c21-8c07-e4ef96ce198f tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.713623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f33b-f067-4c21-8c07-e4ef96ce198f tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.714023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20f33b-f067-4c21-8c07-e4ef96ce198f tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.720190 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3688/7382] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:55 2022] DELETE /identity/v3/users/04e9e877f53b41f7abe46ab0e492a438 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:55.721400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.723034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:55.743271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:55.772867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9a20f33b-f067-4c21-8c07-e4ef96ce198f tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:55.788481 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3695/7383] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:17:55 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:55.817239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bbbaa-8db7-47b4-8965-d2b976db8bcb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.822415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.823076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.823360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.823635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups/f27126087b9c4f37a5f2208b55534312/users/504de572cc524b1e86ed53f26d4d9e04` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.830422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=f27126087b9c4f37a5f2208b55534312, user_id=504de572cc524b1e86ed53f26d4d9e04)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.832377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.864642 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3689/7384] 10.222.0.33 () {60 vars in 1360 bytes} [Sat Sep 10 21:17:55 2022] PUT /identity/v3/groups/f27126087b9c4f37a5f2208b55534312/users/504de572cc524b1e86ed53f26d4d9e04 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:55.869771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20f33b-f067-4c21-8c07-e4ef96ce198f tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.882940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20f33b-f067-4c21-8c07-e4ef96ce198f tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:55.956925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:55.957613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:55.957935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:55.958232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/0632f43cbafb4bb8a221b11604456958/groups/f27126087b9c4f37a5f2208b55534312/roles/bfaecb09c8024bb6b6f0dd0ff7cbc76b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:55.960602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=0632f43cbafb4bb8a221b11604456958, group_id=f27126087b9c4f37a5f2208b55534312, role_id=bfaecb09c8024bb6b6f0dd0ff7cbc76b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:55.965888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:55.992566 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3696/7385] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:55 2022] PUT /identity/v3/domains/0632f43cbafb4bb8a221b11604456958/groups/f27126087b9c4f37a5f2208b55534312/roles/bfaecb09c8024bb6b6f0dd0ff7cbc76b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:55.996860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:55.999241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.032271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.034541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.129728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3261ae-e67a-48f1-afda-1eac5da0e084 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.138815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3dde5-4c43-4586-a798-47efeaed5088 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.144621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.145278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.145569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.145861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] PATH_INFO: `/v3/projects/3a91bf698a2c49bb8a4b43267e9fba5c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.147921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a91bf698a2c49bb8a4b43267e9fba5c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.150953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.212358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:56.218039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a91bf698a2c49bb8a4b43267e9fba5c', 'request_id': 'req-778470a3-2c67-4940-a488-53d72ba343d7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.221070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:56.227211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a91bf698a2c49bb8a4b43267e9fba5c', 'request_id': 'req-778470a3-2c67-4940-a488-53d72ba343d7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.235321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '504de572cc524b1e86ed53f26d4d9e04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e32a18ea720342d583553dcb59c7753a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.236052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.236345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.236623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] PATH_INFO: `/v3/auth/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.237074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a91bf698a2c49bb8a4b43267e9fba5c', 'request_id': 'req-778470a3-2c67-4940-a488-53d72ba343d7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.237179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.238285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.250961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] The token cache is being invalidate because project 3a91bf698a2c49bb8a4b43267e9fba5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:56.251283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.251928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-778470a3-2c67-4940-a488-53d72ba343d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.253458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3697/7386] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:17:56 2022] DELETE /identity/v3/projects/3a91bf698a2c49bb8a4b43267e9fba5c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:56.258536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0e41-7ff6-4a36-b100-679a89e3bc9b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.258834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0e41-7ff6-4a36-b100-679a89e3bc9b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.259117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0e41-7ff6-4a36-b100-679a89e3bc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.279456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c08e0e41-7ff6-4a36-b100-679a89e3bc9b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:56.280677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c08e0e41-7ff6-4a36-b100-679a89e3bc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.313758 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3690/7387] 10.222.0.33 () {58 vars in 1143 bytes} [Sat Sep 10 21:17:55 2022] GET /identity/v3/auth/domains => generated 1170 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:56.318163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.319981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.345603 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3698/7388] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:56.354061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6a32b-4109-49de-8bb4-4a8fb19288dd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.354358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6a32b-4109-49de-8bb4-4a8fb19288dd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.354637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6a32b-4109-49de-8bb4-4a8fb19288dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.378160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a6d6a32b-4109-49de-8bb4-4a8fb19288dd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:56.379371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a6d6a32b-4109-49de-8bb4-4a8fb19288dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.387495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8391f20-f0e2-45d0-af8f-34eb3686df52 tempest-TokensV3TestJSON-1651920208 tempest-TokensV3TestJSON-1651920208-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.388222 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3699/7389] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:56.392423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6a32b-4109-49de-8bb4-4a8fb19288dd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.402493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6a32b-4109-49de-8bb4-4a8fb19288dd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.407184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.407767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.408048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.408319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.408947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:17:56.409246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.410868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.426121 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3700/7390] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:17:56.427082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.438368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6b36b-9a91-484a-ac86-4a4f99088d4d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.443898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.444492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.444776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.444919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.445038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.445446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.445669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.445770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.446034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/0632f43cbafb4bb8a221b11604456958/groups/f27126087b9c4f37a5f2208b55534312/roles/bfaecb09c8024bb6b6f0dd0ff7cbc76b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.447644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.448272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=0632f43cbafb4bb8a221b11604456958, group_id=f27126087b9c4f37a5f2208b55534312, role_id=bfaecb09c8024bb6b6f0dd0ff7cbc76b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.451019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.456515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role bfaecb09c8024bb6b6f0dd0ff7cbc76b was removed from group f27126087b9c4f37a5f2208b55534312 on domain 0632f43cbafb4bb8a221b11604456958. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:56.456813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.457521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.472740 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3691/7391] 10.222.0.33 () {58 vars in 1467 bytes} [Sat Sep 10 21:17:56 2022] DELETE /identity/v3/domains/0632f43cbafb4bb8a221b11604456958/groups/f27126087b9c4f37a5f2208b55534312/roles/bfaecb09c8024bb6b6f0dd0ff7cbc76b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:56.477062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.477898 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3701/7392] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:56 2022] POST /identity/v3/projects => generated 407 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:56.478515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.480724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.482638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.523169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:56.554880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dde61-c87f-4e5a-be48-51708a0e195a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.561229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.561862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.562149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.562425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.562974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063d563-fa68-4a2a-92d0-d7b2bb61f584 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.563999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.568739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.593236 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3702/7393] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:56 2022] POST /identity/v3/users => generated 304 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:17:56.597489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.609483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078afe9-1e4c-429b-8e9f-e0a854638575 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.616757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.617689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.618241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.618753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.619728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.622638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.638153 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3703/7394] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:56 2022] GET /identity/v3/roles => generated 2343 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:56.638783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.639492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.639795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.640065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups/f27126087b9c4f37a5f2208b55534312/users/504de572cc524b1e86ed53f26d4d9e04` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.642087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=f27126087b9c4f37a5f2208b55534312, user_id=504de572cc524b1e86ed53f26d4d9e04)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.642228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.644012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.656294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '504de572cc524b1e86ed53f26d4d9e04'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.662941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c344c0-5ec8-4a39-a0f8-21df07ed1480 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.670581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72152a01-314f-422b-919e-1509907f39db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.671760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72152a01-314f-422b-919e-1509907f39db admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.672187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72152a01-314f-422b-919e-1509907f39db admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.672639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72152a01-314f-422b-919e-1509907f39db admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.673403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72152a01-314f-422b-919e-1509907f39db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.675485 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3692/7395] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:17:56 2022] DELETE /identity/v3/groups/f27126087b9c4f37a5f2208b55534312/users/504de572cc524b1e86ed53f26d4d9e04 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:56.676466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72152a01-314f-422b-919e-1509907f39db admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.680131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.684769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:56.704913 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3704/7396] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:56 2022] GET /identity/v3/roles => generated 2343 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:56.706274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72152a01-314f-422b-919e-1509907f39db admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.727031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72152a01-314f-422b-919e-1509907f39db admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.758143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c0c0d-39fb-463d-bbf3-7426f399a1dc tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.765774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190333cd-7694-4b42-84a7-c4fb1f8b9312 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.766427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190333cd-7694-4b42-84a7-c4fb1f8b9312 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.766797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190333cd-7694-4b42-84a7-c4fb1f8b9312 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.767059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190333cd-7694-4b42-84a7-c4fb1f8b9312 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups/f27126087b9c4f37a5f2208b55534312` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.768276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190333cd-7694-4b42-84a7-c4fb1f8b9312 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_group(group_id=f27126087b9c4f37a5f2208b55534312)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.770346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190333cd-7694-4b42-84a7-c4fb1f8b9312 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.810037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.810855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.811287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.811827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] PATH_INFO: `/v3/projects/cb8a070dd3714c4aa2a0cd73de393601/users/3e32b734264c462794a8cbb16cca5cf0/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.830516 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3693/7397] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:17:56 2022] DELETE /identity/v3/groups/f27126087b9c4f37a5f2208b55534312 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:56.831852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8a070dd3714c4aa2a0cd73de393601, user_id=3e32b734264c462794a8cbb16cca5cf0, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.835442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b15b-e1e8-40df-be3f-af629c3b3f3c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.835882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b15b-e1e8-40df-be3f-af629c3b3f3c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.836171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b15b-e1e8-40df-be3f-af629c3b3f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.836293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.853949 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3705/7398] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:56 2022] PUT /identity/v3/projects/cb8a070dd3714c4aa2a0cd73de393601/users/3e32b734264c462794a8cbb16cca5cf0/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:56.855014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:56.866529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-12c1b15b-e1e8-40df-be3f-af629c3b3f3c None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:56.867643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-12c1b15b-e1e8-40df-be3f-af629c3b3f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.870992 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3694/7399] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:56 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:17:56.872294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5766f-7855-4b21-bb06-c19995af2b71 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.872642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5766f-7855-4b21-bb06-c19995af2b71 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.872985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5766f-7855-4b21-bb06-c19995af2b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.898521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:56.908494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c2c5766f-7855-4b21-bb06-c19995af2b71 None None] MFA Rules not processed for user `3e32b734264c462794a8cbb16cca5cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:56.910636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c2c5766f-7855-4b21-bb06-c19995af2b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:56.933017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6225d-1533-4357-a9e4-2132cbd35731 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:56.937971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:56.938608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.938893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.939170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles/bfaecb09c8024bb6b6f0dd0ff7cbc76b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.940575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_role(role_id=bfaecb09c8024bb6b6f0dd0ff7cbc76b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:56.942213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:56.982762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role bfaecb09c8024bb6b6f0dd0ff7cbc76b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:56.983166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.984240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4e53e53b-5eb3-42e3-96ba-66f55e9eb564 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:56.986311 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3706/7400] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:56 2022] DELETE /identity/v3/roles/bfaecb09c8024bb6b6f0dd0ff7cbc76b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:56.987685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:56.987975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:56.988254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:56.991307 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3695/7401] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:56.992140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5766f-7855-4b21-bb06-c19995af2b71 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.004298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5766f-7855-4b21-bb06-c19995af2b71 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.010297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:57.011530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.062051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:57.066123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.066762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.067047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.067329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/0632f43cbafb4bb8a221b11604456958` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.069470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=0632f43cbafb4bb8a221b11604456958)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.071147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.076787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3707/7402] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:57.077600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.080300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.093608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '0632f43cbafb4bb8a221b11604456958'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.094589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '0632f43cbafb4bb8a221b11604456958'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.122373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '0632f43cbafb4bb8a221b11604456958'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.123013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '0632f43cbafb4bb8a221b11604456958'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.125476 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3696/7403] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:17:56 2022] PATCH /identity/v3/domains/0632f43cbafb4bb8a221b11604456958 => generated 276 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:57.128080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.131088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.164074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55500278-63b6-4e00-a3cd-275a17eadc0a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.208497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760657c3-2235-4d1f-a077-1fa778efca15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.213108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.213764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.214103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.214369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.242728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.244059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:57.245068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.245421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.245752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.246508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.250139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.281811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.288184 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3708/7404] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:17:57 2022] POST /identity/v3/projects => generated 409 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:57.291587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.292682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.297610 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3697/7405] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:17:57 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:57.298561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.300724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.320952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733984f4-4d9d-47b2-ae37-c698fbc66bd4 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.367746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce739c6-3a63-4e00-8e2a-02ca1744a9d2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.375711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.376481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.376878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.377233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/0632f43cbafb4bb8a221b11604456958` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.388628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=0632f43cbafb4bb8a221b11604456958)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.390376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.402879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.404138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.404138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.404332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.406020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.409457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.410069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:57.415488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '0632f43cbafb4bb8a221b11604456958'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.419525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '0632f43cbafb4bb8a221b11604456958'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.437884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3698/7406] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:17:57 2022] POST /identity/v3/users => generated 311 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:17:57.439014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.450698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29a553-4be2-4858-9ea2-3c4114e3a603 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.457067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.457812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.458151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.458443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.459368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.460962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.470852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0632f43cbafb4bb8a221b11604456958', 'request_id': 'req-311140f9-1f2e-4cbe-becd-c33d6d8c442a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.473194 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3699/7407] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:57 2022] GET /identity/v3/roles => generated 2112 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:17:57.477514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0632f43cbafb4bb8a221b11604456958', 'request_id': 'req-311140f9-1f2e-4cbe-becd-c33d6d8c442a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.477967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.484314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0632f43cbafb4bb8a221b11604456958', 'request_id': 'req-311140f9-1f2e-4cbe-becd-c33d6d8c442a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.488461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38cfd-eee3-4735-ad62-b4e912e28708 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.498860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] The token cache is being invalidate because project 0632f43cbafb4bb8a221b11604456958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:57.499181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.499821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.507874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '0632f43cbafb4bb8a221b11604456958', 'request_id': 'req-311140f9-1f2e-4cbe-becd-c33d6d8c442a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.508576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '0632f43cbafb4bb8a221b11604456958', 'request_id': 'req-311140f9-1f2e-4cbe-becd-c33d6d8c442a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.512139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '0632f43cbafb4bb8a221b11604456958', 'request_id': 'req-311140f9-1f2e-4cbe-becd-c33d6d8c442a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.517975 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3709/7408] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:17:57 2022] DELETE /identity/v3/domains/0632f43cbafb4bb8a221b11604456958 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:57.522910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.524321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.552156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.553709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.554023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.554302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] PATH_INFO: `/v3/projects/d605536c2cc64f098dc6cae130ead825/users/f2992745949b4fd6a06fefeca1b80cf1/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.570038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d605536c2cc64f098dc6cae130ead825, user_id=f2992745949b4fd6a06fefeca1b80cf1, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.572634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.586240 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3700/7409] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:57 2022] PUT /identity/v3/projects/d605536c2cc64f098dc6cae130ead825/users/f2992745949b4fd6a06fefeca1b80cf1/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:57.590099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.591732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.594354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311140f9-1f2e-4cbe-becd-c33d6d8c442a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.639958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:UPDATE=1,keystone:INSERT=6,keystone:DELETE=2,keystone:SELECT=91 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:57.656515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.657444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.657728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.658055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/941de5908b9d499e87b02790386a94bf/users/504de572cc524b1e86ed53f26d4d9e04/roles/645dc72beb294d498e26c5241621b6c2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.660507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=941de5908b9d499e87b02790386a94bf, user_id=504de572cc524b1e86ed53f26d4d9e04, role_id=645dc72beb294d498e26c5241621b6c2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.662309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db104-5a27-4101-a3fd-eaf4b5576fc2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.664449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.668002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.668613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.668900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.669176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.669840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.670013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role 645dc72beb294d498e26c5241621b6c2 was removed from user 504de572cc524b1e86ed53f26d4d9e04 on domain 941de5908b9d499e87b02790386a94bf. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:57.670305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.671010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.671686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.680265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:57.684240 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3701/7410] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:57 2022] GET /identity/v3/roles => generated 2112 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:57.685746 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3710/7411] 10.222.0.33 () {58 vars in 1464 bytes} [Sat Sep 10 21:17:57 2022] DELETE /identity/v3/domains/941de5908b9d499e87b02790386a94bf/users/504de572cc524b1e86ed53f26d4d9e04/roles/645dc72beb294d498e26c5241621b6c2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:57.688410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.690284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.690444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.692453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.704084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6a81d-a34e-40ba-9cb1-d20c9bdbb8a1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.761007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766fbc4-5bc9-4be9-b09d-03ed26f362b5 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.761803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaaf9f-a069-4750-be12-3a82044aed05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.762609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaaf9f-a069-4750-be12-3a82044aed05 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.762932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaaf9f-a069-4750-be12-3a82044aed05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.763238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaaf9f-a069-4750-be12-3a82044aed05 admin admin] PATH_INFO: `/v3/projects/d605536c2cc64f098dc6cae130ead825/users/f2992745949b4fd6a06fefeca1b80cf1/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.765241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbaaf9f-a069-4750-be12-3a82044aed05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d605536c2cc64f098dc6cae130ead825, user_id=f2992745949b4fd6a06fefeca1b80cf1, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.767121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.767755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.767969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbaaf9f-a069-4750-be12-3a82044aed05 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.768099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.768304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles/645dc72beb294d498e26c5241621b6c2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.769700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_role(role_id=645dc72beb294d498e26c5241621b6c2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.772120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.780802 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3702/7412] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:57 2022] PUT /identity/v3/projects/d605536c2cc64f098dc6cae130ead825/users/f2992745949b4fd6a06fefeca1b80cf1/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:57.791996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.792379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.792661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.809140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role 645dc72beb294d498e26c5241621b6c2 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:57.809428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.810128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.811943 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3711/7413] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:57 2022] DELETE /identity/v3/roles/645dc72beb294d498e26c5241621b6c2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:57.818754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.820143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.821590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] MFA Rules not processed for user `f2992745949b4fd6a06fefeca1b80cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:57.822818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.897708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6e632-9658-44d6-8b27-0074d12ff2c6 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:57.904661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:57.904896 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3703/7414] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:57 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:17:57.905499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:57.905791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:57.906105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/941de5908b9d499e87b02790386a94bf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:57.907389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=941de5908b9d499e87b02790386a94bf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:57.909274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:57.919219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.920727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:57.947296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '941de5908b9d499e87b02790386a94bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.948301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '941de5908b9d499e87b02790386a94bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.990683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '941de5908b9d499e87b02790386a94bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.991447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '941de5908b9d499e87b02790386a94bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:57.997127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3712/7415] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:17:57 2022] PATCH /identity/v3/domains/941de5908b9d499e87b02790386a94bf => generated 276 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:57.998337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:57.999867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.012297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f07bfb-ceef-4114-a117-db66489bbc16 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.026553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:58.044196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:58.071069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.071993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.072360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.072639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.074574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.079403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3e473-2714-44dc-84de-5dcf9847bde3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.086234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.086867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.087154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.087427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/941de5908b9d499e87b02790386a94bf` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.100201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=941de5908b9d499e87b02790386a94bf)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.102047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.127966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '941de5908b9d499e87b02790386a94bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.132386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '941de5908b9d499e87b02790386a94bf'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.155092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.156897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.169246 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3704/7416] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:57 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:17:58.191543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '941de5908b9d499e87b02790386a94bf', 'request_id': 'req-656b88bb-f679-4fba-9d7a-afd7a51b7021'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.198979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '941de5908b9d499e87b02790386a94bf', 'request_id': 'req-656b88bb-f679-4fba-9d7a-afd7a51b7021'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.206481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '941de5908b9d499e87b02790386a94bf', 'request_id': 'req-656b88bb-f679-4fba-9d7a-afd7a51b7021'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.222787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] The token cache is being invalidate because project 941de5908b9d499e87b02790386a94bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:58.223476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.224297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.234849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '941de5908b9d499e87b02790386a94bf', 'request_id': 'req-656b88bb-f679-4fba-9d7a-afd7a51b7021'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.235765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '941de5908b9d499e87b02790386a94bf', 'request_id': 'req-656b88bb-f679-4fba-9d7a-afd7a51b7021'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.239589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-656b88bb-f679-4fba-9d7a-afd7a51b7021 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '941de5908b9d499e87b02790386a94bf', 'request_id': 'req-656b88bb-f679-4fba-9d7a-afd7a51b7021'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.246803 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3713/7417] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:17:57 2022] DELETE /identity/v3/domains/941de5908b9d499e87b02790386a94bf => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:58.251246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:58.264213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d643f4-5f82-4b91-bdd4-71be740354c1 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.304543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaffa-fc55-41e6-a8d9-fa648f2b73f8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.305474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaffa-fc55-41e6-a8d9-fa648f2b73f8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.305844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaffa-fc55-41e6-a8d9-fa648f2b73f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.336162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e95aaffa-fc55-41e6-a8d9-fa648f2b73f8 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:58.336437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.337050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.337333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.337667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/7d0ad7e21560473ba0e509be9d374e5c/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/roles/cba5cdff14504901ace62120fd85a293` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.339463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e95aaffa-fc55-41e6-a8d9-fa648f2b73f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.343531 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3714/7418] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:58 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 21:17:58.347589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=7d0ad7e21560473ba0e509be9d374e5c, group_id=7279da4ca7bd4c2c9ec202e0ad1fe916, role_id=cba5cdff14504901ace62120fd85a293)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.348709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.349011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.349321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.352766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.363094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role cba5cdff14504901ace62120fd85a293 was removed from group 7279da4ca7bd4c2c9ec202e0ad1fe916 on domain 7d0ad7e21560473ba0e509be9d374e5c. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:58.363579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.364408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.377338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:58.378695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.385232 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3705/7419] 10.222.0.33 () {58 vars in 1467 bytes} [Sat Sep 10 21:17:58 2022] DELETE /identity/v3/domains/7d0ad7e21560473ba0e509be9d374e5c/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/roles/cba5cdff14504901ace62120fd85a293 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:58.386529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:58.388009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.461541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3715/7420] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:17:58.461541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:58.462633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.473759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:58.474657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cdf0b-007b-478c-9755-89502b5cb404 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.480989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.482030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.482379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.482724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.487157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.500448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:58.551656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9655718-c1e2-48ef-bdbb-b2f6b8b7441a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.558033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.559105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.559468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.559761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/users/504de572cc524b1e86ed53f26d4d9e04` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.561443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=7279da4ca7bd4c2c9ec202e0ad1fe916, user_id=504de572cc524b1e86ed53f26d4d9e04)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.563500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.572529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '504de572cc524b1e86ed53f26d4d9e04'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.576354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.578035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.589069 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3706/7421] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:17:58 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:17:58.590663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:58.591156 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3716/7422] 10.222.0.33 () {58 vars in 1344 bytes} [Sat Sep 10 21:17:58 2022] DELETE /identity/v3/groups/7279da4ca7bd4c2c9ec202e0ad1fe916/users/504de572cc524b1e86ed53f26d4d9e04 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:58.593149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:58.601552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c25ee-ba19-4d45-88cb-dba09a08c744 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.668871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa5656-e1e2-4dc8-8db6-ad3501099aa3 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.669446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa5656-e1e2-4dc8-8db6-ad3501099aa3 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.669730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa5656-e1e2-4dc8-8db6-ad3501099aa3 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.670041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa5656-e1e2-4dc8-8db6-ad3501099aa3 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.678940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826083fe-fedc-4ed9-8d5d-344388cec8fd tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.684591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.685231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.685518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.685794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/groups/7279da4ca7bd4c2c9ec202e0ad1fe916` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.686990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_group(group_id=7279da4ca7bd4c2c9ec202e0ad1fe916)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.688901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.718415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-bdaa5656-e1e2-4dc8-8db6-ad3501099aa3 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.727704 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3707/7423] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:17:58 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:58.737865 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3717/7424] 10.222.0.33 () {58 vars in 1227 bytes} [Sat Sep 10 21:17:58 2022] DELETE /identity/v3/groups/7279da4ca7bd4c2c9ec202e0ad1fe916 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:58.742440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa5656-e1e2-4dc8-8db6-ad3501099aa3 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:58.753089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa5656-e1e2-4dc8-8db6-ad3501099aa3 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.819148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.819797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.820084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.820369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles/cba5cdff14504901ace62120fd85a293` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.821778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_role(role_id=cba5cdff14504901ace62120fd85a293)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.823545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.845651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:58.863475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role cba5cdff14504901ace62120fd85a293 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:58.863801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.864610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.867526 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3708/7425] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:58 2022] DELETE /identity/v3/roles/cba5cdff14504901ace62120fd85a293 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:17:58.871453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:58.873213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:58.887417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698f3d2-cd82-46d3-87e0-d570926d37ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:58.940389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:58.953177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:58.953928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:58.954223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:58.954500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/7d0ad7e21560473ba0e509be9d374e5c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:58.955759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=7d0ad7e21560473ba0e509be9d374e5c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:58.957441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:58.994361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:58.995136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.025562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.026258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.031261 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3718/7426] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:17:58 2022] PATCH /identity/v3/domains/7d0ad7e21560473ba0e509be9d374e5c => generated 277 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:59.033752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:59.049409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc81fa2-3db1-46f8-8cbb-8c8199d49247 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:59.128563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:59.129211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.129501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.129784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/7d0ad7e21560473ba0e509be9d374e5c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.143382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=7d0ad7e21560473ba0e509be9d374e5c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:59.145431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:59.173964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.178727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.237460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c', 'request_id': 'req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.247422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c', 'request_id': 'req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.255546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c', 'request_id': 'req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.266241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] The token cache is being invalidate because project 7d0ad7e21560473ba0e509be9d374e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:59.266531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.267171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.275516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c', 'request_id': 'req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.276197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c', 'request_id': 'req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.276859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '7d0ad7e21560473ba0e509be9d374e5c', 'request_id': 'req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.285109 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3709/7427] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:17:59 2022] DELETE /identity/v3/domains/7d0ad7e21560473ba0e509be9d374e5c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:59.290038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:59.291442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:59.304166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea041f-305a-40ab-9989-b10cc03614c8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:59.360222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:59.360876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.361160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.361438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/d189ee72ffab4faeb54eac6d88be3d48/users/504de572cc524b1e86ed53f26d4d9e04/roles/1277ff8b8cc841a3a5c07a0de0ccf380` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.363610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d189ee72ffab4faeb54eac6d88be3d48, user_id=504de572cc524b1e86ed53f26d4d9e04, role_id=1277ff8b8cc841a3a5c07a0de0ccf380)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:59.366350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:59.371432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role 1277ff8b8cc841a3a5c07a0de0ccf380 was removed from user 504de572cc524b1e86ed53f26d4d9e04 on domain d189ee72ffab4faeb54eac6d88be3d48. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:59.371724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.372353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.385713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3719/7428] 10.222.0.33 () {58 vars in 1464 bytes} [Sat Sep 10 21:17:59 2022] DELETE /identity/v3/domains/d189ee72ffab4faeb54eac6d88be3d48/users/504de572cc524b1e86ed53f26d4d9e04/roles/1277ff8b8cc841a3a5c07a0de0ccf380 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:59.389995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:59.391403 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:59.404547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5361b1-4d74-4faa-8c3b-46727a4c3e79 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:59.430290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:59.463498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:59.464139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.464429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.464709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles/1277ff8b8cc841a3a5c07a0de0ccf380` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.466143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_role(role_id=1277ff8b8cc841a3a5c07a0de0ccf380)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:59.467744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:59.502164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role 1277ff8b8cc841a3a5c07a0de0ccf380 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:59.502454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.503117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.504869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3710/7429] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:17:59 2022] DELETE /identity/v3/roles/1277ff8b8cc841a3a5c07a0de0ccf380 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:59.509772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:59.511297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:59.524355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656082-1f31-4a2b-82a1-9a0e34b2f8b8 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:59.585182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:59.585847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.586235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.586519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/d189ee72ffab4faeb54eac6d88be3d48` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.587779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d189ee72ffab4faeb54eac6d88be3d48)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:59.589468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:59.612188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.612940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.650151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.650773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-756d9ec8-6a03-499a-82d1-8ca08e4c1f9c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.653144 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3720/7430] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:17:59 2022] PATCH /identity/v3/domains/d189ee72ffab4faeb54eac6d88be3d48 => generated 277 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:17:59.657215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:59.658708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:59.671632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e757ae1-a545-44a0-bc41-39c2b26ae2ed tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:59.729412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:59.730099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.730244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.730524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/d189ee72ffab4faeb54eac6d88be3d48` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.732998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.733295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.733582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.742706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d189ee72ffab4faeb54eac6d88be3d48)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:59.744485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:59.752825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:17:59.754058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:59.757393 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3721/7431] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:17:59 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 21:17:59.767229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.770687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.817599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48', 'request_id': 'req-0a88330f-9444-4ca2-aeb0-84708cd6a88a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.824885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48', 'request_id': 'req-0a88330f-9444-4ca2-aeb0-84708cd6a88a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.831793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48', 'request_id': 'req-0a88330f-9444-4ca2-aeb0-84708cd6a88a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.834078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:59.844902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] The token cache is being invalidate because project d189ee72ffab4faeb54eac6d88be3d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:17:59.845184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.845895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.853407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48', 'request_id': 'req-0a88330f-9444-4ca2-aeb0-84708cd6a88a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.854070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48', 'request_id': 'req-0a88330f-9444-4ca2-aeb0-84708cd6a88a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.854695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd189ee72ffab4faeb54eac6d88be3d48', 'request_id': 'req-0a88330f-9444-4ca2-aeb0-84708cd6a88a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:17:59.863075 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3711/7432] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:17:59 2022] DELETE /identity/v3/domains/d189ee72ffab4faeb54eac6d88be3d48 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:17:59.871359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:59.872800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:59.879302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:UPDATE=2,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:17:59.941772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35271e-dbf6-47a8-84b8-249de438aa8e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:59.947296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:59.947908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.948195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.948477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.949146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:59.950845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:17:59.962316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3722/7433] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:17:59.966230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:17:59.967648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:17:59.985340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88330f-9444-4ca2-aeb0-84708cd6a88a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:17:59.991142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:17:59.991477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:17:59.991768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:17:59.992049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects/7ede7ae8765244e4aa532a7f8dff0941/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:17:59.994091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7ede7ae8765244e4aa532a7f8dff0941, user_id=cf395756ee424d1e89c23dd476d43a38, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:17:59.996741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.010396 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3712/7434] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:17:59 2022] PUT /identity/v3/projects/7ede7ae8765244e4aa532a7f8dff0941/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:00.014707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.026204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0df02-67b5-42c1-bd9d-1abfdf32db17 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.088647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.089257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.089537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.089816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.090430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.091608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.159034 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3723/7435] 10.222.0.33 () {58 vars in 1146 bytes} [Sat Sep 10 21:18:00 2022] GET /identity/v3/auth/projects => generated 824 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:18:00.160696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.175316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e01a0-c9ac-4115-910d-18701d464a4a tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.186747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.187568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.187931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.192991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects/7ede7ae8765244e4aa532a7f8dff0941/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.195878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=7ede7ae8765244e4aa532a7f8dff0941, user_id=cf395756ee424d1e89c23dd476d43a38, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.198765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.209848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role d819f0a222ec4024ac762c673f345cf5 was removed from user cf395756ee424d1e89c23dd476d43a38 on project 7ede7ae8765244e4aa532a7f8dff0941. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:00.209848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:00.210374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:00.227002 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3713/7436] 10.222.0.33 () {58 vars in 1467 bytes} [Sat Sep 10 21:18:00 2022] DELETE /identity/v3/projects/7ede7ae8765244e4aa532a7f8dff0941/users/cf395756ee424d1e89c23dd476d43a38/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:00.232616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.234485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:00.248947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c627f36-dbe4-4b7e-8b5a-82c871c92a5b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.312378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.313126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.313472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.313815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.314544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.316511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.326126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:00.343655 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3724/7437] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/users => generated 323 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:00.344822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.355978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5fa86-a244-4291-8674-ac3651b30d5c tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.361982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.362610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.362878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.363573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.363825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.366366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.397858 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3714/7438] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/projects => generated 362 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:00.401083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.411204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4874f44-0bf1-4720-bf9d-5a02d0c11692 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.471315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.471921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.472206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.472482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.473158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.475131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.501340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3725/7439] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:00.506330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.517363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16497b10-96cd-4fc1-9b0e-ac250b3173a1 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.575845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.576456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.576772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.577058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.577712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.579345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.594141 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3715/7440] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/roles => generated 239 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:00.598469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.609609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7ddf-321a-4810-a11e-c259d70a3ecb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.614408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebbf8b-2abf-4f65-be22-0ff342453ef4 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.615655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebbf8b-2abf-4f65-be22-0ff342453ef4 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.615946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebbf8b-2abf-4f65-be22-0ff342453ef4 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.616224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebbf8b-2abf-4f65-be22-0ff342453ef4 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects/33b4b368fcc1450e830ba19eedb669ba/users/158c2628d582456f9d9f913a09d8a966/roles/50ff2b8d836d4ca494f9e213e1129a28` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.632343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebbf8b-2abf-4f65-be22-0ff342453ef4 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_grant(project_id=33b4b368fcc1450e830ba19eedb669ba, user_id=158c2628d582456f9d9f913a09d8a966, role_id=50ff2b8d836d4ca494f9e213e1129a28)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.635029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebbf8b-2abf-4f65-be22-0ff342453ef4 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.647840 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3726/7441] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:00 2022] PUT /identity/v3/projects/33b4b368fcc1450e830ba19eedb669ba/users/158c2628d582456f9d9f913a09d8a966/roles/50ff2b8d836d4ca494f9e213e1129a28 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:00.652361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.663506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58403b-f648-4c61-a4d2-408c17c1721b tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.725614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b19f6c-1d6a-4a4c-a91a-86db8a024809 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.726310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b19f6c-1d6a-4a4c-a91a-86db8a024809 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.726597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b19f6c-1d6a-4a4c-a91a-86db8a024809 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.726875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b19f6c-1d6a-4a4c-a91a-86db8a024809 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects/25d776330e274858a3322bbe9b336b59/users/158c2628d582456f9d9f913a09d8a966/roles/50ff2b8d836d4ca494f9e213e1129a28` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.729078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b19f6c-1d6a-4a4c-a91a-86db8a024809 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:create_grant(project_id=25d776330e274858a3322bbe9b336b59, user_id=158c2628d582456f9d9f913a09d8a966, role_id=50ff2b8d836d4ca494f9e213e1129a28)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.732084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b19f6c-1d6a-4a4c-a91a-86db8a024809 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.747079 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3716/7442] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:00 2022] PUT /identity/v3/projects/25d776330e274858a3322bbe9b336b59/users/158c2628d582456f9d9f913a09d8a966/roles/50ff2b8d836d4ca494f9e213e1129a28 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:00.757210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba3b2-e6ff-455e-9382-cb72cf91a0e0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.757437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba3b2-e6ff-455e-9382-cb72cf91a0e0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.757710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba3b2-e6ff-455e-9382-cb72cf91a0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.763059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2f78c-a51d-4eb2-a1d1-ce4ea22c5478 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.763469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2f78c-a51d-4eb2-a1d1-ce4ea22c5478 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.763758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2f78c-a51d-4eb2-a1d1-ce4ea22c5478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.778328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:00.784168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4ba2f78c-a51d-4eb2-a1d1-ce4ea22c5478 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:00.786357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4ba2f78c-a51d-4eb2-a1d1-ce4ea22c5478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:00.787422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-937ba3b2-e6ff-455e-9382-cb72cf91a0e0 None None] MFA Rules not processed for user `158c2628d582456f9d9f913a09d8a966`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:00.788913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-937ba3b2-e6ff-455e-9382-cb72cf91a0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:00.792113 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3727/7443] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/auth/tokens => generated 335 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:00.796334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171cfb-feeb-452d-a794-830184f4b7a5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.796625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171cfb-feeb-452d-a794-830184f4b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.796906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171cfb-feeb-452d-a794-830184f4b7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.808568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a9171cfb-feeb-452d-a794-830184f4b7a5 None None] MFA Rules not processed for user `158c2628d582456f9d9f913a09d8a966`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:00.809674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a9171cfb-feeb-452d-a794-830184f4b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:00.845266 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3717/7444] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:00.853159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda84f-958f-4481-b802-1a9f4c8f9d55 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.853447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda84f-958f-4481-b802-1a9f4c8f9d55 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.853722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda84f-958f-4481-b802-1a9f4c8f9d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.872954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-27bda84f-958f-4481-b802-1a9f4c8f9d55 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:00.873346 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3728/7445] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 10 21:18:00.874175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-27bda84f-958f-4481-b802-1a9f4c8f9d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:00.877365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9171cfb-feeb-452d-a794-830184f4b7a5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:00.877765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3718/7446] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 21:18:00.882356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8660-4d6b-4603-9a02-ff635a619843 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.882642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8660-4d6b-4603-9a02-ff635a619843 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.882919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8660-4d6b-4603-9a02-ff635a619843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.925919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:00.944782 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.server.flask.application [None req-247e8660-4d6b-4603-9a02-ff635a619843 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 10 21:18:00.946452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9171cfb-feeb-452d-a794-830184f4b7a5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:00.946688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3719/7447] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 66 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Sep 10 21:18:00.951545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d4467-0c2b-423c-9a65-b1d6cddaf08d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:00.951691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f1ac-f83d-4cd1-bc13-6d6ac9bbc314 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.952084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f1ac-f83d-4cd1-bc13-6d6ac9bbc314 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.952199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d4467-0c2b-423c-9a65-b1d6cddaf08d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.952438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d4467-0c2b-423c-9a65-b1d6cddaf08d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.952548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231f1ac-f83d-4cd1-bc13-6d6ac9bbc314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.952719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d4467-0c2b-423c-9a65-b1d6cddaf08d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.961898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d4467-0c2b-423c-9a65-b1d6cddaf08d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:00.968105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d4467-0c2b-423c-9a65-b1d6cddaf08d tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:00.988127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3729/7448] 10.222.0.33 () {56 vars in 1308 bytes} [Sat Sep 10 21:18:00 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:00.992658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454480f-46a2-4f86-b92c-d2482f7c9cbf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:00.992950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454480f-46a2-4f86-b92c-d2482f7c9cbf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:00.993245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454480f-46a2-4f86-b92c-d2482f7c9cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:00.996183 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.server.flask.application [None req-6231f1ac-f83d-4cd1-bc13-6d6ac9bbc314 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 10 21:18:00.997583 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3720/7449] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 47 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Sep 10 21:18:01.001632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023779-c62b-474c-a1bb-2b7ae4ff7495 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.001950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023779-c62b-474c-a1bb-2b7ae4ff7495 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.002256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023779-c62b-474c-a1bb-2b7ae4ff7495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.004362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d454480f-46a2-4f86-b92c-d2482f7c9cbf None None] MFA Rules not processed for user `158c2628d582456f9d9f913a09d8a966`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:01.006195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d454480f-46a2-4f86-b92c-d2482f7c9cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:01.027451 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.server.flask.application [None req-42023779-c62b-474c-a1bb-2b7ae4ff7495 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 10 21:18:01.028719 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3721/7450] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:01 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 28 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) Sep 10 21:18:01.070671 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3730/7451] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3225 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 406 bytes (2 switches on core 0) Sep 10 21:18:01.075022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42023779-c62b-474c-a1bb-2b7ae4ff7495 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.088519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42023779-c62b-474c-a1bb-2b7ae4ff7495 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.093026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.093641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.093956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.094232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/roles/50ff2b8d836d4ca494f9e213e1129a28` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.095616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_role(role_id=50ff2b8d836d4ca494f9e213e1129a28)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:01.097202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:01.136710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invalidating the token cache because role 50ff2b8d836d4ca494f9e213e1129a28 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:01.137000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.137663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.139461 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3722/7452] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:01 2022] DELETE /identity/v3/roles/50ff2b8d836d4ca494f9e213e1129a28 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:01.143628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454480f-46a2-4f86-b92c-d2482f7c9cbf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.155625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454480f-46a2-4f86-b92c-d2482f7c9cbf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.227405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.227960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.228428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.229174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects/25d776330e274858a3322bbe9b336b59` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.229847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_project(project_id=25d776330e274858a3322bbe9b336b59)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:01.232214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:01.296866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d776330e274858a3322bbe9b336b59', 'request_id': 'req-baf474e3-7c5e-48d6-b1ca-d4ad828428af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.304696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d776330e274858a3322bbe9b336b59', 'request_id': 'req-baf474e3-7c5e-48d6-b1ca-d4ad828428af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.317096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d776330e274858a3322bbe9b336b59', 'request_id': 'req-baf474e3-7c5e-48d6-b1ca-d4ad828428af'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.331695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] The token cache is being invalidate because project 25d776330e274858a3322bbe9b336b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:01.331959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.332683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.334554 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3731/7453] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:01 2022] DELETE /identity/v3/projects/25d776330e274858a3322bbe9b336b59 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:01.338664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.340178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:01.354093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23e0c9-c2e4-4ae8-afb2-3f2645e78443 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.417976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.418773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.419173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.419480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects/25d776330e274858a3322bbe9b336b59` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.428269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_project(project_id=25d776330e274858a3322bbe9b336b59)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:01.430439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Could not find project: 25d776330e274858a3322bbe9b336b59.: keystone.exception.ProjectNotFound: Could not find project: 25d776330e274858a3322bbe9b336b59. Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 10 21:18:01.443619 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 25d776330e274858a3322bbe9b336b59. Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:01.444807 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3723/7454] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:01 2022] DELETE /identity/v3/projects/25d776330e274858a3322bbe9b336b59 => generated 113 bytes in 107 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:18:01.449215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.450805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:01.459692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:01.467068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf474e3-7c5e-48d6-b1ca-d4ad828428af tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.473231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.473984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.474906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.475793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/projects/33b4b368fcc1450e830ba19eedb669ba` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.476458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_project(project_id=33b4b368fcc1450e830ba19eedb669ba)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:01.478543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:01.530568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b4b368fcc1450e830ba19eedb669ba', 'request_id': 'req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.538649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b4b368fcc1450e830ba19eedb669ba', 'request_id': 'req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.546310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b4b368fcc1450e830ba19eedb669ba', 'request_id': 'req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.560830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] The token cache is being invalidate because project 33b4b368fcc1450e830ba19eedb669ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:01.561127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.561775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.563459 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3732/7455] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:01 2022] DELETE /identity/v3/projects/33b4b368fcc1450e830ba19eedb669ba => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:01.568303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.570196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:01.585022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ad84-6663-4180-a4c5-fdcc32d2c4ce tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.589618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:UPDATE=2,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:01.652091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.652790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.653083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.653441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/users/158c2628d582456f9d9f913a09d8a966` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.654798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorizing `identity:delete_user(user_id=158c2628d582456f9d9f913a09d8a966)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:01.656886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:01.704170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158c2628d582456f9d9f913a09d8a966'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.713468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158c2628d582456f9d9f913a09d8a966'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.714267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158c2628d582456f9d9f913a09d8a966'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.722635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158c2628d582456f9d9f913a09d8a966'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.723838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158c2628d582456f9d9f913a09d8a966'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:01.745498 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3724/7456] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:01 2022] DELETE /identity/v3/users/158c2628d582456f9d9f913a09d8a966 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:01.750918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.762191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022c9b1-fa5d-4090-9d26-3d72486aeeb3 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.822298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.822932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.823218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.823496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/0632f43cbafb4bb8a221b11604456958` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.831535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Could not find domain: 0632f43cbafb4bb8a221b11604456958.: keystone.exception.DomainNotFound: Could not find domain: 0632f43cbafb4bb8a221b11604456958. Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0632f43cbafb4bb8a221b11604456958. Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:01.837445 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:01.838609 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 0632f43cbafb4bb8a221b11604456958. Sep 10 21:18:01.839612 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:18:01.840868 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3733/7457] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:18:01 2022] PATCH /identity/v3/domains/0632f43cbafb4bb8a221b11604456958 => generated 112 bytes in 88 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:18:01.842714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.844238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:01.858600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4536d-b413-456c-ad21-048cf5c54efb tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.864484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.865631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.866434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.866434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/941de5908b9d499e87b02790386a94bf` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.874037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Could not find domain: 941de5908b9d499e87b02790386a94bf.: keystone.exception.DomainNotFound: Could not find domain: 941de5908b9d499e87b02790386a94bf. Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 941de5908b9d499e87b02790386a94bf. Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:01.879347 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:01.881008 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 941de5908b9d499e87b02790386a94bf. Sep 10 21:18:01.882047 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:01.884143 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3725/7458] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:18:01 2022] PATCH /identity/v3/domains/941de5908b9d499e87b02790386a94bf => generated 112 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:18:01.889309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.902755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472843-0705-4f54-80d6-48a01fd5a5fa tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.908634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.909255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.909537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.909808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/7d0ad7e21560473ba0e509be9d374e5c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.916905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Could not find domain: 7d0ad7e21560473ba0e509be9d374e5c.: keystone.exception.DomainNotFound: Could not find domain: 7d0ad7e21560473ba0e509be9d374e5c. Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 7d0ad7e21560473ba0e509be9d374e5c. Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:01.922338 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:01.923702 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application with Lock( Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 7d0ad7e21560473ba0e509be9d374e5c. Sep 10 21:18:01.924680 ncdv-master-1 devstack@keystone.service[93527]: ERROR keystone.server.flask.application Sep 10 21:18:01.925727 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3734/7459] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:18:01 2022] PATCH /identity/v3/domains/7d0ad7e21560473ba0e509be9d374e5c => generated 112 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:18:01.927657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.940320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8330a4-60fd-46b9-b5ed-fde7c8b95852 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:01.945626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990aab83-a724-4f43-b7fe-116ed093cf15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf395756ee424d1e89c23dd476d43a38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c41cebf054f841f784bd54c652d99792', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:01.946292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aab83-a724-4f43-b7fe-116ed093cf15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:01.946648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aab83-a724-4f43-b7fe-116ed093cf15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:01.946931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aab83-a724-4f43-b7fe-116ed093cf15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] PATH_INFO: `/v3/domains/d189ee72ffab4faeb54eac6d88be3d48` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:01.954062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990aab83-a724-4f43-b7fe-116ed093cf15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-990aab83-a724-4f43-b7fe-116ed093cf15 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Could not find domain: d189ee72ffab4faeb54eac6d88be3d48.: keystone.exception.DomainNotFound: Could not find domain: d189ee72ffab4faeb54eac6d88be3d48. Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d189ee72ffab4faeb54eac6d88be3d48. Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:01.960017 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:01.961150 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d189ee72ffab4faeb54eac6d88be3d48. Sep 10 21:18:01.962257 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:01.963301 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3726/7460] 10.222.0.33 () {60 vars in 1249 bytes} [Sat Sep 10 21:18:01 2022] PATCH /identity/v3/domains/d189ee72ffab4faeb54eac6d88be3d48 => generated 112 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 10 21:18:01.965352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:01.969007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:02.037125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298908b-3b62-4f32-a70b-f247855b0a32 tempest-TokensV3TestJSON-1973641284 tempest-TokensV3TestJSON-1973641284-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:02.042368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:02.043029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:02.043309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:02.043587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] PATH_INFO: `/v3/users/51b977acabe34218b96055251a3f2196` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:02.044757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b977acabe34218b96055251a3f2196)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:02.046907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:02.062979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:02.093852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b977acabe34218b96055251a3f2196'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.105856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b977acabe34218b96055251a3f2196'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.112935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b977acabe34218b96055251a3f2196'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.120520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b977acabe34218b96055251a3f2196'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.121228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b977acabe34218b96055251a3f2196'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.127067 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3735/7461] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:01 2022] DELETE /identity/v3/users/51b977acabe34218b96055251a3f2196 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:02.132325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:02.132611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:02.132886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:02.152009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:02.153193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:02.212110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3727/7462] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:02.222085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:02.223477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:02.286072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601be91-616e-462f-8be7-f9f77f604ee7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:02.289860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:02.290459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:02.290742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:02.291011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:02.300341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:02.301803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:02.312676 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3736/7463] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:02.617698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:02.619665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:02.634698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bce5e-f6b0-4065-bf68-1b78c3d3bb5c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:02.649301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:02.650507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:02.650793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:02.651073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] PATH_INFO: `/v3/projects/7ede7ae8765244e4aa532a7f8dff0941` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:02.652398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ede7ae8765244e4aa532a7f8dff0941)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:02.654862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:02.658344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:02.718536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ede7ae8765244e4aa532a7f8dff0941', 'request_id': 'req-c472f553-5dfe-4880-a082-c1dd808aa59e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.725594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ede7ae8765244e4aa532a7f8dff0941', 'request_id': 'req-c472f553-5dfe-4880-a082-c1dd808aa59e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.734481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ede7ae8765244e4aa532a7f8dff0941', 'request_id': 'req-c472f553-5dfe-4880-a082-c1dd808aa59e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.746689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] The token cache is being invalidate because project 7ede7ae8765244e4aa532a7f8dff0941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:02.747245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.749285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.750819 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3728/7464] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:02 2022] DELETE /identity/v3/projects/7ede7ae8765244e4aa532a7f8dff0941 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:02.754660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:02.766586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76435e7c-1402-4127-a8d6-e1a8628e0fd2 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:02.829148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:02.829794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:02.830115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:02.830395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] PATH_INFO: `/v3/users/cf395756ee424d1e89c23dd476d43a38` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:02.831559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf395756ee424d1e89c23dd476d43a38)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:02.834334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:02.880052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:02.888370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf395756ee424d1e89c23dd476d43a38'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.901658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf395756ee424d1e89c23dd476d43a38'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.909393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf395756ee424d1e89c23dd476d43a38'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.918753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf395756ee424d1e89c23dd476d43a38'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.919388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf395756ee424d1e89c23dd476d43a38'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:02.925713 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3737/7465] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:02 2022] DELETE /identity/v3/users/cf395756ee424d1e89c23dd476d43a38 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:03.236201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:03.239368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:03.265566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472f553-5dfe-4880-a082-c1dd808aa59e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:03.331538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:03.332378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:03.332764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:03.333148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] PATH_INFO: `/v3/projects/c41cebf054f841f784bd54c652d99792` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:03.334589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41cebf054f841f784bd54c652d99792)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:03.336566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:03.387555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41cebf054f841f784bd54c652d99792', 'request_id': 'req-a905e8a2-97b4-4725-bd16-106fec665889'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.397100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41cebf054f841f784bd54c652d99792', 'request_id': 'req-a905e8a2-97b4-4725-bd16-106fec665889'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.405623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41cebf054f841f784bd54c652d99792', 'request_id': 'req-a905e8a2-97b4-4725-bd16-106fec665889'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.418633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] The token cache is being invalidate because project c41cebf054f841f784bd54c652d99792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:03.418857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.419698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.421867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3729/7466] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:03 2022] DELETE /identity/v3/projects/c41cebf054f841f784bd54c652d99792 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:03.426404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:03.428188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:03.443713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7333-954f-4bc0-a2c1-fec60a3c4af4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:03.502480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:03.503105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:03.503388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:03.503659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] PATH_INFO: `/v3/users/504de572cc524b1e86ed53f26d4d9e04` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:03.504914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=504de572cc524b1e86ed53f26d4d9e04)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:03.507031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:03.553269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504de572cc524b1e86ed53f26d4d9e04'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.565426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504de572cc524b1e86ed53f26d4d9e04'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.573244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504de572cc524b1e86ed53f26d4d9e04'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.581530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504de572cc524b1e86ed53f26d4d9e04'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.582155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-89b13bb0-424b-4165-acf3-fc8823c36213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504de572cc524b1e86ed53f26d4d9e04'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:03.589315 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3738/7467] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:03 2022] DELETE /identity/v3/users/504de572cc524b1e86ed53f26d4d9e04 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:03.904461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:03.906324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:03.924441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905e8a2-97b4-4725-bd16-106fec665889 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:03.952677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:04.002491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.003140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.003427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.003706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] PATH_INFO: `/v3/projects/e32a18ea720342d583553dcb59c7753a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.004856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32a18ea720342d583553dcb59c7753a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.007053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.071810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32a18ea720342d583553dcb59c7753a', 'request_id': 'req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.079819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32a18ea720342d583553dcb59c7753a', 'request_id': 'req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.091418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32a18ea720342d583553dcb59c7753a', 'request_id': 'req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.104631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] The token cache is being invalidate because project e32a18ea720342d583553dcb59c7753a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:04.104932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.105606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4dd196f-fdbd-4da7-850e-63d70a0ee20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.107508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3730/7468] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:03 2022] DELETE /identity/v3/projects/e32a18ea720342d583553dcb59c7753a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:04.112423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd44a24-0c47-42fc-bb45-39a5ee226ab5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.112956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd44a24-0c47-42fc-bb45-39a5ee226ab5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.112956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd44a24-0c47-42fc-bb45-39a5ee226ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.138104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5fd44a24-0c47-42fc-bb45-39a5ee226ab5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:04.139530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5fd44a24-0c47-42fc-bb45-39a5ee226ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:04.174868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:04.212853 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3739/7469] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:04.222205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b44b-1b43-4f1c-b3ad-a648cdb120ca None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.222478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b44b-1b43-4f1c-b3ad-a648cdb120ca None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.222761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3b44b-1b43-4f1c-b3ad-a648cdb120ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.252378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-23d3b44b-1b43-4f1c-b3ad-a648cdb120ca None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:04.253820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-23d3b44b-1b43-4f1c-b3ad-a648cdb120ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:04.264024 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3731/7470] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:04.268934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd44a24-0c47-42fc-bb45-39a5ee226ab5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.281870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd44a24-0c47-42fc-bb45-39a5ee226ab5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.288319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.289148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.289446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.289724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.290472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:04.290998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.292676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.309938 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3740/7471] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:04.311723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3b44b-1b43-4f1c-b3ad-a648cdb120ca None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.322874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3b44b-1b43-4f1c-b3ad-a648cdb120ca None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.327706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.328312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.328595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.328873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.329662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.331611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.365546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3732/7472] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:04 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:04.370013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.381109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d6c6c-a91f-46d2-a27f-e61a986d152c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.446066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.446692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.446978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.447257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.447939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.449922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.472581 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3741/7473] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:04 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:04.476791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.487745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77365f-4b53-4f1e-b7ab-e54937687755 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.493658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.494318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.494599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.494874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.495507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.497086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.507942 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3733/7474] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:04.512127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.522243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de822b3b-abb4-4f11-8de8-e374d4018909 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.526995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36826c8-bfd8-4299-bf6b-e0c21c310616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.527592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36826c8-bfd8-4299-bf6b-e0c21c310616 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.527877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36826c8-bfd8-4299-bf6b-e0c21c310616 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.528153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36826c8-bfd8-4299-bf6b-e0c21c310616 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.528789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36826c8-bfd8-4299-bf6b-e0c21c310616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.530414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36826c8-bfd8-4299-bf6b-e0c21c310616 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.542115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3742/7475] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:04.546076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.557351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3691372-e9d6-409c-9fca-b2a295b0bcb5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.562300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.563009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.563295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.563573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] PATH_INFO: `/v3/projects/7eb8c0108d064de5ab209472bb4c7223/users/38b042f99e2f4774935fd883c9a165e7/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.578991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb8c0108d064de5ab209472bb4c7223, user_id=38b042f99e2f4774935fd883c9a165e7, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.581650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.594546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3734/7476] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:04 2022] PUT /identity/v3/projects/7eb8c0108d064de5ab209472bb4c7223/users/38b042f99e2f4774935fd883c9a165e7/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:04.605042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde7610-0a8e-480d-82cd-997d44963d9c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.605332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde7610-0a8e-480d-82cd-997d44963d9c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.605612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde7610-0a8e-480d-82cd-997d44963d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.634651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0fde7610-0a8e-480d-82cd-997d44963d9c None None] MFA Rules not processed for user `38b042f99e2f4774935fd883c9a165e7`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:04.635871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0fde7610-0a8e-480d-82cd-997d44963d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:04.701563 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3743/7477] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:04.706346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.716871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df504a6a-4636-42b6-a6fe-b901f8bd6875 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.721283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e5230-93f8-4d14-8062-3b1e3dbc03ae tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b042f99e2f4774935fd883c9a165e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eb8c0108d064de5ab209472bb4c7223', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.721923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e5230-93f8-4d14-8062-3b1e3dbc03ae tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.722333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e5230-93f8-4d14-8062-3b1e3dbc03ae tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.722619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e5230-93f8-4d14-8062-3b1e3dbc03ae tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] PATH_INFO: `/v3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.724877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3735/7478] 10.222.0.33 () {58 vars in 1104 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/v3 => generated 255 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:04.729532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde7610-0a8e-480d-82cd-997d44963d9c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.738664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde7610-0a8e-480d-82cd-997d44963d9c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.743526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a40db2-1499-488e-aa9e-c480dc66c8c6 tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b042f99e2f4774935fd883c9a165e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eb8c0108d064de5ab209472bb4c7223', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.744119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40db2-1499-488e-aa9e-c480dc66c8c6 tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.744405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40db2-1499-488e-aa9e-c480dc66c8c6 tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.744681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a40db2-1499-488e-aa9e-c480dc66c8c6 tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] PATH_INFO: `/v3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.746042 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3744/7479] 10.222.0.33 () {58 vars in 1104 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/v3 => generated 255 bytes in 17 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:04.750810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e5230-93f8-4d14-8062-3b1e3dbc03ae tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.760647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e5230-93f8-4d14-8062-3b1e3dbc03ae tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.764921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25085a0b-b361-43f8-b464-3de87f021f5f tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38b042f99e2f4774935fd883c9a165e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eb8c0108d064de5ab209472bb4c7223', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.765513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25085a0b-b361-43f8-b464-3de87f021f5f tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.765802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25085a0b-b361-43f8-b464-3de87f021f5f tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.766111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25085a0b-b361-43f8-b464-3de87f021f5f tempest-TestApiDiscovery-287672513 tempest-TestApiDiscovery-287672513-project] PATH_INFO: `/v3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.767525 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3736/7480] 10.222.0.33 () {58 vars in 1104 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/v3 => generated 255 bytes in 17 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:04.772648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415c21-54ae-42ec-b8c7-96e1b57101f8 None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.772924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415c21-54ae-42ec-b8c7-96e1b57101f8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.773200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415c21-54ae-42ec-b8c7-96e1b57101f8 None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.774593 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3745/7481] 10.222.0.33 () {56 vars in 895 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/ => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 21:18:04.779253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef9f46-7c18-435d-afb8-04e7aa8581cc None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.779540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef9f46-7c18-435d-afb8-04e7aa8581cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.779817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef9f46-7c18-435d-afb8-04e7aa8581cc None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.781637 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3737/7482] 10.222.0.33 () {56 vars in 895 bytes} [Sat Sep 10 21:18:04 2022] GET /identity/ => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:18:04.785584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75415c21-54ae-42ec-b8c7-96e1b57101f8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:04.818734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:04.868078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75415c21-54ae-42ec-b8c7-96e1b57101f8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:04.873117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:04.873750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.874123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.874395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] PATH_INFO: `/v3/users/38b042f99e2f4774935fd883c9a165e7` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:04.875531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b042f99e2f4774935fd883c9a165e7)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:04.877526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:04.929844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b042f99e2f4774935fd883c9a165e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.942342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b042f99e2f4774935fd883c9a165e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.953405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b042f99e2f4774935fd883c9a165e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.962413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b042f99e2f4774935fd883c9a165e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.963049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b042f99e2f4774935fd883c9a165e7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:04.971768 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3746/7483] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:04 2022] DELETE /identity/v3/users/38b042f99e2f4774935fd883c9a165e7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:04.981439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:04.981970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:04.982085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.003354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:05.004566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.043514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:05.073658 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3738/7484] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:05.084718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.086404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.127761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.129171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.142557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae72be-3d61-4ece-b7fb-3f9a43105249 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:05.147496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:05.148014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.148306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.148588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] PATH_INFO: `/v3/users/3e32b734264c462794a8cbb16cca5cf0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.149731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e32b734264c462794a8cbb16cca5cf0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:05.151704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:05.162984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b358b5-b4bd-4117-b1fc-41163234a451 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:05.167937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cd4c9-d622-460b-987e-e701c6348986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:05.168734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cd4c9-d622-460b-987e-e701c6348986 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.169089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cd4c9-d622-460b-987e-e701c6348986 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.169458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cd4c9-d622-460b-987e-e701c6348986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.180992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cd4c9-d622-460b-987e-e701c6348986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:05.182691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cd4c9-d622-460b-987e-e701c6348986 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:05.196375 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3747/7485] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:05.209620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e32b734264c462794a8cbb16cca5cf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.217421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e32b734264c462794a8cbb16cca5cf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.218161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e32b734264c462794a8cbb16cca5cf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.226483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e32b734264c462794a8cbb16cca5cf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.227457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e32b734264c462794a8cbb16cca5cf0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.247797 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3739/7486] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:05 2022] DELETE /identity/v3/users/3e32b734264c462794a8cbb16cca5cf0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:05.253036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.253688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.254126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.284819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:05.287269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.343251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:05.365017 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3748/7487] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:05.371752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.373251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.451491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5407ac-3c2b-4871-b7c0-e769c08e4170 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:05.455405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c438a-8133-4825-a7fe-085259de1ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:05.456090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c438a-8133-4825-a7fe-085259de1ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.456444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c438a-8133-4825-a7fe-085259de1ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.456790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c438a-8133-4825-a7fe-085259de1ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.466552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c438a-8133-4825-a7fe-085259de1ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:05.468098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c438a-8133-4825-a7fe-085259de1ab0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:05.478527 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3740/7488] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:05.604459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.605855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.620139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac5553-faa9-4887-bbe6-351d55caad99 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:05.627343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:05.628218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.628522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.629132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] PATH_INFO: `/v3/projects/7eb8c0108d064de5ab209472bb4c7223` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.630776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb8c0108d064de5ab209472bb4c7223)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:05.632892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:05.697004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb8c0108d064de5ab209472bb4c7223', 'request_id': 'req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.704957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb8c0108d064de5ab209472bb4c7223', 'request_id': 'req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.712348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb8c0108d064de5ab209472bb4c7223', 'request_id': 'req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.726349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] The token cache is being invalidate because project 7eb8c0108d064de5ab209472bb4c7223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:05.726781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.727514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bdcb7a5a-2b80-4218-9346-3da3c9c7a075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.729153 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3749/7489] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:05 2022] DELETE /identity/v3/projects/7eb8c0108d064de5ab209472bb4c7223 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:05.734220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.734508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.734788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.755744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:05.756959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.806794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:05.821878 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3741/7490] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:05.829860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fac63-639a-432c-8245-16c072d11aa6 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.830424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fac63-639a-432c-8245-16c072d11aa6 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.830715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fac63-639a-432c-8245-16c072d11aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.840610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.842234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.857076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-091fac63-639a-432c-8245-16c072d11aa6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:05.858415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-091fac63-639a-432c-8245-16c072d11aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:05.864209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79a0ca-c848-43e6-977d-dfeae4215434 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:05.869525 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3750/7491] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:05.871215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:05.871880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.872186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.872463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] PATH_INFO: `/v3/projects/cb8a070dd3714c4aa2a0cd73de393601` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.873603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fac63-639a-432c-8245-16c072d11aa6 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.873760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8a070dd3714c4aa2a0cd73de393601)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:05.875928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:05.885239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fac63-639a-432c-8245-16c072d11aa6 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:05.891653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:05.892273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:05.892554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:05.893012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:05.893672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:05.894274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:05.895576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:05.908811 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3751/7492] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:05.913040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.923732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fa16d-898f-4c2d-bad4-9eeb49c94a5e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:05.936235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8a070dd3714c4aa2a0cd73de393601', 'request_id': 'req-03bba519-fba5-4807-ad35-f5cbf8167ad1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.944073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8a070dd3714c4aa2a0cd73de393601', 'request_id': 'req-03bba519-fba5-4807-ad35-f5cbf8167ad1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.953387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8a070dd3714c4aa2a0cd73de393601', 'request_id': 'req-03bba519-fba5-4807-ad35-f5cbf8167ad1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.972479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] The token cache is being invalidate because project cb8a070dd3714c4aa2a0cd73de393601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:05.972781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.973474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:05.976595 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3742/7493] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:05 2022] DELETE /identity/v3/projects/cb8a070dd3714c4aa2a0cd73de393601 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:05.980641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:05.982641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.019323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.020287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.020672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.022041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.022942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.024949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.068835 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3752/7494] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:05 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:06.073238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.077581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.083243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bba519-fba5-4807-ad35-f5cbf8167ad1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.143065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:06.168224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.169376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.169933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.170372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] PATH_INFO: `/v3/users/f2992745949b4fd6a06fefeca1b80cf1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.170844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14e149-b409-47f0-b8ae-bc0d7fbce438 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.173270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2992745949b4fd6a06fefeca1b80cf1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.175647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.176822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.178257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.178691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.179149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.179866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.182229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.215198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:06.222547 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3753/7495] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:06 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:06.224949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.237222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9476-23bf-4452-9de1-606256046ac2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.244287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.244974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.245474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.245754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.246482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.247241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2992745949b4fd6a06fefeca1b80cf1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.248053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.257428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2992745949b4fd6a06fefeca1b80cf1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.258771 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2992745949b4fd6a06fefeca1b80cf1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.262991 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3754/7496] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:06.264175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.269840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2992745949b4fd6a06fefeca1b80cf1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.271077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2992745949b4fd6a06fefeca1b80cf1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.277044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0168f1a-faa5-4bb1-812a-0cc671158e83 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.282896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.283683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.283968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.284255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] PATH_INFO: `/v3/projects/835848c477be4d71ba199abf168b1622/users/69240c1e9ebe42ef806bd042c0c4e457/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.301185 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3743/7497] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:05 2022] DELETE /identity/v3/users/f2992745949b4fd6a06fefeca1b80cf1 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:06.318923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835848c477be4d71ba199abf168b1622, user_id=69240c1e9ebe42ef806bd042c0c4e457, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.327502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.360036 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3755/7498] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:06 2022] PUT /identity/v3/projects/835848c477be4d71ba199abf168b1622/users/69240c1e9ebe42ef806bd042c0c4e457/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:06.364881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.366541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.483034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fb7c8-632d-4104-9b4c-4ab400022869 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.566200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a85db8-90ef-4ccf-995b-7ff9cd121cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.566563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85db8-90ef-4ccf-995b-7ff9cd121cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.566910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85db8-90ef-4ccf-995b-7ff9cd121cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.567251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a85db8-90ef-4ccf-995b-7ff9cd121cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.568232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a85db8-90ef-4ccf-995b-7ff9cd121cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.570033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a85db8-90ef-4ccf-995b-7ff9cd121cb5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.581927 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3744/7499] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:06.585946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.597127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec79b7-f23f-48ee-8066-61d9f1d42574 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.603563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.604277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.604571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.605085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] PATH_INFO: `/v3/projects/835848c477be4d71ba199abf168b1622/users/69240c1e9ebe42ef806bd042c0c4e457/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.607381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] RBAC: Authorizing `identity:create_grant(project_id=835848c477be4d71ba199abf168b1622, user_id=69240c1e9ebe42ef806bd042c0c4e457, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.610114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.622901 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3756/7500] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:06 2022] PUT /identity/v3/projects/835848c477be4d71ba199abf168b1622/users/69240c1e9ebe42ef806bd042c0c4e457/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:06.635689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a117-49db-44a3-a067-ed8a7e415390 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.636145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a117-49db-44a3-a067-ed8a7e415390 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.636426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a117-49db-44a3-a067-ed8a7e415390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.661524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.667164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.691547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1937a117-49db-44a3-a067-ed8a7e415390 None None] MFA Rules not processed for user `69240c1e9ebe42ef806bd042c0c4e457`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:06.693202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1937a117-49db-44a3-a067-ed8a7e415390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.742055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:06.742673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa1be4-bfd3-4596-aedd-06b4826e981c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.748041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.748628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.748931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.749206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] PATH_INFO: `/v3/projects/d605536c2cc64f098dc6cae130ead825` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.750749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d605536c2cc64f098dc6cae130ead825)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.752927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.774600 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3745/7501] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:06.779121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a117-49db-44a3-a067-ed8a7e415390 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.791197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a117-49db-44a3-a067-ed8a7e415390 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.802626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd605536c2cc64f098dc6cae130ead825', 'request_id': 'req-694fdc7c-40bf-45b3-9136-1ac555be4be5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.808151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd605536c2cc64f098dc6cae130ead825', 'request_id': 'req-694fdc7c-40bf-45b3-9136-1ac555be4be5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.814492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd605536c2cc64f098dc6cae130ead825', 'request_id': 'req-694fdc7c-40bf-45b3-9136-1ac555be4be5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.827410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] The token cache is being invalidate because project d605536c2cc64f098dc6cae130ead825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:06.827737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.828368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-694fdc7c-40bf-45b3-9136-1ac555be4be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:06.830181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3757/7502] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:06 2022] DELETE /identity/v3/projects/d605536c2cc64f098dc6cae130ead825 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:06.835599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32873a2b-4355-479c-bb80-40228e15a504 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.835890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32873a2b-4355-479c-bb80-40228e15a504 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.836406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32873a2b-4355-479c-bb80-40228e15a504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.851903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.852518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.852815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.853087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.853719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.855667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.857093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-32873a2b-4355-479c-bb80-40228e15a504 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:06.858324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-32873a2b-4355-479c-bb80-40228e15a504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.866622 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3758/7503] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:06.874035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814f440-5e56-4f9b-8a56-30ee5de38a0e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.874324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814f440-5e56-4f9b-8a56-30ee5de38a0e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.874597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814f440-5e56-4f9b-8a56-30ee5de38a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.884494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:06.890351 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3746/7504] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:06 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:06.895878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.898752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2814f440-5e56-4f9b-8a56-30ee5de38a0e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:06.899200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.900592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2814f440-5e56-4f9b-8a56-30ee5de38a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:06.965710 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3759/7505] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:06.966167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f0da3-78cd-4f7a-82cd-3be93f7c40ae admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.969720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814f440-5e56-4f9b-8a56-30ee5de38a0e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:06.971427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.972048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.972330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.972606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.973237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.975459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:06.979636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814f440-5e56-4f9b-8a56-30ee5de38a0e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:06.984775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:06.985256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:06.985544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:06.985852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:06.986518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:06.986829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:06.988584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.002895 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3747/7506] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:06 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:07.003412 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3760/7507] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:18:07.004263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.006518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.016464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd66bb-c23c-4819-a5b9-098f86227acf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.018011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e46f9-c458-4c13-8174-eb9defdc962b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.023619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.023797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.024248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.024375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.024556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.024674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.024840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.024978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.025515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.025633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.032542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.032740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.039395 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3748/7508] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:07.044808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.057034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6099b3-6524-4fc0-a99e-08ca08a68caa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.066845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.067522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.068535 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3761/7509] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:07 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:07.068712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.069142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.069924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.071975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.072468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.085766 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3749/7510] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:07.087347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.089568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:07.121440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:07.142340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067937a7-35c4-4492-8da1-91bb2d019396 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.148258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.148861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.149143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.149522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.150175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.152092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.152552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc31d9a-5b31-4cb0-a3d2-482d7a04ebd6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.176740 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3762/7511] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:07 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:07.176740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.189213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349a9a0-fe14-447c-b6ca-e8a1927d3c6d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.194466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.195252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.195554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.195822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] PATH_INFO: `/v3/projects/572c4c0b343e4648a95e99122fb8e9b2/users/8b58a764f4934354a02b463b7a9ecf2d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.205180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212cf1e-0aa2-4639-9158-fb7c477aa499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.205785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212cf1e-0aa2-4639-9158-fb7c477aa499 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.206090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212cf1e-0aa2-4639-9158-fb7c477aa499 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.206367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212cf1e-0aa2-4639-9158-fb7c477aa499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.208137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f212cf1e-0aa2-4639-9158-fb7c477aa499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:07.212186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=572c4c0b343e4648a95e99122fb8e9b2, user_id=8b58a764f4934354a02b463b7a9ecf2d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.214884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.230666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212cf1e-0aa2-4639-9158-fb7c477aa499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.232515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212cf1e-0aa2-4639-9158-fb7c477aa499 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.232929 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3763/7512] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:07 2022] PUT /identity/v3/projects/572c4c0b343e4648a95e99122fb8e9b2/users/8b58a764f4934354a02b463b7a9ecf2d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:07.233737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.250108 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3750/7513] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:07.252284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.253589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.253738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.272856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:07.299106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] MFA Rules not processed for user `8b58a764f4934354a02b463b7a9ecf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:07.300501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:07.313067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da7c0-17b4-46ef-9a11-9502289ab7f7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.318493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.319108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.319390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.319668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.320303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.321915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.333104 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3764/7514] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:07.336889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.347568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d33a8a-27fd-419f-a7bf-a672390c7375 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.352811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.353540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.354100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.354441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.355208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.357198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.372041 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3765/7515] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:07.373402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.378125 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3751/7516] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:07.386883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.387069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97820-e084-43de-9147-6cef0a0254d2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.388779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:07.392556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ff87d-9b32-477d-aef8-c0c90872a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.393425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ff87d-9b32-477d-aef8-c0c90872a174 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.393578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ff87d-9b32-477d-aef8-c0c90872a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.393762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ff87d-9b32-477d-aef8-c0c90872a174 admin admin] PATH_INFO: `/v3/projects/7a3aa71f8465400895874771cb4d5ffe/users/08856c2088314fa6bcc7340f3752e438/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.414792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ff87d-9b32-477d-aef8-c0c90872a174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3aa71f8465400895874771cb4d5ffe, user_id=08856c2088314fa6bcc7340f3752e438, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.417918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ff87d-9b32-477d-aef8-c0c90872a174 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.437667 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3766/7517] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:07 2022] PUT /identity/v3/projects/7a3aa71f8465400895874771cb4d5ffe/users/08856c2088314fa6bcc7340f3752e438/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:07.450417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e804f-57a3-4948-82e8-5bd3b7c28b9b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.450794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e804f-57a3-4948-82e8-5bd3b7c28b9b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.451079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e804f-57a3-4948-82e8-5bd3b7c28b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.482450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ca308-121d-43b7-acd8-a4f9201c602e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.492746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-239e804f-57a3-4948-82e8-5bd3b7c28b9b None None] MFA Rules not processed for user `08856c2088314fa6bcc7340f3752e438`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:07.497309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-239e804f-57a3-4948-82e8-5bd3b7c28b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:07.517619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:07.558619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.559512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.559901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.560251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.563226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:07.594821 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3767/7518] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:07.599410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e804f-57a3-4948-82e8-5bd3b7c28b9b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.659113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.661012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.668590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e804f-57a3-4948-82e8-5bd3b7c28b9b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.673420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.674063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.674349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.674628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.675267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.676608 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3752/7519] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:07.677269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.711242 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3768/7520] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:07 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:07.712378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.723024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ceaf0-ff30-4a1c-b39b-fd538c4d3efe service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.799697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.800588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.800943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.801293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.802266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.804495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.832721 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3753/7521] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:07 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:07.834405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.846360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe203-31b0-4fd7-a051-2959104b1120 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.853678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.854622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.854997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.855335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.856405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.858928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.861666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:07.879322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.879910 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3769/7522] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:07.881576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.882296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:07.895580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c60c50-24ac-49c2-8799-90d57555b8bf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.903304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.903756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84a2cc-e558-4ed3-a231-437d219540e9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:07.904400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.905184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.905473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] PATH_INFO: `/v3/projects/7065688e6b884b918526fe87f6491042/users/f34d3cbddd6d4af39e0ba401dfab4026/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.908807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:07.909657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:07.910062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:07.910415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] PATH_INFO: `/v3/users/2c96909ab4eb449d81769b4c1a52ec88` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:07.911715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c96909ab4eb449d81769b4c1a52ec88)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.914003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.925778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7065688e6b884b918526fe87f6491042, user_id=f34d3cbddd6d4af39e0ba401dfab4026, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:07.928877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:07.945753 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3770/7523] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:07 2022] PUT /identity/v3/projects/7065688e6b884b918526fe87f6491042/users/f34d3cbddd6d4af39e0ba401dfab4026/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:07.950790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:07.978336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c96909ab4eb449d81769b4c1a52ec88'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:07.987311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c96909ab4eb449d81769b4c1a52ec88'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:07.987798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c96909ab4eb449d81769b4c1a52ec88'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:07.989723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:07.998547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c96909ab4eb449d81769b4c1a52ec88'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:07.999868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c96909ab4eb449d81769b4c1a52ec88'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.026744 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3754/7524] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:07 2022] DELETE /identity/v3/users/2c96909ab4eb449d81769b4c1a52ec88 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:08.035955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef01b89-083f-419e-ae10-ca2c1254ab0c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.107475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d946a-c4a4-42e9-8555-ca28ef301003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:08.108473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d946a-c4a4-42e9-8555-ca28ef301003 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.108932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d946a-c4a4-42e9-8555-ca28ef301003 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.109276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d946a-c4a4-42e9-8555-ca28ef301003 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.110137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d946a-c4a4-42e9-8555-ca28ef301003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:08.111815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d946a-c4a4-42e9-8555-ca28ef301003 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:08.125964 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3771/7525] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:08.127675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.129231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.149080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765e973-ee52-4b70-a631-87411446f68d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.154729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:08.156220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.156571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.156916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] PATH_INFO: `/v3/projects/7065688e6b884b918526fe87f6491042/users/f34d3cbddd6d4af39e0ba401dfab4026/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.159022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7065688e6b884b918526fe87f6491042, user_id=f34d3cbddd6d4af39e0ba401dfab4026, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:08.162673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:08.174271 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3755/7526] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:08 2022] PUT /identity/v3/projects/7065688e6b884b918526fe87f6491042/users/f34d3cbddd6d4af39e0ba401dfab4026/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:08.185269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.185806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.186332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.216910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] MFA Rules not processed for user `f34d3cbddd6d4af39e0ba401dfab4026`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:08.218170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.255757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.257666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.316711 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3772/7527] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:08.330038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.331661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.337474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:08.363471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b6d2-ff13-4dcb-8bbb-6426d9a1ac6d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.371987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:08.371987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.371987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.371987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] PATH_INFO: `/v3/projects/e82a0bb2cc874614b0061f4c71ce082d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.384194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e82a0bb2cc874614b0061f4c71ce082d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:08.386457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:08.408624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98443321-3ba5-417f-a73e-dd4de777333a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.414610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ba0f2-cbe6-4ec3-91fc-1252c8ad5708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:08.415234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ba0f2-cbe6-4ec3-91fc-1252c8ad5708 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.415543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ba0f2-cbe6-4ec3-91fc-1252c8ad5708 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.415823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ba0f2-cbe6-4ec3-91fc-1252c8ad5708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.417595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-714ba0f2-cbe6-4ec3-91fc-1252c8ad5708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.458152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e82a0bb2cc874614b0061f4c71ce082d', 'request_id': 'req-e6a517ba-ef68-43bd-9ea2-43f9062b906f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.466328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e82a0bb2cc874614b0061f4c71ce082d', 'request_id': 'req-e6a517ba-ef68-43bd-9ea2-43f9062b906f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.475168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e82a0bb2cc874614b0061f4c71ce082d', 'request_id': 'req-e6a517ba-ef68-43bd-9ea2-43f9062b906f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.486832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] The token cache is being invalidate because project e82a0bb2cc874614b0061f4c71ce082d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:08.486987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ba0f2-cbe6-4ec3-91fc-1252c8ad5708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:08.487113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.487770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.488344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ba0f2-cbe6-4ec3-91fc-1252c8ad5708 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:08.489288 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3756/7528] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:08 2022] DELETE /identity/v3/projects/e82a0bb2cc874614b0061f4c71ce082d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:08.490459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.491753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.504099 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3773/7529] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:08.506127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75533bdc-e3d5-45e1-8b0f-b36c54fc35aa None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.506419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75533bdc-e3d5-45e1-8b0f-b36c54fc35aa None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.506767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75533bdc-e3d5-45e1-8b0f-b36c54fc35aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.527092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-75533bdc-e3d5-45e1-8b0f-b36c54fc35aa None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:08.528299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-75533bdc-e3d5-45e1-8b0f-b36c54fc35aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.544764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:08.574851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a517ba-ef68-43bd-9ea2-43f9062b906f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.580152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:08.580796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.581081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.581361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] PATH_INFO: `/v3/users/69240c1e9ebe42ef806bd042c0c4e457` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.582546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69240c1e9ebe42ef806bd042c0c4e457)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:08.585576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:08.600629 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3774/7530] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:08.610905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77a1fe-9138-488d-8f11-029a7528d528 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.611198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77a1fe-9138-488d-8f11-029a7528d528 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.611479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77a1fe-9138-488d-8f11-029a7528d528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.641346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bc77a1fe-9138-488d-8f11-029a7528d528 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:08.642628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bc77a1fe-9138-488d-8f11-029a7528d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.646946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69240c1e9ebe42ef806bd042c0c4e457'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.655512 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3775/7531] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:08.655812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69240c1e9ebe42ef806bd042c0c4e457'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.656698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69240c1e9ebe42ef806bd042c0c4e457'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.656943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77a1fe-9138-488d-8f11-029a7528d528 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.667773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69240c1e9ebe42ef806bd042c0c4e457'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.668623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77a1fe-9138-488d-8f11-029a7528d528 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.668769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1d09bf6b-e45f-4bd9-8c87-c26f03d21850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69240c1e9ebe42ef806bd042c0c4e457'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:08.675432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:08.676504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.677708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.678254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.679085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:08.679563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:08.683289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:08.699211 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3757/7532] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:08 2022] DELETE /identity/v3/users/69240c1e9ebe42ef806bd042c0c4e457 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:08.704816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3776/7533] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:08.705013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.705013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.705013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.705741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.718733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.731860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:08.733289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.746407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61273373-8009-47e6-88c4-340167fce281 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.789150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:08.836547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:08.837169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:08.837451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:08.837727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:08.838941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:08.839394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3758/7534] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:08.844807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:08.850355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.852124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:08.893926 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3777/7535] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:08 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:08.895069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:08.948828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d6943-3a8b-4500-b5d0-13f1d240e200 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:08.990777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:09.010347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86413d-4119-4861-b8f6-f320299ae167 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.025026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.026064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.026199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.026478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.027038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.030401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.075850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.076420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.076651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.076929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.077904 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3778/7536] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:08 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:09.081223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.091366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.092854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.106051 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3759/7537] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:09.113706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d493556-f12c-471f-a215-101f5246e99e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.120098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.120725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.121007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.121276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.121950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.123873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.137883 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3779/7538] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:09.139688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.150553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76e05-cd28-477e-959d-00f7cefaa914 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.157593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4523e0-5457-414b-b84f-cd6253c5dcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.158318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4523e0-5457-414b-b84f-cd6253c5dcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.158604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4523e0-5457-414b-b84f-cd6253c5dcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.158912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4523e0-5457-414b-b84f-cd6253c5dcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.159571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4523e0-5457-414b-b84f-cd6253c5dcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.162648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4523e0-5457-414b-b84f-cd6253c5dcb7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.182908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.192272 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3760/7539] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:09.198390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcec56-bad3-4bb5-aaae-af681e80939a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.206445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.207263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.207627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.207993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] PATH_INFO: `/v3/projects/f4f036f0122f4879aacb1db06eb8b588/users/b4791d13ab1c45bf8fabec70191426da/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.228321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f036f0122f4879aacb1db06eb8b588, user_id=b4791d13ab1c45bf8fabec70191426da, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.231563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.249139 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3780/7540] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:09 2022] PUT /identity/v3/projects/f4f036f0122f4879aacb1db06eb8b588/users/b4791d13ab1c45bf8fabec70191426da/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:09.262747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e8f1-79c1-4441-8692-ff6f140f599f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.263133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e8f1-79c1-4441-8692-ff6f140f599f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.263572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046e8f1-79c1-4441-8692-ff6f140f599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.303223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f046e8f1-79c1-4441-8692-ff6f140f599f None None] MFA Rules not processed for user `b4791d13ab1c45bf8fabec70191426da`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:09.305354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f046e8f1-79c1-4441-8692-ff6f140f599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:09.390577 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3761/7541] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:09.395055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.462973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ee47a-cbf5-495e-888f-0616f0c2c015 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.467737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.468335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.468618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.468895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.469562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.471514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.503339 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3781/7542] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:09 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:09.504356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046e8f1-79c1-4441-8692-ff6f140f599f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.516188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046e8f1-79c1-4441-8692-ff6f140f599f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.579433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.579992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.580273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.580625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.581181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.583148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.606972 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3762/7543] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:09 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:09.611306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.617817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:09.623831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b0b3-8097-4fdf-85a2-e052d8f4a543 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.631560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.632440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.632821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.633175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.633904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.635537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.646184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.650540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:09.653689 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3782/7544] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:09.658397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.667327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545c28e-4003-4b94-81e4-6979b405a669 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.672474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.673125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.673508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.673679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] PATH_INFO: `/v3/projects/835848c477be4d71ba199abf168b1622` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.675006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835848c477be4d71ba199abf168b1622)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.677886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.683361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c822f-93f1-452b-8b80-7af98587d2e5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.690406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.691433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.692090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.692509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] PATH_INFO: `/v3/projects/48270d4f7205461aa0563e2ef69c6c44/users/09038e5611424270a0afa7928b3cf791/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.710108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:09.713921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48270d4f7205461aa0563e2ef69c6c44, user_id=09038e5611424270a0afa7928b3cf791, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.716916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.733272 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3783/7545] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:09 2022] PUT /identity/v3/projects/48270d4f7205461aa0563e2ef69c6c44/users/09038e5611424270a0afa7928b3cf791/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:09.737085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.743674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835848c477be4d71ba199abf168b1622', 'request_id': 'req-5a226a69-01fd-4f41-b7d8-f4774710cce3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.750754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835848c477be4d71ba199abf168b1622', 'request_id': 'req-5a226a69-01fd-4f41-b7d8-f4774710cce3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.759574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835848c477be4d71ba199abf168b1622', 'request_id': 'req-5a226a69-01fd-4f41-b7d8-f4774710cce3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.774296 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] The token cache is being invalidate because project 835848c477be4d71ba199abf168b1622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:09.774668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.775430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.778291 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3763/7546] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:09 2022] DELETE /identity/v3/projects/835848c477be4d71ba199abf168b1622 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:09.785890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.788535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:09.824158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5cba-f73b-4629-8fe6-64f184703c04 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.875350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a226a69-01fd-4f41-b7d8-f4774710cce3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.881417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.882264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.882889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.883015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] PATH_INFO: `/v3/users/8b58a764f4934354a02b463b7a9ecf2d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.884068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b58a764f4934354a02b463b7a9ecf2d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.893356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.899962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.901592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.902217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.902512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.903308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.908250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.924747 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3784/7547] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:09.937472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:09.939350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:09.953863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7bf92-e8ca-4f16-8320-8c5abfb09639 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:09.960662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b58a764f4934354a02b463b7a9ecf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.961294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a10322-df82-4bd1-a3f1-b811dd5d9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:09.962297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a10322-df82-4bd1-a3f1-b811dd5d9090 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.962683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a10322-df82-4bd1-a3f1-b811dd5d9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.962962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a10322-df82-4bd1-a3f1-b811dd5d9090 admin admin] PATH_INFO: `/v3/projects/48270d4f7205461aa0563e2ef69c6c44/users/09038e5611424270a0afa7928b3cf791/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:09.965644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a10322-df82-4bd1-a3f1-b811dd5d9090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48270d4f7205461aa0563e2ef69c6c44, user_id=09038e5611424270a0afa7928b3cf791, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:09.975070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a10322-df82-4bd1-a3f1-b811dd5d9090 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:09.975335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b58a764f4934354a02b463b7a9ecf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.975335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b58a764f4934354a02b463b7a9ecf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.983560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b58a764f4934354a02b463b7a9ecf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.984386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b58a764f4934354a02b463b7a9ecf2d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:09.994705 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3785/7548] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:09 2022] PUT /identity/v3/projects/48270d4f7205461aa0563e2ef69c6c44/users/09038e5611424270a0afa7928b3cf791/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:09.996385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaef011-3874-498c-9e12-63b682033987 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:09.997044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaef011-3874-498c-9e12-63b682033987 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:09.997044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaef011-3874-498c-9e12-63b682033987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:10.012181 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3764/7549] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:09 2022] DELETE /identity/v3/users/8b58a764f4934354a02b463b7a9ecf2d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:10.045805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6eaef011-3874-498c-9e12-63b682033987 None None] MFA Rules not processed for user `09038e5611424270a0afa7928b3cf791`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:10.048251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6eaef011-3874-498c-9e12-63b682033987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:10.166165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:10.233594 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3786/7550] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:10.245846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:10.247301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:10.300978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:10.426714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaef011-3874-498c-9e12-63b682033987 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:10.428474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6eaef011-3874-498c-9e12-63b682033987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:10.430616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1fb9f-ab93-4ed9-9ddf-e56990491731 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:10.439736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c7674-a7c3-4b63-8064-05221c64db2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:10.440602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c7674-a7c3-4b63-8064-05221c64db2f service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:10.441096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c7674-a7c3-4b63-8064-05221c64db2f service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:10.441952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c7674-a7c3-4b63-8064-05221c64db2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:10.443504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c29c7674-a7c3-4b63-8064-05221c64db2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:10.545007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c7674-a7c3-4b63-8064-05221c64db2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:10.547015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c7674-a7c3-4b63-8064-05221c64db2f service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:10.554684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaef011-3874-498c-9e12-63b682033987 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:10.561026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:10.561886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:10.562862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:10.562862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] PATH_INFO: `/v3/projects/572c4c0b343e4648a95e99122fb8e9b2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:10.564077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=572c4c0b343e4648a95e99122fb8e9b2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:10.566247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:10.585904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3765/7551] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:10.651118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572c4c0b343e4648a95e99122fb8e9b2', 'request_id': 'req-8a0a25d1-d62b-40ac-9686-d210686ef2ad'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:10.668886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572c4c0b343e4648a95e99122fb8e9b2', 'request_id': 'req-8a0a25d1-d62b-40ac-9686-d210686ef2ad'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:10.676068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572c4c0b343e4648a95e99122fb8e9b2', 'request_id': 'req-8a0a25d1-d62b-40ac-9686-d210686ef2ad'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:10.703686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] The token cache is being invalidate because project 572c4c0b343e4648a95e99122fb8e9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:10.704104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:10.704796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8a0a25d1-d62b-40ac-9686-d210686ef2ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:10.710369 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3787/7552] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:10 2022] DELETE /identity/v3/projects/572c4c0b343e4648a95e99122fb8e9b2 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:10.715189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55dde7-5130-4483-bd0e-06f32e6c0ee0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:10.715657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55dde7-5130-4483-bd0e-06f32e6c0ee0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:10.716088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55dde7-5130-4483-bd0e-06f32e6c0ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:10.743212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2f55dde7-5130-4483-bd0e-06f32e6c0ee0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:10.744582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2f55dde7-5130-4483-bd0e-06f32e6c0ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:10.828737 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3766/7553] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:10.840452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033808-e528-4376-b924-3d678559273f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:10.841040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033808-e528-4376-b924-3d678559273f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:10.841405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033808-e528-4376-b924-3d678559273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:10.876286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6f033808-e528-4376-b924-3d678559273f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:10.882490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f033808-e528-4376-b924-3d678559273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:10.927417 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3788/7554] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:10.928766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55dde7-5130-4483-bd0e-06f32e6c0ee0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:10.941589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55dde7-5130-4483-bd0e-06f32e6c0ee0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:10.955427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:10.956243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:10.956534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:10.956963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:10.957645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:10.959276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:10.962443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:10.992522 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3767/7555] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:10.997220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f033808-e528-4376-b924-3d678559273f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.021819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f033808-e528-4376-b924-3d678559273f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.034836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.035817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.036337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.036827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.037671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.050864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.062328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.065369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:11.164783 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3789/7556] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:10 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:11.170849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.187249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:11.206370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790caa0b-79af-4bf4-861e-c9132a06c489 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.270256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:11.274643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.277394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.277894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.278355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.281032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:11.294793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d57c1-d253-4ede-81de-d82d4bad2088 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.302761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.303747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.304205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.304612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.305282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.307662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.357181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3790/7557] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:11 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:11.358404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.372982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c5ed-71ec-4e7b-a014-0619ada21dbd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.379638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.381365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.381776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.382176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.383289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.384701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.397953 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3791/7558] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:11.403499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.404367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.406782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.417496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de222ea-9c03-4e6e-ad43-5c231147d127 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.423888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b0006-1953-4e4d-b129-22f453b6d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.426229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b0006-1953-4e4d-b129-22f453b6d8ab admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.426636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b0006-1953-4e4d-b129-22f453b6d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.427738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b0006-1953-4e4d-b129-22f453b6d8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.428627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b0006-1953-4e4d-b129-22f453b6d8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.430937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b0006-1953-4e4d-b129-22f453b6d8ab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.441109 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3768/7559] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:11.445871 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3792/7560] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:11.449751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.463096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84402f6c-9e86-4c55-baf2-0354ababc044 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.468139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.471162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.471162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.471162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] PATH_INFO: `/v3/projects/84bb2a0b95cc44c8b7063269058a7c6c/users/3f73ab2bccfd48b1939710dded7bbace/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.492090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] RBAC: Authorizing `identity:create_grant(project_id=84bb2a0b95cc44c8b7063269058a7c6c, user_id=3f73ab2bccfd48b1939710dded7bbace, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.495785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.519067 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3769/7561] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:11 2022] PUT /identity/v3/projects/84bb2a0b95cc44c8b7063269058a7c6c/users/3f73ab2bccfd48b1939710dded7bbace/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:11.534106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30387632-00c5-47a0-8798-834cff1a231c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.534106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30387632-00c5-47a0-8798-834cff1a231c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.534106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30387632-00c5-47a0-8798-834cff1a231c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.566613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-30387632-00c5-47a0-8798-834cff1a231c None None] MFA Rules not processed for user `3f73ab2bccfd48b1939710dded7bbace`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:11.568728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-30387632-00c5-47a0-8798-834cff1a231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:11.657919 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3793/7562] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:11.662687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.778496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34798c6-63b3-433a-9678-50b6bc79caae admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.784383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.785084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.785427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.785778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.786762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.789353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.829990 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3770/7563] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:11 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:11.834430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30387632-00c5-47a0-8798-834cff1a231c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:11.848255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30387632-00c5-47a0-8798-834cff1a231c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:11.925670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444013b-f1bc-467f-a8b4-e96565d14356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:11.926435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444013b-f1bc-467f-a8b4-e96565d14356 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:11.926716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444013b-f1bc-467f-a8b4-e96565d14356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:11.927524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444013b-f1bc-467f-a8b4-e96565d14356 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:11.928653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444013b-f1bc-467f-a8b4-e96565d14356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:11.930942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444013b-f1bc-467f-a8b4-e96565d14356 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:11.971699 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3794/7564] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:11 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:11.975491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:12.001370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:12.010593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd3d72-4ddd-4165-9adb-feb0d7d2f6e8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:12.028754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:12.029506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.030221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.031749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.032499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:12.034282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:12.046142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7681-651a-47f3-9aca-637158315772 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.046142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7681-651a-47f3-9aca-637158315772 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.046142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c7681-651a-47f3-9aca-637158315772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.064870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3771/7565] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:12.066835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:12.080113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6161c6-04f5-4607-b704-3e8b7751b463 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:12.086629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:12.087498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.087862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.088277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.088993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:12.090950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:12.107171 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3772/7566] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:12.109276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:12.112983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-363c7681-651a-47f3-9aca-637158315772 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:12.114911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-363c7681-651a-47f3-9aca-637158315772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:12.130786 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3795/7567] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:12 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:12.130786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c7681-651a-47f3-9aca-637158315772 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:12.130786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-363c7681-651a-47f3-9aca-637158315772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:12.131061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ad2ad-0ba2-43d9-9958-b17cbe442fd1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:12.135258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71208f43-e3d8-474b-a646-b440323c4501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:12.136086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208f43-e3d8-474b-a646-b440323c4501 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.136503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208f43-e3d8-474b-a646-b440323c4501 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.136876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208f43-e3d8-474b-a646-b440323c4501 admin admin] PATH_INFO: `/v3/projects/ce671e0785a54cc5b2862ed8cbd80034/users/0fa8da0ae1d44de3867b823a13c15aed/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.141532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:12.162130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71208f43-e3d8-474b-a646-b440323c4501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce671e0785a54cc5b2862ed8cbd80034, user_id=0fa8da0ae1d44de3867b823a13c15aed, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:12.166353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71208f43-e3d8-474b-a646-b440323c4501 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:12.197915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3773/7568] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:12 2022] PUT /identity/v3/projects/ce671e0785a54cc5b2862ed8cbd80034/users/0fa8da0ae1d44de3867b823a13c15aed/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:12.201103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480061-36b3-498e-9a93-240097532d12 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.201397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480061-36b3-498e-9a93-240097532d12 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.201675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480061-36b3-498e-9a93-240097532d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.217161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c7681-651a-47f3-9aca-637158315772 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:12.250325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-51480061-36b3-498e-9a93-240097532d12 None None] MFA Rules not processed for user `0fa8da0ae1d44de3867b823a13c15aed`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:12.251559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-51480061-36b3-498e-9a93-240097532d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:12.292357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:12.293126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.293661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.294148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.372885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:12.375871 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3774/7569] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:12.391036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51480061-36b3-498e-9a93-240097532d12 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:12.391983 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3796/7570] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:18:12 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:12.394139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-51480061-36b3-498e-9a93-240097532d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:12.475790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:12.477683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:12.498792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51480061-36b3-498e-9a93-240097532d12 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:12.503756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:12.506622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.507791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.508100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.607040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c66c88-ebf3-4bed-bcd2-9c05f2c0dcc5 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:12.615103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2241d8-6d2a-4954-9eda-e15d34a645ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:12.616210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2241d8-6d2a-4954-9eda-e15d34a645ef service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.616695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2241d8-6d2a-4954-9eda-e15d34a645ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.617197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2241d8-6d2a-4954-9eda-e15d34a645ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.619811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ba2241d8-6d2a-4954-9eda-e15d34a645ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:12.654013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:12.655484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:12.670730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3775/7571] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:12.703446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:12.745477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2241d8-6d2a-4954-9eda-e15d34a645ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:12.747406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2241d8-6d2a-4954-9eda-e15d34a645ef service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:12.777512 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3797/7572] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:12.873916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:12.879497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:12.894980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1c495-00cc-48b4-b2ff-3c9da75b6882 service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:12.908082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:12.909754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:12.910427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:12.910937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:12.925327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:12.928453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:12.949371 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3776/7573] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:13.397585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765f5a-0902-45ed-ae17-861b885a3e47 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:13.398151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765f5a-0902-45ed-ae17-861b885a3e47 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:13.398151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02765f5a-0902-45ed-ae17-861b885a3e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:13.419107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:13.421941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:13.429634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-02765f5a-0902-45ed-ae17-861b885a3e47 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:13.432723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-02765f5a-0902-45ed-ae17-861b885a3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:13.437308 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3798/7574] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:13 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:13.441383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02765f5a-0902-45ed-ae17-861b885a3e47 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:13.509487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f01109-323a-4a6d-8316-95ffea27003b service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:13.513939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3fb7f-ee5f-4a59-9bb2-824337d2362a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:13.514581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3fb7f-ee5f-4a59-9bb2-824337d2362a service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:13.514944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3fb7f-ee5f-4a59-9bb2-824337d2362a service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:13.515592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3fb7f-ee5f-4a59-9bb2-824337d2362a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:13.517079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5ee3fb7f-ee5f-4a59-9bb2-824337d2362a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:13.541497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02765f5a-0902-45ed-ae17-861b885a3e47 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:13.549088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:13.550594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:13.550884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:13.551167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] PATH_INFO: `/v3/users/3f73ab2bccfd48b1939710dded7bbace` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:13.552947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f73ab2bccfd48b1939710dded7bbace)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:13.556004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:13.614160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3fb7f-ee5f-4a59-9bb2-824337d2362a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:13.615816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3fb7f-ee5f-4a59-9bb2-824337d2362a service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:13.629406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f73ab2bccfd48b1939710dded7bbace'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:13.629755 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3777/7575] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:13.633790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59902e-97a4-42b3-8079-5014768f7159 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:13.634131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59902e-97a4-42b3-8079-5014768f7159 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:13.634376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59902e-97a4-42b3-8079-5014768f7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:13.654925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f73ab2bccfd48b1939710dded7bbace'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:13.664886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f73ab2bccfd48b1939710dded7bbace'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:13.677163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f73ab2bccfd48b1939710dded7bbace'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-5a59902e-97a4-42b3-8079-5014768f7159 None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Sep 10 21:18:13.677589 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:13.678867 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:13.679934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3778/7576] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:13 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 49 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 21:18:13.684041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfff38-59c0-44dd-bbc5-bc1bea201da8 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:13.684288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfff38-59c0-44dd-bbc5-bc1bea201da8 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:13.684573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfff38-59c0-44dd-bbc5-bc1bea201da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:13.690620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-25a5688d-f9fd-4f79-8315-e2c1be6ee095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f73ab2bccfd48b1939710dded7bbace'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:13.710398 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3799/7577] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:13 2022] DELETE /identity/v3/users/3f73ab2bccfd48b1939710dded7bbace => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:13.716588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:13.717040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:13.717328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:13.749493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:13.750779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:13.768176 ncdv-master-1 devstack@keystone.service[93528]: WARNING keystone.server.flask.application [None req-f5bfff38-59c0-44dd-bbc5-bc1bea201da8 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 10 21:18:13.772200 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3779/7578] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:13 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 89 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 10 21:18:13.781060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446de68-f8a9-4209-b418-44de54331204 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:13.781281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446de68-f8a9-4209-b418-44de54331204 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:13.781565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446de68-f8a9-4209-b418-44de54331204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:13.857190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:13.866188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0446de68-f8a9-4209-b418-44de54331204 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:13.867811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0446de68-f8a9-4209-b418-44de54331204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:13.893504 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3800/7579] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:13.911596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:13.914657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:14.045969 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3780/7580] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:14.050442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446de68-f8a9-4209-b418-44de54331204 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:14.066599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446de68-f8a9-4209-b418-44de54331204 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:14.070373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848065fc-5bd9-40f5-ad3d-c96479042774 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:14.072873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:14.074520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:14.075065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:14.075191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:14.075319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:14.076021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:14.076394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:14.076790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:14.090530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:14.094966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:14.109371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:14.133554 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3801/7581] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:14.198772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:14.219326 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3781/7582] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:18:14 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:14.861135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:14.862709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:14.876586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef260f-9be7-4c59-a3cc-c4c5e0038886 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:14.881842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:14.882469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:14.882749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:14.883025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] PATH_INFO: `/v3/projects/84bb2a0b95cc44c8b7063269058a7c6c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:14.884141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84bb2a0b95cc44c8b7063269058a7c6c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:14.886106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:14.938534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84bb2a0b95cc44c8b7063269058a7c6c', 'request_id': 'req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:14.946169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84bb2a0b95cc44c8b7063269058a7c6c', 'request_id': 'req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:14.952918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84bb2a0b95cc44c8b7063269058a7c6c', 'request_id': 'req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:14.967106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] The token cache is being invalidate because project 84bb2a0b95cc44c8b7063269058a7c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:14.967407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:14.968052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f1ab8957-a2ac-40d5-bd42-40a93f0e0d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:14.969952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3802/7583] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:14 2022] DELETE /identity/v3/projects/84bb2a0b95cc44c8b7063269058a7c6c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:14.974264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:14.976267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:14.992550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa0e6e-32e8-4dc5-a17e-7feeadfe2380 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:15.066162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:15.066896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:15.067250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:15.067775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] PATH_INFO: `/v3/users/0fa8da0ae1d44de3867b823a13c15aed` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:15.069106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa8da0ae1d44de3867b823a13c15aed)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:15.071179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:15.132216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa8da0ae1d44de3867b823a13c15aed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.147895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa8da0ae1d44de3867b823a13c15aed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.153765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa8da0ae1d44de3867b823a13c15aed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.162493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa8da0ae1d44de3867b823a13c15aed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.163128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fd83cc0e-34a1-4e34-a239-f8d871b01a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa8da0ae1d44de3867b823a13c15aed'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.181452 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3782/7584] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:14 2022] DELETE /identity/v3/users/0fa8da0ae1d44de3867b823a13c15aed => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:15.226381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce991b3-ca62-49eb-b4e2-244f05d66f89 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:15.226806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce991b3-ca62-49eb-b4e2-244f05d66f89 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:15.227363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce991b3-ca62-49eb-b4e2-244f05d66f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:15.266464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cce991b3-ca62-49eb-b4e2-244f05d66f89 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:15.269201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cce991b3-ca62-49eb-b4e2-244f05d66f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:15.275895 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3803/7585] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:15 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:15.280605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:15.281139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:15.281598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:15.328245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:15.330152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:15.438530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:UPDATE=1,keystone:INSERT=3,keystone:DELETE=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:15.452100 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3783/7586] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:15.460973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce991b3-ca62-49eb-b4e2-244f05d66f89 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:15.479427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce991b3-ca62-49eb-b4e2-244f05d66f89 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:15.485207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd2b4-bce7-491b-b1c0-896528493729 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:15.485207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd2b4-bce7-491b-b1c0-896528493729 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:15.485612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd2b4-bce7-491b-b1c0-896528493729 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:15.486472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd2b4-bce7-491b-b1c0-896528493729 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:15.588427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:15.590300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:15.686873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-cdfbd2b4-bce7-491b-b1c0-896528493729 tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.717435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3804/7587] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:18:15 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:15.724615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a065787-d0ca-4bcb-8ad9-8ece141d0064 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:15.733087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:15.734123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:15.734625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:15.735144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] PATH_INFO: `/v3/projects/ce671e0785a54cc5b2862ed8cbd80034` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:15.736834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce671e0785a54cc5b2862ed8cbd80034)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:15.757356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:15.854699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce671e0785a54cc5b2862ed8cbd80034', 'request_id': 'req-f26cd13c-0a16-471a-8168-11e12053a249'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.871218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce671e0785a54cc5b2862ed8cbd80034', 'request_id': 'req-f26cd13c-0a16-471a-8168-11e12053a249'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.890986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce671e0785a54cc5b2862ed8cbd80034', 'request_id': 'req-f26cd13c-0a16-471a-8168-11e12053a249'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.909476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] The token cache is being invalidate because project ce671e0785a54cc5b2862ed8cbd80034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:15.909914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.910799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:15.912869 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3784/7588] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:15 2022] DELETE /identity/v3/projects/ce671e0785a54cc5b2862ed8cbd80034 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:15.919241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e502e5-88fd-4751-9405-4b884bba37cf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:15.919787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e502e5-88fd-4751-9405-4b884bba37cf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:15.919938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e502e5-88fd-4751-9405-4b884bba37cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:15.947054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-51e502e5-88fd-4751-9405-4b884bba37cf None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:15.949302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-51e502e5-88fd-4751-9405-4b884bba37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:16.014817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:16.049520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:16.055507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:16.095559 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3805/7589] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:16.107545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d5ed4-fd1b-4fad-8921-d3628e003a04 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.108320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d5ed4-fd1b-4fad-8921-d3628e003a04 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.108320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d5ed4-fd1b-4fad-8921-d3628e003a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.175274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-151d5ed4-fd1b-4fad-8921-d3628e003a04 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:16.179473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-151d5ed4-fd1b-4fad-8921-d3628e003a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:16.193408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cd13c-0a16-471a-8168-11e12053a249 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:16.198486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:16.199516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.200436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.200436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.203658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:16.220447 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3806/7590] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:16.224904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d5ed4-fd1b-4fad-8921-d3628e003a04 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:16.252947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d5ed4-fd1b-4fad-8921-d3628e003a04 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:16.268159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:16.271269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.272698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.273896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.274779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:16.275071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:16.279659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:16.299580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:16.310842 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3807/7591] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:16.315490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:16.336606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b58160-98b8-46fb-8bea-5c6b433ec257 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:16.354058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:16.356729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:16.374707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:16.375398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.375679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.375957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.383223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:16.385616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:16.393269 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3785/7592] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:16 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:16.470885 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3808/7593] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:16 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:16.475916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:16.493847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a67d01-96c6-441c-b163-2459e6265fb7 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:16.615454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc54230-c678-4406-b8c0-a2dfa96d6c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:16.618411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc54230-c678-4406-b8c0-a2dfa96d6c7f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.618892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc54230-c678-4406-b8c0-a2dfa96d6c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.619358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc54230-c678-4406-b8c0-a2dfa96d6c7f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.620352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc54230-c678-4406-b8c0-a2dfa96d6c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:16.622683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc54230-c678-4406-b8c0-a2dfa96d6c7f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:16.676464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3786/7594] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:16 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:16.678719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:16.695501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f81ef-08e3-4f3b-9a8c-17499691a860 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:16.708029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:16.708912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.709315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.710555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.711788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:16.716601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:16.727405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509764a4-9858-44a3-b875-597b26e08c69 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.728072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509764a4-9858-44a3-b875-597b26e08c69 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.729708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509764a4-9858-44a3-b875-597b26e08c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.746678 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3809/7595] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:16.747841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:16.765633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a033cb-643f-4e22-a174-72184f740e1d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:16.778562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412496ab-d409-4ad5-86f9-1b35ce4fe825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:16.779207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412496ab-d409-4ad5-86f9-1b35ce4fe825 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.779474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412496ab-d409-4ad5-86f9-1b35ce4fe825 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.779753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412496ab-d409-4ad5-86f9-1b35ce4fe825 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.780653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412496ab-d409-4ad5-86f9-1b35ce4fe825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:16.782298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412496ab-d409-4ad5-86f9-1b35ce4fe825 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:16.782891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-509764a4-9858-44a3-b875-597b26e08c69 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:16.785028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-509764a4-9858-44a3-b875-597b26e08c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:16.793622 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3787/7596] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:16 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:16.794515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3810/7597] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:16.800146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8375c-803a-4bfe-a1fa-0e80752e3f9c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.800462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8375c-803a-4bfe-a1fa-0e80752e3f9c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.800651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8375c-803a-4bfe-a1fa-0e80752e3f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.802761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509764a4-9858-44a3-b875-597b26e08c69 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:16.816632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509764a4-9858-44a3-b875-597b26e08c69 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:16.824695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2572cb-367c-467e-a309-dd126abc8c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:16.825492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2572cb-367c-467e-a309-dd126abc8c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.825898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2572cb-367c-467e-a309-dd126abc8c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.826498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2572cb-367c-467e-a309-dd126abc8c4a admin admin] PATH_INFO: `/v3/projects/f629e77bdabd4efc93a9f716871ed177/users/af6be669e99b41deb4d26d5aa7e428b4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.849203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2572cb-367c-467e-a309-dd126abc8c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f629e77bdabd4efc93a9f716871ed177, user_id=af6be669e99b41deb4d26d5aa7e428b4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:16.851439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dff8375c-803a-4bfe-a1fa-0e80752e3f9c None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:16.852602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dff8375c-803a-4bfe-a1fa-0e80752e3f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:16.854396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2572cb-367c-467e-a309-dd126abc8c4a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:16.884681 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3788/7598] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:16 2022] PUT /identity/v3/projects/f629e77bdabd4efc93a9f716871ed177/users/af6be669e99b41deb4d26d5aa7e428b4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:16.901416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3409-7ffe-4dd5-832b-c84dd4896ca3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:16.902143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3409-7ffe-4dd5-832b-c84dd4896ca3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:16.904130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b3409-7ffe-4dd5-832b-c84dd4896ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:16.986520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-047b3409-7ffe-4dd5-832b-c84dd4896ca3 None None] MFA Rules not processed for user `af6be669e99b41deb4d26d5aa7e428b4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:16.987769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-047b3409-7ffe-4dd5-832b-c84dd4896ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:17.009556 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3811/7599] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:17.016944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8375c-803a-4bfe-a1fa-0e80752e3f9c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:17.058138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8375c-803a-4bfe-a1fa-0e80752e3f9c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:17.136153 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3789/7600] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:17.140850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b3409-7ffe-4dd5-832b-c84dd4896ca3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:17.175245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0294e0-cd00-4807-beef-548bbcf71b3a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f96c6c82386e4cf28bfba40a124bf956', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3cfd38709d244f18a36b3e7d02f0fc6a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:17.176036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0294e0-cd00-4807-beef-548bbcf71b3a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.176528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0294e0-cd00-4807-beef-548bbcf71b3a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.178182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0294e0-cd00-4807-beef-548bbcf71b3a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956/password` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.190987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:17.260495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b3409-7ffe-4dd5-832b-c84dd4896ca3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:17.264289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:17.269244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:17.269244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.269244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.269244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.269763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:17.272128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f0294e0-cd00-4807-beef-548bbcf71b3a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:17.272608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:17.288797 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3812/7601] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:18:17 2022] POST /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956/password => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:17.314259 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3790/7602] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:17 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:17.315837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0294e0-cd00-4807-beef-548bbcf71b3a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:17.334076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0294e0-cd00-4807-beef-548bbcf71b3a tempest-IdentityV3UsersTest-1381068876 tempest-IdentityV3UsersTest-1381068876-project] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:17.418456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:17.419040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.419567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.419856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.420645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:17.424046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:17.462711 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3813/7603] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:17 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:17.467324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:17.486333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2c0e-b24b-4279-add7-e4efd26dc07f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:17.501489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:17.507584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.508834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.509251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.514670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:17.516375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:17.544116 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3791/7604] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:17.544318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:17.560685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e088-2cf2-41f5-80e1-6fd9ece24f46 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:17.583627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:17.585110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.585269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.585490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] PATH_INFO: `/v3/projects/3dd3c941735c456ab0328d6b8e6b84c4/users/22977fc3edeb4a9796395ad4495fef8d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.611280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd3c941735c456ab0328d6b8e6b84c4, user_id=22977fc3edeb4a9796395ad4495fef8d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:17.615690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:17.641299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:17.647543 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3814/7605] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:17 2022] PUT /identity/v3/projects/3dd3c941735c456ab0328d6b8e6b84c4/users/22977fc3edeb4a9796395ad4495fef8d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:17.654632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271207fa-72ae-4281-8e3c-317909ec8966 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:17.735297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ecf44-2c68-49e5-b6bd-24cb0c76ac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:17.737031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ecf44-2c68-49e5-b6bd-24cb0c76ac62 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.737892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ecf44-2c68-49e5-b6bd-24cb0c76ac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.737892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ecf44-2c68-49e5-b6bd-24cb0c76ac62 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.740559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ecf44-2c68-49e5-b6bd-24cb0c76ac62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:17.745931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ecf44-2c68-49e5-b6bd-24cb0c76ac62 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:17.782936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:17.797875 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3792/7606] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:17.807680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8100798-c843-4c5b-a278-12d47744b502 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:17.828426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:17.831013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.832157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.832489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] PATH_INFO: `/v3/projects/3dd3c941735c456ab0328d6b8e6b84c4/users/22977fc3edeb4a9796395ad4495fef8d/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.840590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd3c941735c456ab0328d6b8e6b84c4, user_id=22977fc3edeb4a9796395ad4495fef8d, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:17.846642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:17.886350 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3815/7607] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:17 2022] PUT /identity/v3/projects/3dd3c941735c456ab0328d6b8e6b84c4/users/22977fc3edeb4a9796395ad4495fef8d/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:17.902223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b85f-3166-4bca-93f5-65bde3fb5b95 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:17.902777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b85f-3166-4bca-93f5-65bde3fb5b95 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:17.903965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b85f-3166-4bca-93f5-65bde3fb5b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:17.952602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0110b85f-3166-4bca-93f5-65bde3fb5b95 None None] MFA Rules not processed for user `22977fc3edeb4a9796395ad4495fef8d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:17.954314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0110b85f-3166-4bca-93f5-65bde3fb5b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:18.172030 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3793/7608] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:18.190152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:18.192170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:18.294941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:18.295267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:18.295511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:18.335487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:18.340331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:18.350895 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3794/7609] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:18 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:18.361308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0dcea-c7d2-4539-9fcf-eec77c9982eb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:18.377566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0547ace-153f-44c2-9758-ca031225fdab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:18.380875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0547ace-153f-44c2-9758-ca031225fdab service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:18.381270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0547ace-153f-44c2-9758-ca031225fdab service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:18.381786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0547ace-153f-44c2-9758-ca031225fdab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:18.550279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0547ace-153f-44c2-9758-ca031225fdab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:18.552392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0547ace-153f-44c2-9758-ca031225fdab service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:18.580112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:18.591671 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3816/7610] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:19.327620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:19.331008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:19.356729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:19.357248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:19.357248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:19.400673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:19.405767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] MFA Rules not processed for user `f96c6c82386e4cf28bfba40a124bf956`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:19.407180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:19.496717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635e6a0-265b-408f-95dc-468628a236e6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:19.499723 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3817/7611] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:19.506702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:19.509251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:19.511754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:19.512422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:19.512707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:19.512986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] PATH_INFO: `/v3/users/08856c2088314fa6bcc7340f3752e438` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:19.518639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=08856c2088314fa6bcc7340f3752e438)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:19.522182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:19.531052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86fde4-03a8-4057-8303-b1fffbcd73fa None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:19.537558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:19.538623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:19.539118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:19.539602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] PATH_INFO: `/v3/users/f96c6c82386e4cf28bfba40a124bf956` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:19.542439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96c6c82386e4cf28bfba40a124bf956)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:19.544993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:19.623244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08856c2088314fa6bcc7340f3752e438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.642684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08856c2088314fa6bcc7340f3752e438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.643786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08856c2088314fa6bcc7340f3752e438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.655109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.683541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.700925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.710892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08856c2088314fa6bcc7340f3752e438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.712098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-852b5023-ae12-4ac3-ae29-08fd8c020eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08856c2088314fa6bcc7340f3752e438'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.722464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.726029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1cfcc1f3-e6c6-4fba-8b82-e95f62d58c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96c6c82386e4cf28bfba40a124bf956'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:19.749949 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3818/7612] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:19 2022] DELETE /identity/v3/users/f96c6c82386e4cf28bfba40a124bf956 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:19.753469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:19.753763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:19.754063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:19.761495 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3795/7613] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:19 2022] DELETE /identity/v3/users/08856c2088314fa6bcc7340f3752e438 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:19.763551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:19.763957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:19.764446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:19.801613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:19.804804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:19.820750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:19.824121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:20.000124 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3819/7614] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:20.007670 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3796/7615] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:20.030750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:20.034137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:20.036032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:20.040349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:20.066931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:20.168164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:20.212603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49fa6d-9a2c-40c7-9009-799df285df24 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:20.215134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61459e-4256-4ebd-a32b-34b6682f08ab None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:20.217990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:20.220444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:20.220444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:20.220444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:20.224894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:20.225537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:20.225854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:20.226132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:20.235354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:20.244519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:20.260073 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3820/7616] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:20.262238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:20.262498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:20.263657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:20.264071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:20.283870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3797/7617] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:20.287701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956340b-3839-4e2b-8ba1-5229c1e65ab4 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:20.291859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:20.292481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:20.292761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:20.293103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:20.295082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:20.400185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:20.404996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:20.425330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3821/7618] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:22.034700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:22.036586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:22.045920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:22.047663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:22.070141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5fa27-540c-4dd1-a08e-23b061df76be service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:22.082406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:22.083118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62344a-32b6-4a5e-b40a-992ead4a7536 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:22.083695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.084091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.084405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] PATH_INFO: `/v3/projects/7a3aa71f8465400895874771cb4d5ffe` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.086394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3aa71f8465400895874771cb4d5ffe)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:22.089484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:22.101030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:22.102240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.102746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.104961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] PATH_INFO: `/v3/projects/3cfd38709d244f18a36b3e7d02f0fc6a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.107385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cfd38709d244f18a36b3e7d02f0fc6a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:22.111443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:22.169197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3aa71f8465400895874771cb4d5ffe', 'request_id': 'req-d6d1062b-fb31-488e-a91c-ba359553d25d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.183883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3aa71f8465400895874771cb4d5ffe', 'request_id': 'req-d6d1062b-fb31-488e-a91c-ba359553d25d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.204549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3aa71f8465400895874771cb4d5ffe', 'request_id': 'req-d6d1062b-fb31-488e-a91c-ba359553d25d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.228677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] The token cache is being invalidate because project 7a3aa71f8465400895874771cb4d5ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:22.234264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.235253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.237922 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3798/7619] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:22 2022] DELETE /identity/v3/projects/7a3aa71f8465400895874771cb4d5ffe => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:22.238052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cfd38709d244f18a36b3e7d02f0fc6a', 'request_id': 'req-afb4d4ae-cbc9-4edc-a30e-cb974975c493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.244236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:22.255586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cfd38709d244f18a36b3e7d02f0fc6a', 'request_id': 'req-afb4d4ae-cbc9-4edc-a30e-cb974975c493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.255840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:22.270744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cfd38709d244f18a36b3e7d02f0fc6a', 'request_id': 'req-afb4d4ae-cbc9-4edc-a30e-cb974975c493'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.291455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] The token cache is being invalidate because project 3cfd38709d244f18a36b3e7d02f0fc6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:22.291750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.292412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-afb4d4ae-cbc9-4edc-a30e-cb974975c493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.293966 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3822/7620] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:22 2022] DELETE /identity/v3/projects/3cfd38709d244f18a36b3e7d02f0fc6a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:22.299554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d86ef-d9fd-4bbc-8608-4a581d68cbfe None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.300094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d86ef-d9fd-4bbc-8608-4a581d68cbfe None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.300530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d86ef-d9fd-4bbc-8608-4a581d68cbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.335871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c36d86ef-d9fd-4bbc-8608-4a581d68cbfe None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:22.337664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c36d86ef-d9fd-4bbc-8608-4a581d68cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:22.373370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1062b-fb31-488e-a91c-ba359553d25d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:22.387580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:22.448331 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3823/7621] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:22.458861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f5e7e-3f0c-4102-8595-e27570a7e1ed None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.466493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f5e7e-3f0c-4102-8595-e27570a7e1ed None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.467247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f5e7e-3f0c-4102-8595-e27570a7e1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.486172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:22.486987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.487372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.487774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] PATH_INFO: `/v3/users/f34d3cbddd6d4af39e0ba401dfab4026` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.489523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f34d3cbddd6d4af39e0ba401dfab4026)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:22.491828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:22.499675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bc8f5e7e-3f0c-4102-8595-e27570a7e1ed None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:22.502451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bc8f5e7e-3f0c-4102-8595-e27570a7e1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:22.511519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:22.515090 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3824/7622] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:22.521085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f5e7e-3f0c-4102-8595-e27570a7e1ed None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:22.543668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f5e7e-3f0c-4102-8595-e27570a7e1ed None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:22.558202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:22.558922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.559277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.559730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.560979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:22.561359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:22.563242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:22.575505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f34d3cbddd6d4af39e0ba401dfab4026'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.584116 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3825/7623] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:22.584753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f34d3cbddd6d4af39e0ba401dfab4026'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.585676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:22.586385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f34d3cbddd6d4af39e0ba401dfab4026'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.596994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa1eb-9f42-473b-9032-51a8de50333b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:22.603548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:22.604243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f34d3cbddd6d4af39e0ba401dfab4026'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.604574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.604958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.605289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f34d3cbddd6d4af39e0ba401dfab4026'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:22.605442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.606947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:22.610178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:22.629236 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3799/7624] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:22 2022] DELETE /identity/v3/users/f34d3cbddd6d4af39e0ba401dfab4026 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:22.648512 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3826/7625] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:22 2022] POST /identity/v3/projects => generated 397 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:22.652809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:22.654203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:22.751710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c4df3-e74d-45b3-9177-1cf45637132e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:22.842723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:22.856729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.857500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.858959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.859794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:22.868824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:22.935056 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3800/7626] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:22 2022] POST /identity/v3/users => generated 305 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:22.939672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:22.960739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09fac6-f325-4c24-861d-348270248f20 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:22.983064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:22.988460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:22.988799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:22.989692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:22.999023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.000846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:23.021451 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3827/7627] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:23.026161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:23.043549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a640bf7-a53a-49f5-91b8-5ed681396aa0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:23.074686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:23.076047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.076321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.076640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] PATH_INFO: `/v3/projects/4f288fea511243c1b9c8ccad31f6e1eb/users/bd21d1fefa414e7da035ee3fe3d2eda3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.136894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f288fea511243c1b9c8ccad31f6e1eb, user_id=bd21d1fefa414e7da035ee3fe3d2eda3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.141117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:23.177645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:23.179696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:23.190955 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3801/7628] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:23 2022] PUT /identity/v3/projects/4f288fea511243c1b9c8ccad31f6e1eb/users/bd21d1fefa414e7da035ee3fe3d2eda3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:23.199248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:23.228194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28d10-c86b-40a6-bff5-31e20c4ebc36 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:23.237284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:23.384506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:23.385461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.385898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.386596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] PATH_INFO: `/v3/projects/7065688e6b884b918526fe87f6491042` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.388220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7065688e6b884b918526fe87f6491042)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.390802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:23.399744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23302022-5c79-40f3-b9f8-d661d12061d0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:23.406332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:23.407109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.407417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.407705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.408513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.412638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:23.432205 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3802/7629] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:23.433278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:23.446447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a46ac6-4af0-496d-9588-5c3c827aa1ce admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:23.485864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7065688e6b884b918526fe87f6491042', 'request_id': 'req-19d7987d-3d2f-4e29-8929-4f52c73d637f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:23.498633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7065688e6b884b918526fe87f6491042', 'request_id': 'req-19d7987d-3d2f-4e29-8929-4f52c73d637f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:23.510941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7065688e6b884b918526fe87f6491042', 'request_id': 'req-19d7987d-3d2f-4e29-8929-4f52c73d637f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:23.539147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53004961-a12e-431a-9823-d0e0a04ac535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:23.539847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53004961-a12e-431a-9823-d0e0a04ac535 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.540136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53004961-a12e-431a-9823-d0e0a04ac535 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.540419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53004961-a12e-431a-9823-d0e0a04ac535 admin admin] PATH_INFO: `/v3/projects/4f288fea511243c1b9c8ccad31f6e1eb/users/bd21d1fefa414e7da035ee3fe3d2eda3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.542702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53004961-a12e-431a-9823-d0e0a04ac535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f288fea511243c1b9c8ccad31f6e1eb, user_id=bd21d1fefa414e7da035ee3fe3d2eda3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.552504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53004961-a12e-431a-9823-d0e0a04ac535 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:23.554660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] The token cache is being invalidate because project 7065688e6b884b918526fe87f6491042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:23.555382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:23.556159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-19d7987d-3d2f-4e29-8929-4f52c73d637f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:23.558105 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3828/7630] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:23 2022] DELETE /identity/v3/projects/7065688e6b884b918526fe87f6491042 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:23.566801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5791ce-0078-48b3-98bf-c92073891e0b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.567518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5791ce-0078-48b3-98bf-c92073891e0b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.568137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5791ce-0078-48b3-98bf-c92073891e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.579062 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3803/7631] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:23 2022] PUT /identity/v3/projects/4f288fea511243c1b9c8ccad31f6e1eb/users/bd21d1fefa414e7da035ee3fe3d2eda3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:23.583182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.583486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.583874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.618485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-be5791ce-0078-48b3-98bf-c92073891e0b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:23.621478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-be5791ce-0078-48b3-98bf-c92073891e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:23.639490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:23.651204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] MFA Rules not processed for user `bd21d1fefa414e7da035ee3fe3d2eda3`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:23.653070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:23.767064 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3804/7632] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:23.772639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:23.778181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:23.830894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:23.831979 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3829/7633] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:23.843364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae2f0-d26e-4802-b339-e11da23fdcf0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.843857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae2f0-d26e-4802-b339-e11da23fdcf0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.844220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae2f0-d26e-4802-b339-e11da23fdcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.881723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a80ae2f0-d26e-4802-b339-e11da23fdcf0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:23.883947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a80ae2f0-d26e-4802-b339-e11da23fdcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:23.897155 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3830/7634] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:23.901936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae2f0-d26e-4802-b339-e11da23fdcf0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:23.916216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae2f0-d26e-4802-b339-e11da23fdcf0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:23.922678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:23.923447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.923623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.924034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.924694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:23.925127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.927582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:23.942570 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3831/7635] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:23.944996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9f73d-517f-45c3-94ea-84f6c211ff7c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:23.947231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:23.951217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:23.952066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.953040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.953327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.960182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27253e-97b0-44ac-915a-6560167bfbc4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:23.965934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.966448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:23.972598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:23.973309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:23.973962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:23.974430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:23.975410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:23.978549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.010309 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3805/7636] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:23 2022] POST /identity/v3/projects => generated 399 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:24.015242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.022071 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3832/7637] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:23 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:24.026935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.184290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e6dff-8ebe-46b9-bffc-683cf7382f86 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.191952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735d712-4563-4239-933c-79c45e8c59c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.192783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735d712-4563-4239-933c-79c45e8c59c6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.193168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735d712-4563-4239-933c-79c45e8c59c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.193539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735d712-4563-4239-933c-79c45e8c59c6 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.194432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735d712-4563-4239-933c-79c45e8c59c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.197068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735d712-4563-4239-933c-79c45e8c59c6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.231656 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3833/7638] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:24 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:24.233624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e2c29-daaa-456a-8a8e-646cdd398159 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.234277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e2c29-daaa-456a-8a8e-646cdd398159 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.234580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e2c29-daaa-456a-8a8e-646cdd398159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.249898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a4dd5-05f5-4586-9508-61090b024639 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.267306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.267939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.268227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.268506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.269138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.271218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.272890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6e1e2c29-daaa-456a-8a8e-646cdd398159 None None] MFA Rules not processed for user `b4791d13ab1c45bf8fabec70191426da`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:24.274463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6e1e2c29-daaa-456a-8a8e-646cdd398159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:24.306129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3806/7639] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:24 2022] POST /identity/v3/users => generated 300 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:24.306129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.329904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677432-adc5-4c67-b067-6611519474dd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.343460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.344224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.347031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.347494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.348313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.353961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.367499 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3807/7640] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:24.373673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.387146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f38a20-e2db-4a41-b5a9-99a31aecdf49 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.392488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.393254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.393650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.393998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.398686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.398859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3834/7641] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:24.398859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e2c29-daaa-456a-8a8e-646cdd398159 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.399027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.412880 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3808/7642] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:24.415062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.426932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554d7a8-4ff9-4888-892c-93b1bf80a87e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.434963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e2c29-daaa-456a-8a8e-646cdd398159 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.444779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.445508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.445814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.446200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.446853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.448454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.452903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.455424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:24.456269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.456269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.456614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.459190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.463579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.475184 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3835/7643] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:24.479142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.494620 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3809/7644] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:24.496241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.509259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ffe47-e3d2-4ff3-adb4-940246be44fc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.509649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a234f4-8ab9-4edf-8a02-ba4ada74396f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.514833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c323457-8333-4a10-b45a-bcb7e435465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.515635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c323457-8333-4a10-b45a-bcb7e435465a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.515765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35caed5-1745-493b-bdd5-e451a5e87640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.516362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c323457-8333-4a10-b45a-bcb7e435465a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.516646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c323457-8333-4a10-b45a-bcb7e435465a admin admin] PATH_INFO: `/v3/projects/96101128801f4af7921dcda8afb80892/users/5fd3e3ad4f2b4623baf9099986d4a278/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.516886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35caed5-1745-493b-bdd5-e451a5e87640 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.517177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35caed5-1745-493b-bdd5-e451a5e87640 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.517455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35caed5-1745-493b-bdd5-e451a5e87640 admin admin] PATH_INFO: `/v3/projects/80e1afa417d543debd8650ebcd454602/users/cb2bbd98150442b2b338ae16eaa1fdc5/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.537523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35caed5-1745-493b-bdd5-e451a5e87640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e1afa417d543debd8650ebcd454602, user_id=cb2bbd98150442b2b338ae16eaa1fdc5, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.540566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35caed5-1745-493b-bdd5-e451a5e87640 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.545320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c323457-8333-4a10-b45a-bcb7e435465a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96101128801f4af7921dcda8afb80892, user_id=5fd3e3ad4f2b4623baf9099986d4a278, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.548615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c323457-8333-4a10-b45a-bcb7e435465a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.561638 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3810/7645] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:24 2022] PUT /identity/v3/projects/80e1afa417d543debd8650ebcd454602/users/cb2bbd98150442b2b338ae16eaa1fdc5/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:24.574579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.575162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.575601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.584779 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3836/7646] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:24 2022] PUT /identity/v3/projects/96101128801f4af7921dcda8afb80892/users/5fd3e3ad4f2b4623baf9099986d4a278/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:24.595739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09d88-5b35-4838-ab48-db287966417e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.596124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09d88-5b35-4838-ab48-db287966417e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.596493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f09d88-5b35-4838-ab48-db287966417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.610717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] MFA Rules not processed for user `cb2bbd98150442b2b338ae16eaa1fdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:24.612130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:24.650061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:24.659990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e9f09d88-5b35-4838-ab48-db287966417e None None] MFA Rules not processed for user `5fd3e3ad4f2b4623baf9099986d4a278`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:24.662658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e9f09d88-5b35-4838-ab48-db287966417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:24.719874 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3811/7647] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:24.733264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.735706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:24.852345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f200c64-8211-4eaf-9948-8d1f02e4a559 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:24.857961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:24.858697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:24.859038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:24.860012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:24.899674 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3837/7648] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:24.904612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f09d88-5b35-4838-ab48-db287966417e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:24.969368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:24.974811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:24.994946 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3812/7649] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:24 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:25.030953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f09d88-5b35-4838-ab48-db287966417e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:25.038309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:25.040275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.040764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.041278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.042892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:25.047316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:25.103738 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3838/7650] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:24 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:25.107503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:25.145198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553ebee-a2f3-4b19-af84-9317b47251c1 service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:25.251028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:25.253264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.253762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.254283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.255847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:25.258146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:25.289904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3813/7651] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:25 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:25.296059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:25.314338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955381f-e5a3-40d9-80d8-4bbee68c5d29 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:25.324620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:25.325255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.325681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.326414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.327085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:25.328693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:25.347950 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3839/7652] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:25.351560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:25.362049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:25.369059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e5911-df3a-44cc-a0aa-d90cc9eae788 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:25.378156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9f158-be47-48ec-aae2-bdc9542aedd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:25.380044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9f158-be47-48ec-aae2-bdc9542aedd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.380589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9f158-be47-48ec-aae2-bdc9542aedd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.381077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9f158-be47-48ec-aae2-bdc9542aedd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.382180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9f158-be47-48ec-aae2-bdc9542aedd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:25.384028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9f158-be47-48ec-aae2-bdc9542aedd8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:25.401624 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3814/7653] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:25.403169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:25.432520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28966eb0-a941-4b60-b4b4-a7a7745bd045 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:25.445199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:25.445917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.446211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.446500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] PATH_INFO: `/v3/projects/61d64804ec3a402c99f0c871e8c3709e/users/7ce9efba52f645c4ac845778c3b0ccbb/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.475110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d64804ec3a402c99f0c871e8c3709e, user_id=7ce9efba52f645c4ac845778c3b0ccbb, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:25.478214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:25.503744 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3840/7654] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:25 2022] PUT /identity/v3/projects/61d64804ec3a402c99f0c871e8c3709e/users/7ce9efba52f645c4ac845778c3b0ccbb/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:25.519425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6894354-8c6d-4f66-b829-b4fdb473d0b6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.519721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6894354-8c6d-4f66-b829-b4fdb473d0b6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.520003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6894354-8c6d-4f66-b829-b4fdb473d0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.579760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d6894354-8c6d-4f66-b829-b4fdb473d0b6 None None] MFA Rules not processed for user `7ce9efba52f645c4ac845778c3b0ccbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:25.581441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d6894354-8c6d-4f66-b829-b4fdb473d0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:25.701533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:25.719491 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3815/7655] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:25.728743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:25.732758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6894354-8c6d-4f66-b829-b4fdb473d0b6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:25.813652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34201a-fe49-4f12-8de7-40595dec8330 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:25.827211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:25.827590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.828147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.828443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] PATH_INFO: `/v3/users/bd21d1fefa414e7da035ee3fe3d2eda3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.830909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd21d1fefa414e7da035ee3fe3d2eda3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:25.835744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:25.843722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6894354-8c6d-4f66-b829-b4fdb473d0b6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:25.849373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:25.850186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:25.850476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:25.850756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:25.852572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:25.853766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:25.916907 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3816/7656] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:25 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:25.923211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:25.950859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd21d1fefa414e7da035ee3fe3d2eda3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:25.994075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd21d1fefa414e7da035ee3fe3d2eda3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:26.007273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd21d1fefa414e7da035ee3fe3d2eda3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:26.022636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd21d1fefa414e7da035ee3fe3d2eda3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:26.023274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-5ea57894-ac0f-4371-ba1d-e1cfb02d5517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd21d1fefa414e7da035ee3fe3d2eda3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:26.040397 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3841/7657] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:25 2022] DELETE /identity/v3/users/bd21d1fefa414e7da035ee3fe3d2eda3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:26.045710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.046207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.046492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.084879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:26.086582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:26.091200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f45a-a04b-4f19-817e-d88106a6ff4e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:26.117452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:26.209111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:26.209848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.210192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.210624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.211470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:26.213586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:26.221962 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3842/7658] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:26.254590 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3817/7659] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:25 2022] POST /identity/v3/users => generated 295 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:26.258213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:26.258352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:26.259985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:26.262646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:26.288245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0b59e-1f70-4ee8-ad32-26861afd6294 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:26.294138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:26.294772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.295058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.295330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.296042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:26.297656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:26.314151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3818/7660] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:26.316528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:26.320646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:26.334943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d07b1-4bc2-4d1a-af10-702007ffd150 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:26.352757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:26.353698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.354153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.354667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] PATH_INFO: `/v3/projects/aa5d87a95f8f4531a497c2edd3a42af9/users/9d96220905aa4cfda7e64b1daa6dd465/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.363848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10581f4-132e-4c2b-8e3b-103b1a8b071b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:26.369473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:26.371210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.371720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.376536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.399795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5d87a95f8f4531a497c2edd3a42af9, user_id=9d96220905aa4cfda7e64b1daa6dd465, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:26.404992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:26.413537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:26.415285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:26.438753 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3819/7661] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:26 2022] PUT /identity/v3/projects/aa5d87a95f8f4531a497c2edd3a42af9/users/9d96220905aa4cfda7e64b1daa6dd465/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:26.440186 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3843/7662] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:26.443321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:26.597239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bba054-dc57-4da5-a9d1-dead14b256fa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:26.604075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130be4a-0f94-4b64-acd4-4cfd487b946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:26.604718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130be4a-0f94-4b64-acd4-4cfd487b946b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.605036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130be4a-0f94-4b64-acd4-4cfd487b946b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.605253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130be4a-0f94-4b64-acd4-4cfd487b946b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.606561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130be4a-0f94-4b64-acd4-4cfd487b946b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:26.608285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130be4a-0f94-4b64-acd4-4cfd487b946b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:26.621114 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3820/7663] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:26.625255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:26.639218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2f298-c0db-4893-919a-614005999326 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:26.644940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:26.646082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.646805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.647237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] PATH_INFO: `/v3/projects/aa5d87a95f8f4531a497c2edd3a42af9/users/9d96220905aa4cfda7e64b1daa6dd465/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.649972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5d87a95f8f4531a497c2edd3a42af9, user_id=9d96220905aa4cfda7e64b1daa6dd465, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:26.653693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:26.675572 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3844/7664] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:26 2022] PUT /identity/v3/projects/aa5d87a95f8f4531a497c2edd3a42af9/users/9d96220905aa4cfda7e64b1daa6dd465/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:26.813246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:26.813533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:26.813925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:26.881477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] MFA Rules not processed for user `9d96220905aa4cfda7e64b1daa6dd465`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:26.882739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:27.031669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:27.031933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:27.033797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:27.125090 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3821/7665] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:27.137735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:27.139388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:27.197790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a93fa6-20d2-4423-b517-dc9ab191e98b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:27.205163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:27.206165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:27.206808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:27.207182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] PATH_INFO: `/v3/projects/4f288fea511243c1b9c8ccad31f6e1eb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:27.208637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f288fea511243c1b9c8ccad31f6e1eb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:27.211108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:27.261370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fc8f8-54b4-470d-b414-774d1719bf37 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:27.281021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f288fea511243c1b9c8ccad31f6e1eb', 'request_id': 'req-4156ca80-db70-4756-9e3a-9163bfa63ec7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.298656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f288fea511243c1b9c8ccad31f6e1eb', 'request_id': 'req-4156ca80-db70-4756-9e3a-9163bfa63ec7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.314422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f288fea511243c1b9c8ccad31f6e1eb', 'request_id': 'req-4156ca80-db70-4756-9e3a-9163bfa63ec7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.344270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] The token cache is being invalidate because project 4f288fea511243c1b9c8ccad31f6e1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:27.344963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.346114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.348177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3845/7666] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:27 2022] DELETE /identity/v3/projects/4f288fea511243c1b9c8ccad31f6e1eb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:27.350098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:27.352798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:27.354260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:27.354569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:27.355493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:27.355753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:27.362302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:27.486043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:27.489640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:27.501545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:27.506809 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3822/7667] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:27.524116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156ca80-db70-4756-9e3a-9163bfa63ec7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:27.531208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:27.532928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:27.533940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:27.534357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] PATH_INFO: `/v3/users/cb2bbd98150442b2b338ae16eaa1fdc5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:27.549848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2bbd98150442b2b338ae16eaa1fdc5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:27.549848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:27.662648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2bbd98150442b2b338ae16eaa1fdc5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.706396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2bbd98150442b2b338ae16eaa1fdc5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.738915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2bbd98150442b2b338ae16eaa1fdc5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.758393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2bbd98150442b2b338ae16eaa1fdc5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.760464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1a1a5790-c0fa-4ab2-80e7-ed1fac7c9e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2bbd98150442b2b338ae16eaa1fdc5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:27.769869 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3846/7668] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:27 2022] DELETE /identity/v3/users/cb2bbd98150442b2b338ae16eaa1fdc5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:28.344315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:28.375855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc933b-d581-4c5c-8d98-84e802a8eedb service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:28.503904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:28.504827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:28.505211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:28.505587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] PATH_INFO: `/v3/projects/80e1afa417d543debd8650ebcd454602` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:28.507018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e1afa417d543debd8650ebcd454602)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:28.509231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:28.514386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:28.699526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e1afa417d543debd8650ebcd454602', 'request_id': 'req-92e3288e-43a5-4952-897e-3d3241e6190c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:28.713706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e1afa417d543debd8650ebcd454602', 'request_id': 'req-92e3288e-43a5-4952-897e-3d3241e6190c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:28.729646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e1afa417d543debd8650ebcd454602', 'request_id': 'req-92e3288e-43a5-4952-897e-3d3241e6190c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:28.765173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] The token cache is being invalidate because project 80e1afa417d543debd8650ebcd454602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:28.765481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:28.766315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-92e3288e-43a5-4952-897e-3d3241e6190c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:28.770492 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3823/7669] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:28 2022] DELETE /identity/v3/projects/80e1afa417d543debd8650ebcd454602 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:28.776772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3878f-21fb-444c-a4b5-408d7334769b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:28.778069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3878f-21fb-444c-a4b5-408d7334769b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:28.778475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3878f-21fb-444c-a4b5-408d7334769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:28.842926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-24f3878f-21fb-444c-a4b5-408d7334769b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:28.845327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24f3878f-21fb-444c-a4b5-408d7334769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:29.042867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3847/7670] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:29.044702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be98a46-a98e-4ca4-9430-469a24cc6a44 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.045051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be98a46-a98e-4ca4-9430-469a24cc6a44 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.046393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be98a46-a98e-4ca4-9430-469a24cc6a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.092464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1be98a46-a98e-4ca4-9430-469a24cc6a44 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:29.093920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1be98a46-a98e-4ca4-9430-469a24cc6a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:29.103777 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3824/7671] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:29.108740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3878f-21fb-444c-a4b5-408d7334769b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:29.123248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3878f-21fb-444c-a4b5-408d7334769b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:29.129023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:29.130725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.131171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.131631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.132460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:29.132923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:29.134758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:29.157698 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3848/7672] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:29.163113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be98a46-a98e-4ca4-9430-469a24cc6a44 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:29.186783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be98a46-a98e-4ca4-9430-469a24cc6a44 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:29.202654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:29.203331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.203612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.204408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.204530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:29.206553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:29.263556 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3825/7673] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:29 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:29.264852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:29.282047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a620c0-b651-4791-ae0d-3d57224d61d7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:29.392000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:29.392917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.393211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.393492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.394173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:29.397132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:29.412689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:29.446135 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3849/7674] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:29 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:29.450266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:29.477457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f8046-4260-42fa-996b-85509e313871 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:29.482148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:29.482761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.483131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.483425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.484058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:29.486278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:29.505372 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3826/7675] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:29.509259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:29.531976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69069e-267f-4a46-93f7-b020d1c3b21c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:29.547416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:29.548182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.548565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.548963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.549792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:29.552223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:29.583129 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3850/7676] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:29.588698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:29.616465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2829-76b1-4260-81b2-a0f229558232 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:29.633629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:29.635610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:29.637976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d0f38-a1e2-4c8a-a6fb-f6022458d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:29.638918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d0f38-a1e2-4c8a-a6fb-f6022458d6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.639236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d0f38-a1e2-4c8a-a6fb-f6022458d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.639480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d0f38-a1e2-4c8a-a6fb-f6022458d6d6 admin admin] PATH_INFO: `/v3/projects/bc38caa2157f4734b72a9883ca56b781/users/85a893f59440418b9c9c34343e7672a0/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.672662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d0f38-a1e2-4c8a-a6fb-f6022458d6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc38caa2157f4734b72a9883ca56b781, user_id=85a893f59440418b9c9c34343e7672a0, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:29.676495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d0f38-a1e2-4c8a-a6fb-f6022458d6d6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:29.755737 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3827/7677] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:29 2022] PUT /identity/v3/projects/bc38caa2157f4734b72a9883ca56b781/users/85a893f59440418b9c9c34343e7672a0/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:29.767782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:29.768359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:29.768803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:29.831411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81e57c-29d1-44c7-9d8a-b4dd61b1a961 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:29.853079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] MFA Rules not processed for user `85a893f59440418b9c9c34343e7672a0`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:29.856253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:30.017071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:30.017531 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3828/7678] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:30.017683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.018156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.018455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:30.018571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.022955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:30.026506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:30.086654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:30.180068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:30.182509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:30.194806 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3851/7679] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:30.195749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:30.208172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517e7fd-c88a-40e7-a6fa-90065799e2c7 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:30.215655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:30.216844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.217323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.217791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.218689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:30.221015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:30.228712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254c07c-9867-4a31-93d3-b68198da8022 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:30.242606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:30.243610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.243953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.245109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] PATH_INFO: `/v3/users/d986a5c0854d4ba298481dd3c3b524c4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.249553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d986a5c0854d4ba298481dd3c3b524c4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:30.255620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:30.260054 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3852/7680] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:30 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:30.263959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:30.325360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:30.377073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd986a5c0854d4ba298481dd3c3b524c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:30.396698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd986a5c0854d4ba298481dd3c3b524c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:30.397473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd986a5c0854d4ba298481dd3c3b524c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:30.416033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd986a5c0854d4ba298481dd3c3b524c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:30.416927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7602caf8-948b-44d8-a085-a7f9bbef23b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd986a5c0854d4ba298481dd3c3b524c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:30.434635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5809d6-95ad-4b3f-8c6c-09e003b6b6a5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:30.450752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:30.452673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.453583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.455389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.457404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:30.463379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:30.468315 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3829/7681] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:30 2022] DELETE /identity/v3/users/d986a5c0854d4ba298481dd3c3b524c4 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:30.472971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.473226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.473626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.523775 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3853/7682] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:30 2022] POST /identity/v3/users => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:30.532553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:30.534780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:30.536327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:30.541406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:30.572188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51320856-00be-49c9-80a9-8f284baba8f9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:30.710047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:30.711690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.712154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.712574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.713378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:30.715356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:30.730973 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3854/7683] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:30.737815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:30.749217 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3830/7684] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:30.779015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d785-1d98-4eaf-b947-ead27db19c82 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:30.787980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:30.789004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.789579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.789996 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] PATH_INFO: `/v3/projects/62f9963a132a4c98ba279f4b444d3885/users/11f9a1b3571b4092a48cd1e5f5c20a06/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.802982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:30.805910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:30.829059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f9963a132a4c98ba279f4b444d3885, user_id=11f9a1b3571b4092a48cd1e5f5c20a06, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:30.838226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:30.872478 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3855/7685] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:30 2022] PUT /identity/v3/projects/62f9963a132a4c98ba279f4b444d3885/users/11f9a1b3571b4092a48cd1e5f5c20a06/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:30.873132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:30.948949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:30.969528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787301c5-3b42-4eca-8ccd-16615edd1259 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:30.976460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:30.977123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:30.977545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:30.978018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:30.978859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:30.981126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:31.000274 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3856/7686] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:31.001476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:31.015433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc957d-5164-442d-ba21-8937e0b67e64 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:31.018579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034f525-8eec-4c9b-bc99-dac75a0d378e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:31.027145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900b8a0-ed63-4f89-9b8f-feebdb82bcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:31.027449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b8a0-ed63-4f89-9b8f-feebdb82bcd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:31.027743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b8a0-ed63-4f89-9b8f-feebdb82bcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:31.028025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900b8a0-ed63-4f89-9b8f-feebdb82bcd8 admin admin] PATH_INFO: `/v3/projects/62f9963a132a4c98ba279f4b444d3885/users/11f9a1b3571b4092a48cd1e5f5c20a06/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:31.030898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b8a0-ed63-4f89-9b8f-feebdb82bcd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f9963a132a4c98ba279f4b444d3885, user_id=11f9a1b3571b4092a48cd1e5f5c20a06, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:31.034580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900b8a0-ed63-4f89-9b8f-feebdb82bcd8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:31.050003 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3857/7687] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:31 2022] PUT /identity/v3/projects/62f9963a132a4c98ba279f4b444d3885/users/11f9a1b3571b4092a48cd1e5f5c20a06/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:31.064151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:31.064495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:31.064807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:31.100575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:31.111700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] MFA Rules not processed for user `11f9a1b3571b4092a48cd1e5f5c20a06`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:31.112600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:31.151684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:31.152371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:31.152833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:31.153581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:31.166224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:31.167945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:31.183823 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3831/7688] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:31.236625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3858/7689] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:31.283254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:31.317464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1913-287e-490d-acfa-1826613aca15 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:31.349553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:31.351433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:31.440168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:31.441593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:31.443174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:31.444196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:31.448576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:31.463156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfc95c-1a56-4bb4-bab7-a7df5b29f2b0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:31.476737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee1dbf-c4e1-41ea-8538-91452d395e1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:31.478685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1dbf-c4e1-41ea-8538-91452d395e1e service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:31.479651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1dbf-c4e1-41ea-8538-91452d395e1e service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:31.480033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1dbf-c4e1-41ea-8538-91452d395e1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:31.484951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e4ee1dbf-c4e1-41ea-8538-91452d395e1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:31.594526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:31.597774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:31.613646 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3832/7690] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:31.625326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1dbf-c4e1-41ea-8538-91452d395e1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:31.627184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1dbf-c4e1-41ea-8538-91452d395e1e service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:31.655847 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3859/7691] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:32.427284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:32.429119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:32.561980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:32.589814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff46da-47d9-45e1-8ecb-b917ce152abc service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:32.593286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:32.594002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:32.594291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:32.594700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] PATH_INFO: `/v3/projects/dd7dc6344de94253979adb877c91e102` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:32.597419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7dc6344de94253979adb877c91e102)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:32.600597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:32.690784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7dc6344de94253979adb877c91e102', 'request_id': 'req-0c8d197b-448d-417c-a98a-8b9e61bddd74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:32.700816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7dc6344de94253979adb877c91e102', 'request_id': 'req-0c8d197b-448d-417c-a98a-8b9e61bddd74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:32.712075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7dc6344de94253979adb877c91e102', 'request_id': 'req-0c8d197b-448d-417c-a98a-8b9e61bddd74'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:32.728090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] The token cache is being invalidate because project dd7dc6344de94253979adb877c91e102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:32.728389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:32.729516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0c8d197b-448d-417c-a98a-8b9e61bddd74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:32.731828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3833/7692] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:32 2022] DELETE /identity/v3/projects/dd7dc6344de94253979adb877c91e102 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:32.738053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953853-0a26-49e6-9257-e205cf9853a4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:32.738570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953853-0a26-49e6-9257-e205cf9853a4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:32.738570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953853-0a26-49e6-9257-e205cf9853a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:32.768484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a0953853-0a26-49e6-9257-e205cf9853a4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:32.770724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a0953853-0a26-49e6-9257-e205cf9853a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:32.862042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:32.932178 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3860/7693] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:32.939246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d8ce-774c-45be-8344-3260ac89248f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:32.939532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d8ce-774c-45be-8344-3260ac89248f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:32.939842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d8ce-774c-45be-8344-3260ac89248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:32.993647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e342d8ce-774c-45be-8344-3260ac89248f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:33.009055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e342d8ce-774c-45be-8344-3260ac89248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:33.029933 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3834/7694] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:33.034743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0953853-0a26-49e6-9257-e205cf9853a4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:33.052242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0953853-0a26-49e6-9257-e205cf9853a4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:33.061394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:33.062132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:33.062411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:33.062693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:33.063369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:33.063677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:33.066598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:33.084439 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3861/7695] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:33.088482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342d8ce-774c-45be-8344-3260ac89248f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:33.104390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342d8ce-774c-45be-8344-3260ac89248f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:33.113358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:33.115080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:33.115584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:33.116102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:33.117453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:33.121478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:33.246260 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3835/7696] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:33 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:33.251240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:33.275041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb792a4-770f-4b4d-98b6-c4bc97ccc6e8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:33.483954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:33.484995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:33.485286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:33.486818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:33.488853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:33.496188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:33.542817 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3862/7697] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:33 2022] POST /identity/v3/users => generated 287 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:33.547305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:33.567759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c868dd5-e537-4e9a-8650-f3df5995b9ab admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:33.584037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:33.585279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:33.586480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:33.586994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:33.587831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:33.591078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:33.613194 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3836/7698] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:33.617673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:33.632661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017b5c-b6c0-4746-8000-d9ee15917199 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:33.651982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad934f6-3f5b-432c-b263-46fb9f040920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:33.652956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad934f6-3f5b-432c-b263-46fb9f040920 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:33.653348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad934f6-3f5b-432c-b263-46fb9f040920 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:33.653728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad934f6-3f5b-432c-b263-46fb9f040920 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:33.654524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad934f6-3f5b-432c-b263-46fb9f040920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:33.656336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad934f6-3f5b-432c-b263-46fb9f040920 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:33.676901 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3863/7699] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:33.680937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:33.698125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23ef97-763e-455c-a150-26dc922ea8aa admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:33.712944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:33.714555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:33.728610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:33.729014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] PATH_INFO: `/v3/projects/220429048fc9494ba4fe0b2e0c491cb8/users/51ebe0731bbb4d8cb87dea07defd5a8f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:33.758791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220429048fc9494ba4fe0b2e0c491cb8, user_id=51ebe0731bbb4d8cb87dea07defd5a8f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:33.761725 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:33.793515 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3837/7700] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:33 2022] PUT /identity/v3/projects/220429048fc9494ba4fe0b2e0c491cb8/users/51ebe0731bbb4d8cb87dea07defd5a8f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:33.795433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26ec14-d6cb-480a-b7a6-f8d3475287f2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:33.796233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26ec14-d6cb-480a-b7a6-f8d3475287f2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:33.796749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26ec14-d6cb-480a-b7a6-f8d3475287f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:33.853197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2a26ec14-d6cb-480a-b7a6-f8d3475287f2 None None] MFA Rules not processed for user `51ebe0731bbb4d8cb87dea07defd5a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:33.854749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2a26ec14-d6cb-480a-b7a6-f8d3475287f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:33.965904 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3864/7701] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:33.968160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:34.053511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b938e-4a3d-40dc-9a50-b69f6f2d4f02 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:34.060582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:34.061469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:34.062059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:34.062573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:34.063431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:34.066602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:34.090910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:34.116404 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3838/7702] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:33 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:34.120442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26ec14-d6cb-480a-b7a6-f8d3475287f2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:34.132903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26ec14-d6cb-480a-b7a6-f8d3475287f2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:34.195706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:34.231038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:34.232004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:34.232488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:34.233385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:34.234813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:34.237690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:34.290755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:34.299955 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3865/7703] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:34 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:34.318015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc916c1-862a-4ee4-a32e-b8cdb6191879 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:34.327270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:34.328362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:34.328362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:34.328362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:34.330393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:34.332390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:34.353769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:34.370636 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3839/7704] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:34.383355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72938724-d1fc-4885-8d46-e34db541d4f6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:34.403496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:34.403496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:34.403496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:34.403496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] PATH_INFO: `/v3/projects/e1484c7739ce48f6b5f8eaac8f659995/users/e01196a5285d43448fb5c4224429a37c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:34.426868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1484c7739ce48f6b5f8eaac8f659995, user_id=e01196a5285d43448fb5c4224429a37c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:34.430760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:34.450577 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3866/7705] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:34 2022] PUT /identity/v3/projects/e1484c7739ce48f6b5f8eaac8f659995/users/e01196a5285d43448fb5c4224429a37c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:34.455414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:34.479564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cd7d3-8858-469c-84be-f72cdefcd5e9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:34.621702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b4820-1bf2-43d2-b911-a73efd99282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:34.623139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b4820-1bf2-43d2-b911-a73efd99282d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:34.623872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b4820-1bf2-43d2-b911-a73efd99282d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:34.625078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b4820-1bf2-43d2-b911-a73efd99282d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:34.628522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b4820-1bf2-43d2-b911-a73efd99282d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:34.631044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b4820-1bf2-43d2-b911-a73efd99282d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:34.654293 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3840/7706] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:34.657105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:34.697353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33685-7991-40ed-a7b1-ca8785d9487f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:34.704148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:34.705237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:34.705762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:34.706227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] PATH_INFO: `/v3/projects/e1484c7739ce48f6b5f8eaac8f659995/users/e01196a5285d43448fb5c4224429a37c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:34.711116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1484c7739ce48f6b5f8eaac8f659995, user_id=e01196a5285d43448fb5c4224429a37c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:34.714700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:34.754262 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3867/7707] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:34 2022] PUT /identity/v3/projects/e1484c7739ce48f6b5f8eaac8f659995/users/e01196a5285d43448fb5c4224429a37c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:34.757381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4b40-36a2-4357-adcf-903a2741cfc7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:34.757797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4b40-36a2-4357-adcf-903a2741cfc7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:34.758331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a4b40-36a2-4357-adcf-903a2741cfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:34.831738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5f9a4b40-36a2-4357-adcf-903a2741cfc7 None None] MFA Rules not processed for user `e01196a5285d43448fb5c4224429a37c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:34.835052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5f9a4b40-36a2-4357-adcf-903a2741cfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:34.969008 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3841/7708] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:34.974584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:34.997462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab09e-9bbe-457c-8fda-f450be5502c8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.025661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.026326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.026614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.026891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.027551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.029767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.068038 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3868/7709] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:34 2022] POST /identity/v3/services => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:35.074875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4b40-36a2-4357-adcf-903a2741cfc7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.095164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a4b40-36a2-4357-adcf-903a2741cfc7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.157337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.157807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.158125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.158399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.159085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.165486 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:35.165486 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:18:35.165486 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:18:35.168162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.198571 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3842/7710] 10.222.0.33 () {60 vars in 1213 bytes} [Sat Sep 10 21:18:35 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:35.198776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.218054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5aa3f-43e4-459d-abd2-2c3fb924514b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.229313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.231800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.233256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.235322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.236223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.247437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.280283 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3869/7711] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:35 2022] POST /identity/v3/services => generated 289 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:35.285897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.299190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102a8ed-277b-4294-9d84-fdd20f7c3999 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.352371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.353003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.353656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.353962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.354634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.356254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.386674 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3843/7712] 10.222.0.33 () {60 vars in 1213 bytes} [Sat Sep 10 21:18:35 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 334 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:35.387910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.407865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1351fc2-8218-4693-a50d-3781d5ab256a tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.422230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.422849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.423139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.423416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.424759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.425679 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:35.425679 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:35.425679 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:35.428709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.467548 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3870/7713] 10.222.0.33 () {58 vars in 1191 bytes} [Sat Sep 10 21:18:35 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 765 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:35.470286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.490712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28bfcd-bad5-4d70-87f7-89ccde504e46 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.503674 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.505204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.506096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.506345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.508502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=dd817985fc384ddaa6744aaa3a63206d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.510484 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:35.510484 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:18:35.510484 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:18:35.514646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.536787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.538766 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3844/7714] 10.222.0.33 () {58 vars in 1290 bytes} [Sat Sep 10 21:18:35 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d => generated 334 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:35.551763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db50fb-4afc-4a14-908b-9f0c545abbdc tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.572746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.573411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `HEAD` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.573700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.574010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.574610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=dd817985fc384ddaa6744aaa3a63206d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.575630 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:35.575630 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:35.575630 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:35.576521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.587448 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3871/7715] 10.222.0.33 () {58 vars in 1291 bytes} [Sat Sep 10 21:18:35 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d => generated 0 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:35.592199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.610071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1180a5-9d27-4fdd-b3c2-5dd7e235c518 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.626053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.626931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.627658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.628123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.628926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=dd817985fc384ddaa6744aaa3a63206d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.632207 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:35.632207 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:18:35.632207 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:18:35.633332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.676058 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3845/7716] 10.222.0.33 () {58 vars in 1293 bytes} [Sat Sep 10 21:18:35 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:35.681433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.699892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cd84c-bcc1-40a9-87e0-908b2e639c89 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.713303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.713952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.714242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.714522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.715179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.717913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.724066 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3872/7717] 10.222.0.33 () {58 vars in 1191 bytes} [Sat Sep 10 21:18:35 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:35.729004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.765705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70185d97-9b49-44fb-81c0-6a2e02805d74 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.783548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.786340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.787188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.787737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.788627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=dd817985fc384ddaa6744aaa3a63206d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.790549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.793820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=95,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Could not find Endpoint Group: dd817985fc384ddaa6744aaa3a63206d.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: dd817985fc384ddaa6744aaa3a63206d. Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Sep 10 21:18:35.841569 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Sep 10 21:18:35.843195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Sep 10 21:18:35.843195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Sep 10 21:18:35.843195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Sep 10 21:18:35.843195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Sep 10 21:18:35.843195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: dd817985fc384ddaa6744aaa3a63206d. Sep 10 21:18:35.843195 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:35.846015 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3846/7718] 10.222.0.33 () {58 vars in 1293 bytes} [Sat Sep 10 21:18:35 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/dd817985fc384ddaa6744aaa3a63206d => generated 120 bytes in 118 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:18:35.850533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.873132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75ef28-8ad2-46ea-9d8e-65cbe07e20d4 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:35.889976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:35.892090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:35.893699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:35.894160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services/97542304d68f40ccb4399999b4708f6c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:35.897190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_service(service_id=97542304d68f40ccb4399999b4708f6c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:35.901151 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:35.901151 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:35.901151 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:35.904951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:35.950514 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3873/7719] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:35 2022] DELETE /identity/v3/services/97542304d68f40ccb4399999b4708f6c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:35.955901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:35.974952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985abc3-37d9-4df8-8040-a9bef36290fb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.034910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.036333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.036982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.037520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.038500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.040462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.068172 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3847/7720] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:35 2022] POST /identity/v3/services => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:36.072299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.098367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eb6dc-c554-47ba-854c-d85641f7f742 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.154338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.155894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.156366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.156845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.157739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.160139 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:36.160139 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:36.160139 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:36.164408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.190895 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3874/7721] 10.222.0.33 () {60 vars in 1213 bytes} [Sat Sep 10 21:18:36 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 332 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:36.192891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.211772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd6531-6dca-40b9-afad-1e550ee2210e tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.218029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.219503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.219806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.220102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.221090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.224462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.251067 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3848/7722] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:36 2022] POST /identity/v3/services => generated 290 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:36.256228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.298860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228c891-381a-432e-9916-b66541f6d7ff tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.342264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.343356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.343850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.344330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/f0637ff1b67041da93a287fde9a6ef2c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.345212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=f0637ff1b67041da93a287fde9a6ef2c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.346522 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:36.346522 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:36.346522 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:36.350709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.369181 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3875/7723] 10.222.0.33 () {60 vars in 1313 bytes} [Sat Sep 10 21:18:36 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/f0637ff1b67041da93a287fde9a6ef2c => generated 334 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:36.374869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.403252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a10829-bdb8-41ad-97d7-fec0a0d534c6 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.411317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.412464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.412755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.413036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services/db5b9ecbcdf74336be19157f425e6365` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.413628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_service(service_id=db5b9ecbcdf74336be19157f425e6365)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.416242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.451292 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3849/7724] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:36 2022] DELETE /identity/v3/services/db5b9ecbcdf74336be19157f425e6365 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:36.456333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.475310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a726c-e83e-4122-b584-4a397551e086 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.476292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.490284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:36.498648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.499299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.499581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.499856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/f0637ff1b67041da93a287fde9a6ef2c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.500492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=f0637ff1b67041da93a287fde9a6ef2c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.501574 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:36.501574 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:36.501574 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:36.502985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.522042 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3876/7725] 10.222.0.33 () {58 vars in 1293 bytes} [Sat Sep 10 21:18:36 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/f0637ff1b67041da93a287fde9a6ef2c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:36.526312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.549820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b7670-a98e-449c-ac64-7c53dddbf119 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.557266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.559276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.559758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.560556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services/ef56dac05b07460587b9030cac6269b3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.562129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_service(service_id=ef56dac05b07460587b9030cac6269b3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.565667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.596363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:INSERT=8,keystone:DELETE=5,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:36.605320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5157452-0c05-4649-bcc3-a2d0acb5c64b tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.611657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.612436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.612879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.613256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] PATH_INFO: `/v3/users/5fd3e3ad4f2b4623baf9099986d4a278` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.615472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd3e3ad4f2b4623baf9099986d4a278)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.617573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.626510 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3877/7726] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:36 2022] DELETE /identity/v3/services/ef56dac05b07460587b9030cac6269b3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:36.633483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.666800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616144d-40cf-415b-a7a8-b504feebc5ee tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.676742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.677718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.678471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.679555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/7c5ec4fda11947b3897b7d16dc95ab50` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.679555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=7c5ec4fda11947b3897b7d16dc95ab50)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.681492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.714704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd3e3ad4f2b4623baf9099986d4a278'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:36.719841 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3878/7727] 10.222.0.33 () {58 vars in 1293 bytes} [Sat Sep 10 21:18:36 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/7c5ec4fda11947b3897b7d16dc95ab50 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:36.727193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.737333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd3e3ad4f2b4623baf9099986d4a278'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:36.738589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd3e3ad4f2b4623baf9099986d4a278'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:36.750426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd3e3ad4f2b4623baf9099986d4a278'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:36.751538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-91ee3232-3e25-4d83-86db-79489a290795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd3e3ad4f2b4623baf9099986d4a278'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:36.751727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4019d5-b739-41f6-aad9-56ea44001d52 tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.764900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e01196a5285d43448fb5c4224429a37c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1484c7739ce48f6b5f8eaac8f659995', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:36.765554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.765867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.766141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] PATH_INFO: `/v3/services/c585883c66ed46b0b18c6a8111910260` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.766727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorizing `identity:delete_service(service_id=c585883c66ed46b0b18c6a8111910260)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:36.768333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:36.806037 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3850/7728] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:36 2022] DELETE /identity/v3/users/5fd3e3ad4f2b4623baf9099986d4a278 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:36.810820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:36.811176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:36.811385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:36.844631 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3879/7729] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:36 2022] DELETE /identity/v3/services/c585883c66ed46b0b18c6a8111910260 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:36.847071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:36.857090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:36.899979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:36.903097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:36.911504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63899fc-8631-4fe4-a2b9-7a60b8522dcb tempest-EndPointGroupsTest-1213630045 tempest-EndPointGroupsTest-1213630045-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:36.915768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:37.072011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:37.073306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:37.076294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:37.076912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] PATH_INFO: `/v3/users/51ebe0731bbb4d8cb87dea07defd5a8f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:37.078834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ebe0731bbb4d8cb87dea07defd5a8f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:37.082031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:37.137958 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3851/7730] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:37.137958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:37.142279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:37.197509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ebe0731bbb4d8cb87dea07defd5a8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:37.225254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ebe0731bbb4d8cb87dea07defd5a8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:37.243345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ebe0731bbb4d8cb87dea07defd5a8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:37.259526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ebe0731bbb4d8cb87dea07defd5a8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:37.260318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ebe0731bbb4d8cb87dea07defd5a8f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:37.270533 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3880/7731] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:36 2022] DELETE /identity/v3/users/51ebe0731bbb4d8cb87dea07defd5a8f => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:37.271542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:37.274493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:37.313671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b028507-3d10-45fe-a060-b53c3030da56 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:37.328563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:37.428440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:37.429605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:37.430135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:37.430691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:37.432709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:37.437860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4091d77-4a1c-4da5-97ee-b875d89ff14c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:37.441592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178a1f8-290d-4f6d-9980-701315edad99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:37.442870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a1f8-290d-4f6d-9980-701315edad99 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:37.443516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a1f8-290d-4f6d-9980-701315edad99 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:37.443796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178a1f8-290d-4f6d-9980-701315edad99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:37.446284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1178a1f8-290d-4f6d-9980-701315edad99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:37.529099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178a1f8-290d-4f6d-9980-701315edad99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:37.530844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178a1f8-290d-4f6d-9980-701315edad99 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:37.542959 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3881/7732] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:37.544414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:37.544806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:37.545182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:37.555016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:37.558072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:37.572927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:37.575648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:37.579800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:37.588818 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3852/7733] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:37.593882 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3882/7734] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:37.626503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:37.663069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d2c6f-4df4-4a4d-a5ac-a3594e6e6d9c service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:37.669657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:37.670978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:37.671607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:37.672523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:37.693886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:37.696097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:37.710303 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3853/7735] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:38.899476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:38.902310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:38.921661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208e7ee-400d-44e7-901a-8305c8977e1c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:38.936122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:38.936904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:38.937077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:38.939714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] PATH_INFO: `/v3/projects/96101128801f4af7921dcda8afb80892` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:38.939974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=96101128801f4af7921dcda8afb80892)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:38.942958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:39.050947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:39.053893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:39.076902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d7154-b8e0-4f3f-b6fa-d2dce395e941 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:39.089854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96101128801f4af7921dcda8afb80892', 'request_id': 'req-d0043469-9e31-4967-abd4-82d3b6de4aca'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.105521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96101128801f4af7921dcda8afb80892', 'request_id': 'req-d0043469-9e31-4967-abd4-82d3b6de4aca'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.120813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96101128801f4af7921dcda8afb80892', 'request_id': 'req-d0043469-9e31-4967-abd4-82d3b6de4aca'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.141388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] The token cache is being invalidate because project 96101128801f4af7921dcda8afb80892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:39.141714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.147722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.154154 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3883/7736] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:38 2022] DELETE /identity/v3/projects/96101128801f4af7921dcda8afb80892 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:39.155014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:39.157543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:39.203208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:39.203963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:39.204182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:39.204481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] PATH_INFO: `/v3/projects/220429048fc9494ba4fe0b2e0c491cb8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:39.205580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=220429048fc9494ba4fe0b2e0c491cb8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:39.208774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:39.228811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:39.276197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0043469-9e31-4967-abd4-82d3b6de4aca admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:39.305255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220429048fc9494ba4fe0b2e0c491cb8', 'request_id': 'req-614d5f58-1cca-4277-8c21-7f0525bd0aa4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.317810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220429048fc9494ba4fe0b2e0c491cb8', 'request_id': 'req-614d5f58-1cca-4277-8c21-7f0525bd0aa4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.338144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220429048fc9494ba4fe0b2e0c491cb8', 'request_id': 'req-614d5f58-1cca-4277-8c21-7f0525bd0aa4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.355156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] The token cache is being invalidate because project 220429048fc9494ba4fe0b2e0c491cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:39.355457 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.358025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.359960 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3854/7737] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:39 2022] DELETE /identity/v3/projects/220429048fc9494ba4fe0b2e0c491cb8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:39.363730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:39.365386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:39.406648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:39.407956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:39.408913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:39.409460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] PATH_INFO: `/v3/users/7ce9efba52f645c4ac845778c3b0ccbb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:39.411886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce9efba52f645c4ac845778c3b0ccbb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:39.415496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:39.480116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5f58-1cca-4277-8c21-7f0525bd0aa4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:39.492616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:39.493602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:39.494153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:39.494647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] PATH_INFO: `/v3/users/e01196a5285d43448fb5c4224429a37c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:39.496530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e01196a5285d43448fb5c4224429a37c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:39.499372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:39.511095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce9efba52f645c4ac845778c3b0ccbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.523551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:39.533566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce9efba52f645c4ac845778c3b0ccbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.550830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce9efba52f645c4ac845778c3b0ccbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.570255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce9efba52f645c4ac845778c3b0ccbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.571755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce9efba52f645c4ac845778c3b0ccbb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.573856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e01196a5285d43448fb5c4224429a37c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.585300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e01196a5285d43448fb5c4224429a37c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.586633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e01196a5285d43448fb5c4224429a37c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.587835 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3884/7738] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:39 2022] DELETE /identity/v3/users/7ce9efba52f645c4ac845778c3b0ccbb => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:39.600806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e01196a5285d43448fb5c4224429a37c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.602835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c606391-0dc1-447b-8e4b-b0029ae8aec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e01196a5285d43448fb5c4224429a37c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:39.636341 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3855/7739] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:39 2022] DELETE /identity/v3/users/e01196a5285d43448fb5c4224429a37c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:39.930360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:39.931968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:40.084239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592c592-cac4-4963-91de-20a245854eae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:40.113379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:40.203064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:40.204843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:40.205313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:40.205589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] PATH_INFO: `/v3/users/af6be669e99b41deb4d26d5aa7e428b4` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:40.208248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6be669e99b41deb4d26d5aa7e428b4)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:40.210844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:40.314713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6be669e99b41deb4d26d5aa7e428b4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.361419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6be669e99b41deb4d26d5aa7e428b4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.372652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6be669e99b41deb4d26d5aa7e428b4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.390119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6be669e99b41deb4d26d5aa7e428b4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.401530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6be669e99b41deb4d26d5aa7e428b4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.433190 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3885/7740] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:39 2022] DELETE /identity/v3/users/af6be669e99b41deb4d26d5aa7e428b4 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:40.435145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:40.435612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:40.435840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:40.471317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:40.472730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:40.539877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:40.541240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:40.555746 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3856/7741] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:40.558157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0e9f-20c3-45c1-b3ae-c212cdac6a41 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:40.564190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:40.565188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:40.565566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:40.566099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] PATH_INFO: `/v3/projects/61d64804ec3a402c99f0c871e8c3709e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:40.568017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d64804ec3a402c99f0c871e8c3709e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:40.570154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:40.578132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:40.580162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:40.631003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d64804ec3a402c99f0c871e8c3709e', 'request_id': 'req-e687764e-3db3-4748-894a-fff530859e69'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.638332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d64804ec3a402c99f0c871e8c3709e', 'request_id': 'req-e687764e-3db3-4748-894a-fff530859e69'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.648071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d64804ec3a402c99f0c871e8c3709e', 'request_id': 'req-e687764e-3db3-4748-894a-fff530859e69'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.657703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb1068-2e72-425e-a7af-9966e145b5a1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:40.666942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] The token cache is being invalidate because project 61d64804ec3a402c99f0c871e8c3709e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:40.667293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.667977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.672456 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3886/7742] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:40 2022] DELETE /identity/v3/projects/61d64804ec3a402c99f0c871e8c3709e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:40.679690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:40.682100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:40.708756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:40.728591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:40.729298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:40.729681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:40.730049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:40.732582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:40.740015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:40.762898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687764e-3db3-4748-894a-fff530859e69 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:40.767867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:40.768610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:40.768998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:40.769329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] PATH_INFO: `/v3/users/9d96220905aa4cfda7e64b1daa6dd465` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:40.771299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d96220905aa4cfda7e64b1daa6dd465)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:40.773319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:40.826342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d96220905aa4cfda7e64b1daa6dd465'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.826757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:40.828400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:40.840675 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3857/7743] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:40.841480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:40.843120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:40.854758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d96220905aa4cfda7e64b1daa6dd465'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.855516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c525be6-67cb-4a48-bb51-8c1f3b0274fe service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:40.860572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:40.861235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:40.861574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:40.861755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] PATH_INFO: `/v3/projects/e1484c7739ce48f6b5f8eaac8f659995` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:40.863102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1484c7739ce48f6b5f8eaac8f659995)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:40.865058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:40.872947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d96220905aa4cfda7e64b1daa6dd465'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.900168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d96220905aa4cfda7e64b1daa6dd465'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.900891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2dfbec87-8c92-4876-923d-81bd56a634b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d96220905aa4cfda7e64b1daa6dd465'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.914949 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3887/7744] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:40 2022] DELETE /identity/v3/users/9d96220905aa4cfda7e64b1daa6dd465 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:40.949150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1484c7739ce48f6b5f8eaac8f659995', 'request_id': 'req-5d8789c1-6ab2-4572-a269-4419f201736c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.957434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1484c7739ce48f6b5f8eaac8f659995', 'request_id': 'req-5d8789c1-6ab2-4572-a269-4419f201736c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:40.989436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1484c7739ce48f6b5f8eaac8f659995', 'request_id': 'req-5d8789c1-6ab2-4572-a269-4419f201736c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.019093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] The token cache is being invalidate because project e1484c7739ce48f6b5f8eaac8f659995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:41.019724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.021820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.024783 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3858/7745] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:40 2022] DELETE /identity/v3/projects/e1484c7739ce48f6b5f8eaac8f659995 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:41.030000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fca47e-d810-485e-9590-2fd952e03e08 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.030518 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fca47e-d810-485e-9590-2fd952e03e08 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.031050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fca47e-d810-485e-9590-2fd952e03e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.074903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-69fca47e-d810-485e-9590-2fd952e03e08 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:41.084245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-69fca47e-d810-485e-9590-2fd952e03e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:41.241374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:41.243783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:41.257042 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3888/7746] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:41.266354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb0a1f-9edd-4580-9ab9-056c3a68d67c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.266645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb0a1f-9edd-4580-9ab9-056c3a68d67c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.266922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb0a1f-9edd-4580-9ab9-056c3a68d67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.269548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8789c1-6ab2-4572-a269-4419f201736c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:41.292903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:41.293751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-53fb0a1f-9edd-4580-9ab9-056c3a68d67c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:41.294205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.294818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.295809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-53fb0a1f-9edd-4580-9ab9-056c3a68d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:41.297219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] PATH_INFO: `/v3/projects/f629e77bdabd4efc93a9f716871ed177` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.299947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f629e77bdabd4efc93a9f716871ed177)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:41.303756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:41.308559 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3889/7747] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:41.313448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb0a1f-9edd-4580-9ab9-056c3a68d67c None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:41.325134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb0a1f-9edd-4580-9ab9-056c3a68d67c None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:41.330241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:41.330985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.331364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.331735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.333119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:41.333993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:41.335760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:41.351604 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3890/7748] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:41.355915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:41.373103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906fd3-2159-4edd-9441-3bee0af2ceb0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:41.408316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:41.415377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f629e77bdabd4efc93a9f716871ed177', 'request_id': 'req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.449650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f629e77bdabd4efc93a9f716871ed177', 'request_id': 'req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.462271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f629e77bdabd4efc93a9f716871ed177', 'request_id': 'req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.482305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] The token cache is being invalidate because project f629e77bdabd4efc93a9f716871ed177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:41.482921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.484024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.487116 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3859/7749] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:41 2022] DELETE /identity/v3/projects/f629e77bdabd4efc93a9f716871ed177 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:41.493157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:41.496090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:41.525673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:41.535448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:41.536423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.536915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.537261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.538001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:41.540004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:41.579640 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3891/7750] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:41 2022] POST /identity/v3/projects => generated 369 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:41.580804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:41.584457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:41.622750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed188514-cfcc-4e9f-9b52-a1ee4935ab79 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:41.690911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef725-4e39-42fd-9bdb-8f4782f57461 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:41.703000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:41.703923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.705293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.706443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.709427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:41.711523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:41.729922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:41.730677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.731056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.731685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] PATH_INFO: `/v3/users/22977fc3edeb4a9796395ad4495fef8d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.734200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=22977fc3edeb4a9796395ad4495fef8d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:41.736230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:41.761042 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3892/7751] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:41 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:41.762048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:41.764452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:41.791753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ce1a9-066a-4604-82ee-894ddd63d4cc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:41.803920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:41.805262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:41.805708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:41.806159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] PATH_INFO: `/v3/projects/aa5d87a95f8f4531a497c2edd3a42af9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:41.808385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5d87a95f8f4531a497c2edd3a42af9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:41.816216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:41.843275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22977fc3edeb4a9796395ad4495fef8d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.863397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22977fc3edeb4a9796395ad4495fef8d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.865321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22977fc3edeb4a9796395ad4495fef8d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.883017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22977fc3edeb4a9796395ad4495fef8d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.884050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22977fc3edeb4a9796395ad4495fef8d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.930402 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3860/7752] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:41 2022] DELETE /identity/v3/users/22977fc3edeb4a9796395ad4495fef8d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:41.931477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:41.934805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:41.937429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5d87a95f8f4531a497c2edd3a42af9', 'request_id': 'req-ab1bf131-0322-477d-90fe-ce3946559aa1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.950636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5d87a95f8f4531a497c2edd3a42af9', 'request_id': 'req-ab1bf131-0322-477d-90fe-ce3946559aa1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:41.979003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5d87a95f8f4531a497c2edd3a42af9', 'request_id': 'req-ab1bf131-0322-477d-90fe-ce3946559aa1'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:42.003173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] The token cache is being invalidate because project aa5d87a95f8f4531a497c2edd3a42af9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:42.004233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:42.006489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab1bf131-0322-477d-90fe-ce3946559aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:42.007201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:42.014280 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3893/7753] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:41 2022] DELETE /identity/v3/projects/aa5d87a95f8f4531a497c2edd3a42af9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:42.021932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0290-6576-46dd-ae59-39412b0069f5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.023216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0290-6576-46dd-ae59-39412b0069f5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.023609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0290-6576-46dd-ae59-39412b0069f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.077170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-24cf0290-6576-46dd-ae59-39412b0069f5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:42.079255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24cf0290-6576-46dd-ae59-39412b0069f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:42.093990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993be99c-4aaf-4db5-8660-fdf9a36d1f7c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.099446 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3894/7754] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:42.104165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.104950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.105431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.107525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a5325-0956-48f6-9ce3-c461f47a1ac5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.107911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a5325-0956-48f6-9ce3-c461f47a1ac5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.108543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a5325-0956-48f6-9ce3-c461f47a1ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.110922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.112914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.116947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.146366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3861/7755] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:42.151978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.157764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:42.161114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1c9a5325-0956-48f6-9ce3-c461f47a1ac5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:42.165634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1c9a5325-0956-48f6-9ce3-c461f47a1ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:42.188785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145f02d-73cf-4024-a0c5-e57686b9228f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.194520 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3895/7756] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:42.197742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a5325-0956-48f6-9ce3-c461f47a1ac5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.224270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:42.225763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.226812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.227206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.227809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.228768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.228921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a5325-0956-48f6-9ce3-c461f47a1ac5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.234026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.245204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.246017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.246177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.246399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.247047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:42.247350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.248951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.262215 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3862/7757] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:42.263416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.282013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131734a7-f57a-4fa0-9522-d6dd5afb938c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.292876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26c8fc-c297-485c-8123-70938472aea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.294375 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3896/7758] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:42.294857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26c8fc-c297-485c-8123-70938472aea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.295252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26c8fc-c297-485c-8123-70938472aea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.295467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.295766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26c8fc-c297-485c-8123-70938472aea9 admin admin] PATH_INFO: `/v3/projects/ddb8677ca13a45b896e780202cce6152/users/c6fe051de7d24488b8f42829af27d630/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.307244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d5903-3e42-4ec5-8c3e-db1ba2dc21fa admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.325687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.326371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.326661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.326936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.327589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.329514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.339071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26c8fc-c297-485c-8123-70938472aea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb8677ca13a45b896e780202cce6152, user_id=c6fe051de7d24488b8f42829af27d630, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.343096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26c8fc-c297-485c-8123-70938472aea9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.371660 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3897/7759] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:42.372524 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3863/7760] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:42 2022] PUT /identity/v3/projects/ddb8677ca13a45b896e780202cce6152/users/c6fe051de7d24488b8f42829af27d630/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:42.382753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.383840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582b9c9-1680-48ba-a3b9-83c227961516 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.384383 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582b9c9-1680-48ba-a3b9-83c227961516 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.384772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582b9c9-1680-48ba-a3b9-83c227961516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.469954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec4e8a-ecaa-45df-bdf5-f276dbdb8d32 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.478921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.479686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.479975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.480256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.480895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.483184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.484529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3582b9c9-1680-48ba-a3b9-83c227961516 None None] MFA Rules not processed for user `c6fe051de7d24488b8f42829af27d630`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:42.489791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3582b9c9-1680-48ba-a3b9-83c227961516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:42.513646 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3898/7761] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:42.527145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.541284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91761c73-ef23-42db-a9b6-2337180562b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.551976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.553108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.553482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.553852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.554606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.557680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.574953 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3899/7762] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:42.578647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.592315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0f0f3-a48d-4c82-a01a-7ddbda0952e9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.601223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.602336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.602696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.603048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.603943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.607248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.624249 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3900/7763] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:42.630159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.644973 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3864/7764] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 10 21:18:42.646266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582b9c9-1680-48ba-a3b9-83c227961516 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.652831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ffea-c806-4ae6-b23b-e0d412a294a7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.661160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3905-0bcc-46a7-beb1-ca8e57e13ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.662269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582b9c9-1680-48ba-a3b9-83c227961516 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.662498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3905-0bcc-46a7-beb1-ca8e57e13ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.664531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3905-0bcc-46a7-beb1-ca8e57e13ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.665094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3905-0bcc-46a7-beb1-ca8e57e13ca6 admin admin] PATH_INFO: `/v3/projects/b5963e3c1ac34aa29de539734a0c6466/users/eaf8e35a0a6845768bd0ad61c9b87e22/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.667845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.668778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.669104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.669490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.671089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.673747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.693208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a3905-0bcc-46a7-beb1-ca8e57e13ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5963e3c1ac34aa29de539734a0c6466, user_id=eaf8e35a0a6845768bd0ad61c9b87e22, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.696631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a3905-0bcc-46a7-beb1-ca8e57e13ca6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.738078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3901/7765] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:42 2022] PUT /identity/v3/projects/b5963e3c1ac34aa29de539734a0c6466/users/eaf8e35a0a6845768bd0ad61c9b87e22/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:42.742798 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3865/7766] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:42.743950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.744176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.744549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.746393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.800118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] MFA Rules not processed for user `eaf8e35a0a6845768bd0ad61c9b87e22`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:42.802476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:42.900981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d43e77-c88b-438b-b8e4-d606e1add05c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.912188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.916854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.917151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.918036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.918703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.922647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:42.933151 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3902/7767] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:42.935533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.937310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:42.949960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:42.952572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:42.963059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70a9b4-906c-4db9-8a84-17f1886b202a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:42.972661 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3866/7768] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:42 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:42.974590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:42.978733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:42.981134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:42.982206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:42.983043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:42.983043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] PATH_INFO: `/v3/projects/3dd3c941735c456ab0328d6b8e6b84c4` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:42.986337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd3c941735c456ab0328d6b8e6b84c4)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:42.988343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.001514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a924e4-c43d-474a-a055-d6977c77a007 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.007328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.008029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.008318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.008600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] PATH_INFO: `/v3/users/85a893f59440418b9c9c34343e7672a0` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.009819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a893f59440418b9c9c34343e7672a0)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.011802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.074461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd3c941735c456ab0328d6b8e6b84c4', 'request_id': 'req-03fb6eac-7993-4177-80e7-b0f2e02c0390'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.080488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a893f59440418b9c9c34343e7672a0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.083709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd3c941735c456ab0328d6b8e6b84c4', 'request_id': 'req-03fb6eac-7993-4177-80e7-b0f2e02c0390'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.091406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd3c941735c456ab0328d6b8e6b84c4', 'request_id': 'req-03fb6eac-7993-4177-80e7-b0f2e02c0390'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.099969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a893f59440418b9c9c34343e7672a0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.101367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a893f59440418b9c9c34343e7672a0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.112635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a893f59440418b9c9c34343e7672a0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.115094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a893f59440418b9c9c34343e7672a0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.118127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] The token cache is being invalidate because project 3dd3c941735c456ab0328d6b8e6b84c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:43.118693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.120003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:43.127250 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3903/7769] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:42 2022] DELETE /identity/v3/projects/3dd3c941735c456ab0328d6b8e6b84c4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:43.128596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.131215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.151153 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3867/7770] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:42 2022] DELETE /identity/v3/users/85a893f59440418b9c9c34343e7672a0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:43.151153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.151153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.241395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6eac-7993-4177-80e7-b0f2e02c0390 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.249356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b367e1-3fc0-41fb-9234-d3259dee3ed6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.257422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c42c47-4bfa-48b8-82e4-b61d659c4e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.258419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42c47-4bfa-48b8-82e4-b61d659c4e75 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.258419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42c47-4bfa-48b8-82e4-b61d659c4e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.258797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42c47-4bfa-48b8-82e4-b61d659c4e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.259569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c42c47-4bfa-48b8-82e4-b61d659c4e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.261474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c42c47-4bfa-48b8-82e4-b61d659c4e75 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.279021 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3868/7771] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:43.279021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfa63-c36e-466f-bbc5-a3b84d28bf0e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.279021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfa63-c36e-466f-bbc5-a3b84d28bf0e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.279021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcfa63-c36e-466f-bbc5-a3b84d28bf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.303381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-80fcfa63-c36e-466f-bbc5-a3b84d28bf0e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:43.305067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-80fcfa63-c36e-466f-bbc5-a3b84d28bf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.318381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.319395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.319900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.320479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.322796 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3869/7772] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:43.323536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.324247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.324555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.324823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.346770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:43.348423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.361705 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3870/7773] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:43.362325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.364543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.378399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993ff8d-9768-4b56-8c0e-a6b9f39bf227 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.387059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd64a54-46f7-43f4-828b-e4bac87eb291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.387902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64a54-46f7-43f4-828b-e4bac87eb291 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.388197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64a54-46f7-43f4-828b-e4bac87eb291 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.388480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64a54-46f7-43f4-828b-e4bac87eb291 admin admin] PATH_INFO: `/v3/projects/480e5bd0abc74358a31d3fd8b136d715/users/1c55d6651885486997fdab2e11198622/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.406014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:43.407353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64a54-46f7-43f4-828b-e4bac87eb291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480e5bd0abc74358a31d3fd8b136d715, user_id=1c55d6651885486997fdab2e11198622, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.410219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64a54-46f7-43f4-828b-e4bac87eb291 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.425817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.429525 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3871/7774] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:43 2022] PUT /identity/v3/projects/480e5bd0abc74358a31d3fd8b136d715/users/1c55d6651885486997fdab2e11198622/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:43.430534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.431297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4829b32-2767-4a89-9dfa-f2f026317b45 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.431656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4829b32-2767-4a89-9dfa-f2f026317b45 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.432114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4829b32-2767-4a89-9dfa-f2f026317b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.446271 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3904/7775] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:43.447490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.449483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.458628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a4829b32-2767-4a89-9dfa-f2f026317b45 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:43.460006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4829b32-2767-4a89-9dfa-f2f026317b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.482076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:43.563480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a5296-0414-44f4-bd5a-dc8911781273 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.568195 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3872/7776] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:43.573085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4829b32-2767-4a89-9dfa-f2f026317b45 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.590479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4829b32-2767-4a89-9dfa-f2f026317b45 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.600747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.601655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.602146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.602582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.603387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.605656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.618539 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3873/7777] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:43.620341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.632658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7462-26a9-49c6-9d5c-1c43cd0e73e9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.638285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.639177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.639644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.640338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.641170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:43.641640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.644417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.664120 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3874/7778] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:43.666749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.672912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.674157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.674441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.674718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.680601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041c58f-a052-40ee-b9a4-da49aae9491a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.688744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4136de9-68af-46a1-87e2-e180f64a671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.689584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.689925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136de9-68af-46a1-87e2-e180f64a671a admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.690398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136de9-68af-46a1-87e2-e180f64a671a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.690850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136de9-68af-46a1-87e2-e180f64a671a admin admin] PATH_INFO: `/v3/projects/480e5bd0abc74358a31d3fd8b136d715/users/1c55d6651885486997fdab2e11198622/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.691200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.694255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4136de9-68af-46a1-87e2-e180f64a671a admin admin] RBAC: Authorizing `identity:create_grant(project_id=480e5bd0abc74358a31d3fd8b136d715, user_id=1c55d6651885486997fdab2e11198622, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.698181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4136de9-68af-46a1-87e2-e180f64a671a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.704351 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3905/7779] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:43.705361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.710982 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3875/7780] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:43 2022] PUT /identity/v3/projects/480e5bd0abc74358a31d3fd8b136d715/users/1c55d6651885486997fdab2e11198622/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:43.724338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade6bec-4f10-4e92-a922-e29678df5e0a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.724566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade6bec-4f10-4e92-a922-e29678df5e0a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.724775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade6bec-4f10-4e92-a922-e29678df5e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.725732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb628a5a-3217-4961-bf7a-da0ef834467c service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.769506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7ade6bec-4f10-4e92-a922-e29678df5e0a None None] MFA Rules not processed for user `1c55d6651885486997fdab2e11198622`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:43.771027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7ade6bec-4f10-4e92-a922-e29678df5e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:43.820508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:43.821220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:43.821641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:43.821961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:43.822747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:43.825301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:43.894215 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3876/7781] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:43.895851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade6bec-4f10-4e92-a922-e29678df5e0a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:43.910199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade6bec-4f10-4e92-a922-e29678df5e0a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:43.911878 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3906/7782] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:43 2022] POST /identity/v3/projects => generated 409 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:43.916130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.010648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:44.095427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.096326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.096593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.096892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.097571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.099616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.123956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a0e78-ab4e-4b67-81e3-9f162c2c0f64 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.147770 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3877/7783] 10.222.0.33 () {60 vars in 1152 bytes} [Sat Sep 10 21:18:43 2022] POST /identity/v3/services => generated 240 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:44.151424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.164952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:44.171748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.172529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.172760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.173942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.174649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.180946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.210617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cb348-d5d3-43ec-abd5-c51a8077c79e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.240121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.240806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.241095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.241377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/e4e3e345ade44baf9586d14ca77cf489` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.243008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:delete_service(service_id=e4e3e345ade44baf9586d14ca77cf489)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.248475 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3907/7784] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:43 2022] POST /identity/v3/users => generated 305 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:44.252092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.256988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.276660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a678713-7347-4ee3-87fc-52ac17ed67ae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.282190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.282833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.283121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.283449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.284164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.286846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.298756 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3878/7785] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:44 2022] DELETE /identity/v3/services/e4e3e345ade44baf9586d14ca77cf489 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:44.304669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.333560 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3908/7786] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:44.335175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.352190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017ef2-cd92-4583-aa0b-0ad651681c53 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.354444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752bd4f-0e9a-4825-88e1-239ae013e490 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.364886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.365731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.366240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.366690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.368117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.370465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.373423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.374091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.374529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.374812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.375896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.385313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.400005 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3879/7787] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:44 2022] POST /identity/v3/services => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:44.401123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.405698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:44.413437 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3909/7788] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:44.414603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.439478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cfe6e-9694-4753-bce5-2e7d0fb2c672 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.450443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29539ff-9a7b-4f8f-b308-1b5d762265cc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.457117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.458847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.459203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.459715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] PATH_INFO: `/v3/projects/bc38caa2157f4734b72a9883ca56b781` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.460974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc38caa2157f4734b72a9883ca56b781)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.463161 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.480151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.480895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.481189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.481847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/96414a5ad1a94642ab24acc49d715e22` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.482497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:update_service(service_id=96414a5ad1a94642ab24acc49d715e22)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.484121 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:44.484121 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:44.484121 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:44.485072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.525107 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3910/7789] 10.222.0.33 () {60 vars in 1252 bytes} [Sat Sep 10 21:18:44 2022] PATCH /identity/v3/services/96414a5ad1a94642ab24acc49d715e22 => generated 283 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:44.526063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.564476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc38caa2157f4734b72a9883ca56b781', 'request_id': 'req-7d286817-6275-4328-bd65-a6720142ebbb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:44.586534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc38caa2157f4734b72a9883ca56b781', 'request_id': 'req-7d286817-6275-4328-bd65-a6720142ebbb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:44.590822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e057d88-03e1-4265-8fc1-d742246e7f38 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.610990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc38caa2157f4734b72a9883ca56b781', 'request_id': 'req-7d286817-6275-4328-bd65-a6720142ebbb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:44.671044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] The token cache is being invalidate because project bc38caa2157f4734b72a9883ca56b781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:44.671346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:44.672996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:44.682002 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3880/7790] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:44 2022] DELETE /identity/v3/projects/bc38caa2157f4734b72a9883ca56b781 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:44.683198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.690774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:44.775041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:44.776374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:44.776572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:44.776858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] PATH_INFO: `/v3/projects/4a040022cdce4aaa855fbd8e1211e762/users/4e22d63d90b843fdb6ede77b568414d2/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:44.820232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a040022cdce4aaa855fbd8e1211e762, user_id=4e22d63d90b843fdb6ede77b568414d2, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:44.823154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:44.862693 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3911/7791] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:44 2022] PUT /identity/v3/projects/4a040022cdce4aaa855fbd8e1211e762/users/4e22d63d90b843fdb6ede77b568414d2/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:44.862693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:44.862693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:44.870254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d286817-6275-4328-bd65-a6720142ebbb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:44.929475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:44.943634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:45.023930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12178b-98d8-49f7-ad19-a1014bb88f93 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:45.039194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:45.039878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec157d4-9f5a-4079-a412-be405e520693 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:45.040411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.041429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec157d4-9f5a-4079-a412-be405e520693 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.041429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec157d4-9f5a-4079-a412-be405e520693 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.041750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec157d4-9f5a-4079-a412-be405e520693 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/96414a5ad1a94642ab24acc49d715e22` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.042034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.042607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] PATH_INFO: `/v3/users/11f9a1b3571b4092a48cd1e5f5c20a06` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.043080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec157d4-9f5a-4079-a412-be405e520693 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:get_service(service_id=96414a5ad1a94642ab24acc49d715e22)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:45.044159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f9a1b3571b4092a48cd1e5f5c20a06)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:45.044538 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-dec157d4-9f5a-4079-a412-be405e520693 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:45.044538 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:18:45.044538 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:18:45.045538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec157d4-9f5a-4079-a412-be405e520693 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:45.046893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:45.100735 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3881/7792] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:18:44 2022] GET /identity/v3/services/96414a5ad1a94642ab24acc49d715e22 => generated 283 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:45.107096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ac6d2-a2be-48cc-bf32-0f2c9e1e9839 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.107487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ac6d2-a2be-48cc-bf32-0f2c9e1e9839 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.107672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ac6d2-a2be-48cc-bf32-0f2c9e1e9839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.200707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f9a1b3571b4092a48cd1e5f5c20a06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:45.226743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f9a1b3571b4092a48cd1e5f5c20a06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:45.232677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c95ac6d2-a2be-48cc-bf32-0f2c9e1e9839 None None] MFA Rules not processed for user `4e22d63d90b843fdb6ede77b568414d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:45.234420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c95ac6d2-a2be-48cc-bf32-0f2c9e1e9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:45.247751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f9a1b3571b4092a48cd1e5f5c20a06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:45.267699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f9a1b3571b4092a48cd1e5f5c20a06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:45.268952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f9a1b3571b4092a48cd1e5f5c20a06'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:45.282596 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3912/7793] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:44 2022] DELETE /identity/v3/users/11f9a1b3571b4092a48cd1e5f5c20a06 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:45.283860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:45.287059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:45.349477 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3882/7794] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3419 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:45.351356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5670183-e674-4f22-909d-081f1e96096d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.352513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5670183-e674-4f22-909d-081f1e96096d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.353622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5670183-e674-4f22-909d-081f1e96096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.376965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c5670183-e674-4f22-909d-081f1e96096d None None] MFA Rules not processed for user `eaf8e35a0a6845768bd0ad61c9b87e22`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:45.378720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c5670183-e674-4f22-909d-081f1e96096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:45.397447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b5985-d5f4-45fa-8e91-25bcf7b32e94 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:45.407266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:45.407932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.408223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.408562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/96414a5ad1a94642ab24acc49d715e22` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.409159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:delete_service(service_id=96414a5ad1a94642ab24acc49d715e22)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:45.411674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:45.450626 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3913/7795] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:45 2022] DELETE /identity/v3/services/96414a5ad1a94642ab24acc49d715e22 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:45.455773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:45.458407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:45.524392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:45.537099 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3883/7796] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:45.537444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5670183-e674-4f22-909d-081f1e96096d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:45.584913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5670183-e674-4f22-909d-081f1e96096d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:45.599045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:45.599887 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.600177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.600456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/96414a5ad1a94642ab24acc49d715e22` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.601192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:get_service(service_id=96414a5ad1a94642ab24acc49d715e22)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:45.604950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Could not find service: 96414a5ad1a94642ab24acc49d715e22.: keystone.exception.ServiceNotFound: Could not find service: 96414a5ad1a94642ab24acc49d715e22. Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 96414a5ad1a94642ab24acc49d715e22. Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:18:45.638393 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 21:18:45.639769 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application with Lock( Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self._enter() Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.creator() Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application created_value = creator( Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 96414a5ad1a94642ab24acc49d715e22. Sep 10 21:18:45.641046 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:18:45.643262 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3884/7797] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:18:45 2022] GET /identity/v3/services/96414a5ad1a94642ab24acc49d715e22 => generated 113 bytes in 106 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:18:45.650389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:45.668402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca846d20-386b-4d6f-b9f8-3550558d2c61 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:45.676807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d951b-19b5-4da2-b8d1-90b01b45f38e tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:45.683954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:45.684856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.685245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.685623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.686538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:45.688870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:45.689192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:45.689847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.690146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.690433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.692417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:45.694292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:45.751102 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3914/7798] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:45 2022] POST /identity/v3/projects => generated 411 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:45.755307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:45.762096 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3885/7799] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:45 2022] POST /identity/v3/services => generated 274 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:45.763209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:45.814472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ea0e5-6609-4d54-8a91-35c58436ef3c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:45.823249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:45.930758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eecade-9822-4e24-8d33-40156c991d10 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:45.984208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:45.985710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:45.986449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:45.986919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:45.987723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:45.990989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.013595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.017006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.017323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.017598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.018484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.024252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.038400 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3915/7800] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:45 2022] POST /identity/v3/users => generated 312 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:46.042159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.058467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296806e4-ec83-424d-88f9-96c39bc3b924 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.073425 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3886/7801] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:45 2022] POST /identity/v3/services => generated 275 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:46.078971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.107048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.110252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.110782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.111618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.113407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.117576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.132857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b005f-38d1-4c46-8cc8-a4c32f5974ad tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.134252 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3916/7802] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:46.138150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.146719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.148732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.149036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.150199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.151058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.155674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a7f99-5170-4e54-a462-0d35155534ae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.158216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.167320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.168483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.168964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.169430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] PATH_INFO: `/v3/projects/113b912cf2664f5c8a8f5b9504621401/users/1a785e56b49741388e2c9e2593a9c4cb/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.198377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=113b912cf2664f5c8a8f5b9504621401, user_id=1a785e56b49741388e2c9e2593a9c4cb, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.200509 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3887/7803] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:46 2022] POST /identity/v3/services => generated 275 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:46.203697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.204311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.245323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466b64c-0836-4806-ad0e-4594a806bb76 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.261183 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3917/7804] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:46 2022] PUT /identity/v3/projects/113b912cf2664f5c8a8f5b9504621401/users/1a785e56b49741388e2c9e2593a9c4cb/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:46.266215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.393137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:46.405271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.407440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.409916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.412632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.413686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.415424 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:46.415424 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:18:46.415424 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:18:46.416981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.431391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92da39d-9a78-4898-8eee-d867bb602f20 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.432249 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3888/7805] 10.222.0.33 () {58 vars in 1131 bytes} [Sat Sep 10 21:18:46 2022] GET /identity/v3/services => generated 2979 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:46.434360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.436299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:46.477064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.477847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.478151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.478429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.479236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.479944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271cf61-e843-4fcb-bddd-c6fb10b33fbc tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.481760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.501732 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3918/7806] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:46.502845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.505145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.506405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.513363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.514608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] PATH_INFO: `/v3/projects/62f9963a132a4c98ba279f4b444d3885` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.516258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=62f9963a132a4c98ba279f4b444d3885)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.518297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.539032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d68394-5d50-479a-b267-4b36d95d1b8e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.556721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.557731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.558040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.558317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.562201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1225843364) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:46.562489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.563525 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:46.563525 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:18:46.563525 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:18:46.568493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.590574 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3919/7807] 10.222.0.33 () {58 vars in 1222 bytes} [Sat Sep 10 21:18:46 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1225843364 => generated 417 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:46.591188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.623189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148abe6b-ed45-4407-a689-df5d3c054875 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.652543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62f9963a132a4c98ba279f4b444d3885', 'request_id': 'req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:46.668239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62f9963a132a4c98ba279f4b444d3885', 'request_id': 'req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:46.689754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62f9963a132a4c98ba279f4b444d3885', 'request_id': 'req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:46.693050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:46.718577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] The token cache is being invalidate because project 62f9963a132a4c98ba279f4b444d3885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:46.718898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:46.720777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:46.733643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3889/7808] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:46 2022] DELETE /identity/v3/projects/62f9963a132a4c98ba279f4b444d3885 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:46.735212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:46.741918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:46.807851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137f7a7-9f7c-407c-a5b2-3600b92fee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:46.808552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137f7a7-9f7c-407c-a5b2-3600b92fee33 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.808837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137f7a7-9f7c-407c-a5b2-3600b92fee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.809134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137f7a7-9f7c-407c-a5b2-3600b92fee33 admin admin] PATH_INFO: `/v3/projects/113b912cf2664f5c8a8f5b9504621401/users/1a785e56b49741388e2c9e2593a9c4cb/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.817433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137f7a7-9f7c-407c-a5b2-3600b92fee33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=113b912cf2664f5c8a8f5b9504621401, user_id=1a785e56b49741388e2c9e2593a9c4cb, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:46.821018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137f7a7-9f7c-407c-a5b2-3600b92fee33 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:46.856378 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3920/7809] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:46 2022] PUT /identity/v3/projects/113b912cf2664f5c8a8f5b9504621401/users/1a785e56b49741388e2c9e2593a9c4cb/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:46.858121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3605bb-36f9-4411-ae4f-b308d9c81a06 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:46.858415 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3605bb-36f9-4411-ae4f-b308d9c81a06 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:46.858689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3605bb-36f9-4411-ae4f-b308d9c81a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:46.877743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721658-97f1-4c7f-8b8a-6ca58acb6d4a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:46.891139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ca3605bb-36f9-4411-ae4f-b308d9c81a06 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:46.892468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ca3605bb-36f9-4411-ae4f-b308d9c81a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:47.015177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4a4ad-fdc2-4693-a507-f8c275215c23 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.015806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4a4ad-fdc2-4693-a507-f8c275215c23 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.016093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4a4ad-fdc2-4693-a507-f8c275215c23 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.016392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4a4ad-fdc2-4693-a507-f8c275215c23 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.017039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4a4ad-fdc2-4693-a507-f8c275215c23 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1673275007) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:47.017345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4a4ad-fdc2-4693-a507-f8c275215c23 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.019294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4a4ad-fdc2-4693-a507-f8c275215c23 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.033049 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3890/7810] 10.222.0.33 () {58 vars in 1222 bytes} [Sat Sep 10 21:18:46 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1673275007 => generated 418 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:47.033049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372df65-56fa-4d1a-aea2-f6625eb12432 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.033049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372df65-56fa-4d1a-aea2-f6625eb12432 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.033417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372df65-56fa-4d1a-aea2-f6625eb12432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.072390 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3921/7811] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:46 2022] POST /identity/v3/auth/tokens => generated 3742 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:47.073676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3605bb-36f9-4411-ae4f-b308d9c81a06 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.096032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4372df65-56fa-4d1a-aea2-f6625eb12432 None None] MFA Rules not processed for user `1a785e56b49741388e2c9e2593a9c4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:47.098620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4372df65-56fa-4d1a-aea2-f6625eb12432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:47.105546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3605bb-36f9-4411-ae4f-b308d9c81a06 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.124137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8f261-4411-4348-9ce7-1774d101687d tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.125221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8f261-4411-4348-9ce7-1774d101687d tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.126120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8f261-4411-4348-9ce7-1774d101687d tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.126609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8f261-4411-4348-9ce7-1774d101687d tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.127502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8f261-4411-4348-9ce7-1774d101687d tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1093681888) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:47.128274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8f261-4411-4348-9ce7-1774d101687d tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.131015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8f261-4411-4348-9ce7-1774d101687d tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.138902 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3922/7812] 10.222.0.33 () {58 vars in 1222 bytes} [Sat Sep 10 21:18:47 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1093681888 => generated 418 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:47.140593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f12f-4030-4d4e-b37c-7a9ccc39cac1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.141094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f12f-4030-4d4e-b37c-7a9ccc39cac1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.141605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f12f-4030-4d4e-b37c-7a9ccc39cac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.170658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8ba4f12f-4030-4d4e-b37c-7a9ccc39cac1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:47.173459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8ba4f12f-4030-4d4e-b37c-7a9ccc39cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:47.202136 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3923/7813] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:47 2022] POST /identity/v3/auth/tokens => generated 3742 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:47.203746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f12f-4030-4d4e-b37c-7a9ccc39cac1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.232544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f12f-4030-4d4e-b37c-7a9ccc39cac1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.258782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.264603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.265282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.265590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/dd99bd8d2e584b38bedd9e2953276edc` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.267683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:delete_service(service_id=dd99bd8d2e584b38bedd9e2953276edc)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.268321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.291349 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3891/7814] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:47 2022] POST /identity/v3/auth/tokens => generated 3858 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:47.292246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372df65-56fa-4d1a-aea2-f6625eb12432 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.316748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372df65-56fa-4d1a-aea2-f6625eb12432 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.330496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.331073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.331356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.331634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.332860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:47.333167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.334953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.353549 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3924/7815] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:47 2022] DELETE /identity/v3/services/dd99bd8d2e584b38bedd9e2953276edc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:47.355104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.365451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:47.369390 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3892/7816] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:47.370055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.396164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670a5e1-76b1-41ec-8d40-388236f4712e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.403808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:47.413671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:47.421808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.422510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.422796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.423076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/53a1023226e24301ba1b7aa2f37e3ea3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.424302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:delete_service(service_id=53a1023226e24301ba1b7aa2f37e3ea3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.429750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.473811 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3893/7817] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:47 2022] DELETE /identity/v3/services/53a1023226e24301ba1b7aa2f37e3ea3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:47.474760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.494357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52176239-4b8b-453b-a5e5-8cac4fcea994 tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.506776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8ec66-6780-404d-ba93-aa4e5255bd1f tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.510381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.511236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.511236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.511523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.514015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.514748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.515150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.515526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.516265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.518639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.522490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.524487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.552799 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3925/7818] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:47 2022] GET /identity/v3/auth/tokens => generated 3522 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:47.555928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.565567 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3894/7819] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:47 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:47.570919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.602426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d28b-1163-40b4-b884-832192f9976d service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.750240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d527d-0e83-4675-97f5-21b3be76a668 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.761119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.762015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.766419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.766742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.767442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.770810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.820488 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3895/7820] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:47 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:47.824511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.841469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9337ee-daff-467d-b3b5-5c5066cf844b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.850355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c55d6651885486997fdab2e11198622', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '480e5bd0abc74358a31d3fd8b136d715', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.851155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.851540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.851939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] PATH_INFO: `/v3/services/d6a8f019d4ae4241b1fc2d2bbab88181` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.852630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorizing `identity:delete_service(service_id=d6a8f019d4ae4241b1fc2d2bbab88181)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.858811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.859040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.859576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.859869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.860149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.861108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.879954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.908729 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3896/7821] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:47.913680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.922811 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3926/7822] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:47 2022] DELETE /identity/v3/services/d6a8f019d4ae4241b1fc2d2bbab88181 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:47.928635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:47.934756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:47.938606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603bea4-6b41-47c3-b1bf-a83d0f89e7db admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:47.983171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:47.983827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:47.985359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:47.985359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:47.985359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:47.986689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:47.993688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531beb4-d3dc-49d0-952c-a1a6c13bc72c tempest-ServicesTestJSON-1429858222 tempest-ServicesTestJSON-1429858222-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:48.004455 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3897/7823] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:48.006318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:48.011950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:48.014019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.014456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.014745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] PATH_INFO: `/v3/users/c6fe051de7d24488b8f42829af27d630` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.016273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6fe051de7d24488b8f42829af27d630)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:48.021436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:48.024542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682702e-37b9-4a5d-be85-bbfcd553e887 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:48.049324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba31d3-a763-4547-af6c-d18c477b24dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:48.050349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba31d3-a763-4547-af6c-d18c477b24dd admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.052017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba31d3-a763-4547-af6c-d18c477b24dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.052463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba31d3-a763-4547-af6c-d18c477b24dd admin admin] PATH_INFO: `/v3/projects/7748a65ff6f248debe13c9cb6c91177c/users/c501424e97f642089f7c9da42d1c70c6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.107228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba31d3-a763-4547-af6c-d18c477b24dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7748a65ff6f248debe13c9cb6c91177c, user_id=c501424e97f642089f7c9da42d1c70c6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:48.118984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba31d3-a763-4547-af6c-d18c477b24dd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:48.144383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6fe051de7d24488b8f42829af27d630'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:48.163648 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3898/7824] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:48 2022] PUT /identity/v3/projects/7748a65ff6f248debe13c9cb6c91177c/users/c501424e97f642089f7c9da42d1c70c6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:48.166988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6fe051de7d24488b8f42829af27d630'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:48.168422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=11 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:48.174026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.174682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.175196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.179669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6fe051de7d24488b8f42829af27d630'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:48.199674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6fe051de7d24488b8f42829af27d630'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:48.202256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52098ba9-89d6-42eb-87a8-72b577d58e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6fe051de7d24488b8f42829af27d630'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:48.214411 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3927/7825] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:47 2022] DELETE /identity/v3/users/c6fe051de7d24488b8f42829af27d630 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:48.222073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.222683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.223905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.252647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:48.254896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:48.285088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] MFA Rules not processed for user `c501424e97f642089f7c9da42d1c70c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:48.287410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:48.361875 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3928/7826] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:48.392907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:48.418249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:48.420331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:48.527241 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3899/7827] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:48.528377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:48.530298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:48.550688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31ae2-2682-412c-8fab-1c9204282365 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:48.564421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:48.565323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.567352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.567801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.568654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:48.573440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:48.624463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6757f-3065-4015-bf50-d8d5dedef3b2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:48.636032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:48.636668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.642087 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3900/7828] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:48 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:48.643060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:48.643646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.643998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.673855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:48.677004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:48.691073 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3929/7829] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:48.771500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9556a7-e6f5-4ae0-a0a6-237a18ac1c37 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:48.779347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:48.780262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.780707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.781181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.782648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:48.786925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:48.835117 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3901/7830] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:48 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:48.839107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:48.884166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66426f23-7b25-4db2-ade8-5a0765b9bae8 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:48.902901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:48.906396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.906704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.906963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.909066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:48.917748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:48.952402 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3930/7831] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:48.955163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:48.975701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56346a38-37d0-411f-848e-5014c12a7d66 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:48.996864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c83a7-06ae-453f-81b0-b2bbcf58910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:48.997899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c83a7-06ae-453f-81b0-b2bbcf58910a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:48.998415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c83a7-06ae-453f-81b0-b2bbcf58910a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:48.998901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c83a7-06ae-453f-81b0-b2bbcf58910a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:48.999768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c83a7-06ae-453f-81b0-b2bbcf58910a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:49.001817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c83a7-06ae-453f-81b0-b2bbcf58910a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:49.020558 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3902/7832] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:49.021802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:49.037019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22769a-272c-4361-beee-65eb9544356f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:49.043717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:49.045205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:49.045514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:49.045794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] PATH_INFO: `/v3/projects/bbdb725754214f05af47df40f9722acb/users/7d7cfedd7ff841e983f6f8f6ead327c4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:49.073438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbdb725754214f05af47df40f9722acb, user_id=7d7cfedd7ff841e983f6f8f6ead327c4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:49.077575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:49.115053 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3931/7833] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:49 2022] PUT /identity/v3/projects/bbdb725754214f05af47df40f9722acb/users/7d7cfedd7ff841e983f6f8f6ead327c4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:49.117220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:49.117696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:49.118188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:49.187338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] MFA Rules not processed for user `7d7cfedd7ff841e983f6f8f6ead327c4`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:49.188974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:49.320672 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3903/7834] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:49.324469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:49.417981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:49.475947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:49.477370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46373653-c73d-4abb-851d-45292c349ac7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:49.480305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:49.493601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:49.494093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:49.494477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:49.494749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:49.499621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:49.504038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:49.535056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9beb3-4f8e-4b36-a400-3d27f3885972 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:49.560384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:49.563911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:49.564229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:49.564511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] PATH_INFO: `/v3/projects/ddb8677ca13a45b896e780202cce6152` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:49.568423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb8677ca13a45b896e780202cce6152)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:49.571075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:49.588663 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3932/7835] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:49 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:49.592457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:49.605418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:49.764310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb8677ca13a45b896e780202cce6152', 'request_id': 'req-a313f68f-0938-41aa-87d2-fd061fbbeca3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:49.785014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb8677ca13a45b896e780202cce6152', 'request_id': 'req-a313f68f-0938-41aa-87d2-fd061fbbeca3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:49.806056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb8677ca13a45b896e780202cce6152', 'request_id': 'req-a313f68f-0938-41aa-87d2-fd061fbbeca3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:49.831009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] The token cache is being invalidate because project ddb8677ca13a45b896e780202cce6152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:49.831315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:49.832910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:49.840335 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3904/7836] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:49 2022] DELETE /identity/v3/projects/ddb8677ca13a45b896e780202cce6152 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:49.841635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:49.842996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:49.871571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9377d6-ef76-49ce-820b-a07d94230ea2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:49.951391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:49.951960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:49.952656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:49.952945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:49.953584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:49.956165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:50.000918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313f68f-0938-41aa-87d2-fd061fbbeca3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:50.006947 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3933/7837] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:49 2022] POST /identity/v3/users => generated 298 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:50.008316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:50.012193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:50.016733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:50.018009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:50.018481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:50.018951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] PATH_INFO: `/v3/users/1c55d6651885486997fdab2e11198622` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:50.023685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c55d6651885486997fdab2e11198622)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:50.031088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:50.042539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affec48c-52c3-49fc-94cf-8c05cdc25adf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:50.060780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:50.061337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:50.062373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:50.062781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:50.064352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:50.067033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:50.081952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3934/7838] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:50.087392 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:50.107483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2cfdc-8b40-4261-8df4-eace033e6bd3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:50.175633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:50.177228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:50.177517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:50.177799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] PATH_INFO: `/v3/projects/67ae7b20dc114a978f792b5e9481bf9d/users/895c0ca90f844e0db7feb668f7cc5a47/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:50.179875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c55d6651885486997fdab2e11198622'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:50.189789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c55d6651885486997fdab2e11198622'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:50.191263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c55d6651885486997fdab2e11198622'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:50.217655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c55d6651885486997fdab2e11198622'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:50.220361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c55d6651885486997fdab2e11198622'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:50.235112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ae7b20dc114a978f792b5e9481bf9d, user_id=895c0ca90f844e0db7feb668f7cc5a47, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:50.247817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:50.261528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3905/7839] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:49 2022] DELETE /identity/v3/users/1c55d6651885486997fdab2e11198622 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:50.287728 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3935/7840] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:50 2022] PUT /identity/v3/projects/67ae7b20dc114a978f792b5e9481bf9d/users/895c0ca90f844e0db7feb668f7cc5a47/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:50.288782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:50.290323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:50.334410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:50.424574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c016e60-467a-4cf1-96cf-c23e62b72bcd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:50.598653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539bc14-45ab-4428-a0da-280adc6f5bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:50.599505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539bc14-45ab-4428-a0da-280adc6f5bdd admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:50.600753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539bc14-45ab-4428-a0da-280adc6f5bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:50.601267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539bc14-45ab-4428-a0da-280adc6f5bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:50.603581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539bc14-45ab-4428-a0da-280adc6f5bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:50.608380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539bc14-45ab-4428-a0da-280adc6f5bdd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:50.625053 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3906/7841] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:50.631248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:50.647037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afdc6f-0b7d-4c9d-820f-1e66c68c9ce9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:50.661325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:50.662061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:50.662348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:50.662769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] PATH_INFO: `/v3/projects/67ae7b20dc114a978f792b5e9481bf9d/users/895c0ca90f844e0db7feb668f7cc5a47/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:50.667255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ae7b20dc114a978f792b5e9481bf9d, user_id=895c0ca90f844e0db7feb668f7cc5a47, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:50.672858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:50.704636 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3936/7842] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:50 2022] PUT /identity/v3/projects/67ae7b20dc114a978f792b5e9481bf9d/users/895c0ca90f844e0db7feb668f7cc5a47/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:50.715583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:50.716184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:50.716698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:50.761205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] MFA Rules not processed for user `895c0ca90f844e0db7feb668f7cc5a47`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:50.762744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:50.909186 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3907/7843] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:50.926783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:50.929349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:50.987489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:51.048417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:51.050722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:51.120232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfea94-3db7-40b2-8b6b-ccec074cca63 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:51.134660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee3963-d5b9-49bd-8d2f-dc35baed4783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:51.135301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee3963-d5b9-49bd-8d2f-dc35baed4783 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:51.135580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee3963-d5b9-49bd-8d2f-dc35baed4783 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:51.135856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee3963-d5b9-49bd-8d2f-dc35baed4783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:51.138920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f8ee3963-d5b9-49bd-8d2f-dc35baed4783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:51.156819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:51.202137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9a23c-b7c6-420f-a487-7aa2a04e579a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:51.220831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:51.221597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:51.221759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:51.222082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] PATH_INFO: `/v3/projects/480e5bd0abc74358a31d3fd8b136d715` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:51.224002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=480e5bd0abc74358a31d3fd8b136d715)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:51.226160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:51.287579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee3963-d5b9-49bd-8d2f-dc35baed4783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:51.289460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee3963-d5b9-49bd-8d2f-dc35baed4783 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:51.308020 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3937/7844] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:51.309327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480e5bd0abc74358a31d3fd8b136d715', 'request_id': 'req-138b1533-cdf4-456a-a829-69956eabf25d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:51.319128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480e5bd0abc74358a31d3fd8b136d715', 'request_id': 'req-138b1533-cdf4-456a-a829-69956eabf25d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:51.333295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480e5bd0abc74358a31d3fd8b136d715', 'request_id': 'req-138b1533-cdf4-456a-a829-69956eabf25d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:51.358361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] The token cache is being invalidate because project 480e5bd0abc74358a31d3fd8b136d715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:51.358922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:51.360634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-138b1533-cdf4-456a-a829-69956eabf25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:51.367660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaf935-6c5e-49f4-b9be-03f7bc8a9be5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:51.368315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaf935-6c5e-49f4-b9be-03f7bc8a9be5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:51.368757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaf935-6c5e-49f4-b9be-03f7bc8a9be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:51.372649 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3908/7845] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:51 2022] DELETE /identity/v3/projects/480e5bd0abc74358a31d3fd8b136d715 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:51.397287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8deaf935-6c5e-49f4-b9be-03f7bc8a9be5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:51.399304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8deaf935-6c5e-49f4-b9be-03f7bc8a9be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:51.493114 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3938/7846] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:51.504811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1743a14-52ba-4529-b4ca-efa2c1967587 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:51.505109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1743a14-52ba-4529-b4ca-efa2c1967587 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:51.505265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1743a14-52ba-4529-b4ca-efa2c1967587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:51.535207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c1743a14-52ba-4529-b4ca-efa2c1967587 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:51.537460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c1743a14-52ba-4529-b4ca-efa2c1967587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:51.572695 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3909/7847] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:51.574266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaf935-6c5e-49f4-b9be-03f7bc8a9be5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:51.587177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaf935-6c5e-49f4-b9be-03f7bc8a9be5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:51.594495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:51.595230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:51.595532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:51.595919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:51.596694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:51.597011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:51.602307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:51.648000 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3939/7848] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:51.651809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1743a14-52ba-4529-b4ca-efa2c1967587 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:51.668747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1743a14-52ba-4529-b4ca-efa2c1967587 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:51.680660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:51.681452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:51.682080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:51.682355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:51.683124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:51.685177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:51.745739 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3910/7849] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:51 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:51.746757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:51.767032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc74d35-3bde-4e27-b062-c7561cfab0a0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:51.896107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:51.905180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:51.905698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:51.906320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:51.906663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:51.907896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:51.911597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:51.976080 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3940/7850] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:51 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:51.978131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.011472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598edc96-6705-4961-a968-b1ddd121ebff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:52.033747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:52.036404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.037233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.038212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.040586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:52.043162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:52.084817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.102152 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3911/7851] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:52.109629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ae51-13a5-4c72-8488-9ec05ab84746 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:52.123697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73ec6a-102b-462c-9566-2015a26cb93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:52.124784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73ec6a-102b-462c-9566-2015a26cb93c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.125498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73ec6a-102b-462c-9566-2015a26cb93c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.126040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73ec6a-102b-462c-9566-2015a26cb93c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.127321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73ec6a-102b-462c-9566-2015a26cb93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:52.130412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73ec6a-102b-462c-9566-2015a26cb93c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:52.146206 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3941/7852] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:52.150397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.174537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcca15-a16b-43da-a89c-9b5ea319a534 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:52.189574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:52.192806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.192806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.192806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] PATH_INFO: `/v3/projects/cab5bdf63abd4e8ca2b8d7b3c7b3fe7f/users/2e13289108c54f8c89c1bf43ab4b98ab/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.229531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab5bdf63abd4e8ca2b8d7b3c7b3fe7f, user_id=2e13289108c54f8c89c1bf43ab4b98ab, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:52.233030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:52.270438 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3912/7853] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:52 2022] PUT /identity/v3/projects/cab5bdf63abd4e8ca2b8d7b3c7b3fe7f/users/2e13289108c54f8c89c1bf43ab4b98ab/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:52.272480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4f73-5454-4d6a-883e-d5de7d95f898 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.272991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4f73-5454-4d6a-883e-d5de7d95f898 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.273338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4f73-5454-4d6a-883e-d5de7d95f898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.330606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e88f4f73-5454-4d6a-883e-d5de7d95f898 None None] MFA Rules not processed for user `2e13289108c54f8c89c1bf43ab4b98ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:52.333468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e88f4f73-5454-4d6a-883e-d5de7d95f898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:52.480404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3942/7854] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:52.485195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.589711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c8754-3e16-4753-b820-33a0d332d4c5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:52.615554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:52.616422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.616842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.623627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.624569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:52.627470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:52.661372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:52.678055 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3913/7855] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:52 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:52.683247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f4f73-5454-4d6a-883e-d5de7d95f898 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.702854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f4f73-5454-4d6a-883e-d5de7d95f898 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:52.798571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:52.799630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.800017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.802303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.803460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:52.806006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:52.844752 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3943/7856] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:52 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:52.849223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.861879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a935c33-14b7-4ec1-b1f3-4bd73e85d463 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:52.868038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:52.868664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.868952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.869220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.869894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:52.871487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:52.883814 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3914/7857] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:52.888113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.898901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbfa23-6e00-4076-995c-8c81138ac026 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:52.904089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:52.905225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:52.905420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:52.906230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] PATH_INFO: `/v3/projects/58f5895f8b2f4cf585b86dda9ab4fd76/users/8e4d7fa14359481cac6c2ca9a0ae853f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:52.923316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f5895f8b2f4cf585b86dda9ab4fd76, user_id=8e4d7fa14359481cac6c2ca9a0ae853f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:52.926550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:52.942678 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3944/7858] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:52 2022] PUT /identity/v3/projects/58f5895f8b2f4cf585b86dda9ab4fd76/users/8e4d7fa14359481cac6c2ca9a0ae853f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:52.946482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:52.959896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef8e6-7270-4470-81b7-07d7a0655489 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:53.055958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3178e0-8814-447b-9429-0062473892f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:53.056637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3178e0-8814-447b-9429-0062473892f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.057140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3178e0-8814-447b-9429-0062473892f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.057680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3178e0-8814-447b-9429-0062473892f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.058242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3178e0-8814-447b-9429-0062473892f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:53.059994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3178e0-8814-447b-9429-0062473892f7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:53.073867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3915/7859] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:53.077254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:53.089696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e000b-ea6a-4161-a44d-b012b08be56b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:53.096641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf4527-497b-4710-ac9c-5bc6db557134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:53.098460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf4527-497b-4710-ac9c-5bc6db557134 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.098729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf4527-497b-4710-ac9c-5bc6db557134 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.098952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf4527-497b-4710-ac9c-5bc6db557134 admin admin] PATH_INFO: `/v3/projects/58f5895f8b2f4cf585b86dda9ab4fd76/users/8e4d7fa14359481cac6c2ca9a0ae853f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.101028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf4527-497b-4710-ac9c-5bc6db557134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f5895f8b2f4cf585b86dda9ab4fd76, user_id=8e4d7fa14359481cac6c2ca9a0ae853f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:53.104650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf4527-497b-4710-ac9c-5bc6db557134 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:53.125905 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3945/7860] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:53 2022] PUT /identity/v3/projects/58f5895f8b2f4cf585b86dda9ab4fd76/users/8e4d7fa14359481cac6c2ca9a0ae853f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:53.137270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed30e-375f-4388-9ab5-1fd6ac0883ab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.137642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed30e-375f-4388-9ab5-1fd6ac0883ab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.137965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed30e-375f-4388-9ab5-1fd6ac0883ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.179669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f7bed30e-375f-4388-9ab5-1fd6ac0883ab None None] MFA Rules not processed for user `8e4d7fa14359481cac6c2ca9a0ae853f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:53.181656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f7bed30e-375f-4388-9ab5-1fd6ac0883ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:53.182581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e07c8-36f9-4daf-ad0b-62696467b318 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.182839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e07c8-36f9-4daf-ad0b-62696467b318 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.184151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e07c8-36f9-4daf-ad0b-62696467b318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.227975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-958e07c8-36f9-4daf-ad0b-62696467b318 None None] MFA Rules not processed for user `93648515d8fb42a7aaf61e245db0e07d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:53.232103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-958e07c8-36f9-4daf-ad0b-62696467b318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:53.246214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:53.319269 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3916/7861] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:53.330518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bed30e-375f-4388-9ab5-1fd6ac0883ab None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:53.353011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bed30e-375f-4388-9ab5-1fd6ac0883ab None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:53.361522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:53.362350 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3946/7862] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:18:53.363312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.363605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.363883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.364522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:53.366932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:53.432120 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3917/7863] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:53 2022] POST /identity/v3/projects => generated 359 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:53.433665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e07c8-36f9-4daf-ad0b-62696467b318 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:53.458085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e07c8-36f9-4daf-ad0b-62696467b318 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:53.554871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:53.555771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.556544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.556544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.556961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:53.559797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:53.606121 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3947/7864] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:53 2022] POST /identity/v3/users => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:53.610421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:53.624795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a499ef-315c-4a63-8db7-bf3058bacd01 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:53.631586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:53.632818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.633366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.633979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/0081097c43a54ed289aa1a4bfb29dd26` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.653602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:get_user(user_id=0081097c43a54ed289aa1a4bfb29dd26)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:53.657455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:53.659992 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3918/7865] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:18:53 2022] GET /identity/v3/users/0081097c43a54ed289aa1a4bfb29dd26 => generated 373 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:18:53.664179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:53.683587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b736-8c62-4476-91de-0a2ebf9458ad tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:53.689700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:53.690576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.690917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.691204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/0081097c43a54ed289aa1a4bfb29dd26` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.693061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0081097c43a54ed289aa1a4bfb29dd26)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:53.695151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:53.775858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0081097c43a54ed289aa1a4bfb29dd26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:53.808330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0081097c43a54ed289aa1a4bfb29dd26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:53.827563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0081097c43a54ed289aa1a4bfb29dd26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:53.843601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0081097c43a54ed289aa1a4bfb29dd26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:53.844752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0081097c43a54ed289aa1a4bfb29dd26'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:53.867931 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3948/7866] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:53 2022] DELETE /identity/v3/users/0081097c43a54ed289aa1a4bfb29dd26 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:53.871738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:53.873934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:53.895762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d1b86-db57-4449-9493-e10847ae38d0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:53.946809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:53.987947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:53.988701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:53.988970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:53.989186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/c9b9eb78c1c3400d80a66eec3153da2f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:53.991407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c9b9eb78c1c3400d80a66eec3153da2f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:53.993404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.080532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b9eb78c1c3400d80a66eec3153da2f', 'request_id': 'req-2c440eab-0dee-4b22-a321-e4aa194c7700'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.089117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b9eb78c1c3400d80a66eec3153da2f', 'request_id': 'req-2c440eab-0dee-4b22-a321-e4aa194c7700'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.100269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b9eb78c1c3400d80a66eec3153da2f', 'request_id': 'req-2c440eab-0dee-4b22-a321-e4aa194c7700'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.118980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] The token cache is being invalidate because project c9b9eb78c1c3400d80a66eec3153da2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:54.119330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.120139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.122438 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3919/7867] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:53 2022] DELETE /identity/v3/projects/c9b9eb78c1c3400d80a66eec3153da2f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:54.127127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.144073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ba24b-db61-4eb0-bf4e-5f22998e11b8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.163192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:54.228068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.228689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.229028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.229317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.230144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.232546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.288760 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3949/7868] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:54 2022] POST /identity/v3/projects => generated 360 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:54.293180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.295746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:54.321965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c440eab-0dee-4b22-a321-e4aa194c7700 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.407688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.408545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.408909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.409259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.410583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.413071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.452999 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3920/7869] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:54 2022] POST /identity/v3/users => generated 417 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:54.457409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.470647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d1cd5-17ce-4bd4-8e97-475df1f79222 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.478246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.480547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:54.483581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.485183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.485695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.486344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.487310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.489637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.517647 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3950/7870] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:18:54 2022] POST /identity/v3/roles => generated 239 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:54.522661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.539614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1bcb0-b2b3-4c7a-bb66-9004dfef16f2 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.547537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.548621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.549076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.549562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/96d04dd7735141dc875fb9cf8202be4a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.552923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:54.574411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df901bfa-0246-4d63-b15e-07793644670a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.582933 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.584285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.584902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.586000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] PATH_INFO: `/v3/users/4e22d63d90b843fdb6ede77b568414d2` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.586585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:get_user(user_id=96d04dd7735141dc875fb9cf8202be4a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.588309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e22d63d90b843fdb6ede77b568414d2)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.589092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.591122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.591953 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3951/7871] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:18:54 2022] GET /identity/v3/users/96d04dd7735141dc875fb9cf8202be4a => generated 417 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:54.609068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.631113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2daab6-9c27-4a0e-9a5b-b43d1f5755d8 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.642005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.642891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.643352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.643895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/roles/2da64f989ea4430a887716eb10931178` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.648874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:get_role(role_id=2da64f989ea4430a887716eb10931178)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.650243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.653528 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3952/7872] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:18:54 2022] GET /identity/v3/roles/2da64f989ea4430a887716eb10931178 => generated 239 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:54.658069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.670829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa4229-bcfe-4e4f-aaa3-0ef65e71e0d3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.671288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e22d63d90b843fdb6ede77b568414d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.679932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.680441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.680779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.681059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.681720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.684446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.685021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e22d63d90b843fdb6ede77b568414d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.685780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e22d63d90b843fdb6ede77b568414d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.701031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e22d63d90b843fdb6ede77b568414d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.702022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a4dc8277-66be-4d4d-9a93-28aa218ad862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e22d63d90b843fdb6ede77b568414d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:54.725583 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3953/7873] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:54 2022] POST /identity/v3/projects => generated 360 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:54.732299 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3921/7874] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:54 2022] DELETE /identity/v3/users/4e22d63d90b843fdb6ede77b568414d2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:54.732472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.734789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:54.737463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.737944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.738868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.763508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:54.765397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:54.856846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958669df-4b1b-42ff-beb5-1a55e6a0d270 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.883781 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3922/7875] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:54.894255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.896948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:54.923719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:54.936483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.937857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.937857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.937857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/f9cfe5e62da64d0193d64e0fb3ee91b9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.938962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:get_project(project_id=f9cfe5e62da64d0193d64e0fb3ee91b9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.941187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:54.944212 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3954/7876] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:18:54 2022] GET /identity/v3/projects/f9cfe5e62da64d0193d64e0fb3ee91b9 => generated 360 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:54.954017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:54.966885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46841ae-c83c-4eea-83dc-cac8875101a7 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:54.978886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:54.981020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:54.982170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:54.982594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/f9cfe5e62da64d0193d64e0fb3ee91b9/users/96d04dd7735141dc875fb9cf8202be4a/roles/2da64f989ea4430a887716eb10931178` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:54.987586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f9cfe5e62da64d0193d64e0fb3ee91b9, user_id=96d04dd7735141dc875fb9cf8202be4a, role_id=2da64f989ea4430a887716eb10931178)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:54.991163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:55.018112 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3955/7877] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:54 2022] PUT /identity/v3/projects/f9cfe5e62da64d0193d64e0fb3ee91b9/users/96d04dd7735141dc875fb9cf8202be4a/roles/2da64f989ea4430a887716eb10931178 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:55.020139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:55.058668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41603deb-738f-4a9a-acc0-d28e8012f526 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:55.119438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:55.164897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200b869-826f-48ff-bade-af8611823836 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:55.173193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:55.173193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:55.173472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:55.173681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:55.174435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:55.177310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:55.189361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:55.190415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:55.190959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:55.191976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:55.230084 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3956/7878] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:55 2022] POST /identity/v3/projects => generated 359 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:55.232770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:55.352277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904a495-a13d-466d-8826-eb00efd9e94f tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:55.372083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:55.374414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:55.374687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:55.374798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:55.374999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/077e1e53f954440f8fb0a1d79bf636f5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:55.376264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:55.378990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:get_project(project_id=077e1e53f954440f8fb0a1d79bf636f5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:55.384428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:55.391669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3957/7879] 10.222.0.33 () {58 vars in 1230 bytes} [Sat Sep 10 21:18:55 2022] GET /identity/v3/projects/077e1e53f954440f8fb0a1d79bf636f5 => generated 359 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:55.397020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:55.413099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d7d7-50a5-4c54-a542-3891d5ae9ca1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:55.421949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3923/7880] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:18:55.439739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:55.455477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:55.460790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:55.461214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/077e1e53f954440f8fb0a1d79bf636f5/users/96d04dd7735141dc875fb9cf8202be4a/roles/2da64f989ea4430a887716eb10931178` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:55.471134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_grant(project_id=077e1e53f954440f8fb0a1d79bf636f5, user_id=96d04dd7735141dc875fb9cf8202be4a, role_id=2da64f989ea4430a887716eb10931178)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:55.486187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:55.552982 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3958/7881] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:55 2022] PUT /identity/v3/projects/077e1e53f954440f8fb0a1d79bf636f5/users/96d04dd7735141dc875fb9cf8202be4a/roles/2da64f989ea4430a887716eb10931178 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:55.557291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:55.573132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916a833-9345-45ce-9db1-dc98407223d4 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:55.668551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:55.670292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:55.670581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:55.670906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/96d04dd7735141dc875fb9cf8202be4a/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:55.672214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=96d04dd7735141dc875fb9cf8202be4a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:55.675508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:55.778009 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3924/7882] 10.222.0.33 () {58 vars in 1248 bytes} [Sat Sep 10 21:18:55 2022] GET /identity/v3/users/96d04dd7735141dc875fb9cf8202be4a/projects => generated 843 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:55.779361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:55.797021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa651-c1f0-457e-8585-bdde711807e4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:55.810965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:55.812345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:55.813671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:55.814244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/077e1e53f954440f8fb0a1d79bf636f5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:55.819209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_project(project_id=077e1e53f954440f8fb0a1d79bf636f5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:55.825524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:55.982822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077e1e53f954440f8fb0a1d79bf636f5', 'request_id': 'req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:55.991382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:56.025107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077e1e53f954440f8fb0a1d79bf636f5', 'request_id': 'req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.051066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077e1e53f954440f8fb0a1d79bf636f5', 'request_id': 'req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.099235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] The token cache is being invalidate because project 077e1e53f954440f8fb0a1d79bf636f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:56.099478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.100735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.108625 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3959/7883] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:55 2022] DELETE /identity/v3/projects/077e1e53f954440f8fb0a1d79bf636f5 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:56.109990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:56.111977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:56.128241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4aca3-3492-498e-bede-b7982194c526 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:56.214407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:56.244894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:56.245588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:56.245903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:56.246185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/f9cfe5e62da64d0193d64e0fb3ee91b9` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:56.248151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f9cfe5e62da64d0193d64e0fb3ee91b9)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:56.250151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:56.351374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cfe5e62da64d0193d64e0fb3ee91b9', 'request_id': 'req-32b48607-693d-4176-91f6-02f67ddb6ee4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.363729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cfe5e62da64d0193d64e0fb3ee91b9', 'request_id': 'req-32b48607-693d-4176-91f6-02f67ddb6ee4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.378331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cfe5e62da64d0193d64e0fb3ee91b9', 'request_id': 'req-32b48607-693d-4176-91f6-02f67ddb6ee4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.420096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] The token cache is being invalidate because project f9cfe5e62da64d0193d64e0fb3ee91b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:56.421643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.423056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.426414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3925/7884] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:56 2022] DELETE /identity/v3/projects/f9cfe5e62da64d0193d64e0fb3ee91b9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:56.430738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:56.447488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d67c-c359-45a6-84c6-ee85e7fa9c2e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:56.538307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:56.539069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:56.539361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:56.539636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/roles/2da64f989ea4430a887716eb10931178` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:56.541265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2da64f989ea4430a887716eb10931178)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:56.542907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:56.589277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invalidating the token cache because role 2da64f989ea4430a887716eb10931178 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:56.590031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.590964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.598381 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3960/7885] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:56 2022] DELETE /identity/v3/roles/2da64f989ea4430a887716eb10931178 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:56.599393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:56.600873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:56.654250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b48607-693d-4176-91f6-02f67ddb6ee4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:56.739511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:56.740379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:56.740764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:56.741624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/96d04dd7735141dc875fb9cf8202be4a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:56.743199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_user(user_id=96d04dd7735141dc875fb9cf8202be4a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:56.745273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:56.805576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d04dd7735141dc875fb9cf8202be4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.819719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d04dd7735141dc875fb9cf8202be4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.820922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d04dd7735141dc875fb9cf8202be4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.839717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d04dd7735141dc875fb9cf8202be4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.840774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d04dd7735141dc875fb9cf8202be4a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:56.861717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:56.874789 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3926/7886] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:56 2022] DELETE /identity/v3/users/96d04dd7735141dc875fb9cf8202be4a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:56.877455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:56.882052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:56.907367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a059233-ab26-429e-ad03-3538b045e0a9 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:56.979032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:56.983092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:57.023365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:57.024426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:57.024922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:57.025422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/4e146af997fd4e5c92af002ef8bbf301` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:57.027050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4e146af997fd4e5c92af002ef8bbf301)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:57.030378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:57.111203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67bd40-f3d8-4270-9bc9-3acef01d4677 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:57.120150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:57.120806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:57.121086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:57.121363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] PATH_INFO: `/v3/projects/4a040022cdce4aaa855fbd8e1211e762` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:57.123666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a040022cdce4aaa855fbd8e1211e762)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:57.128170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:57.150252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:57.152796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e146af997fd4e5c92af002ef8bbf301', 'request_id': 'req-3732b443-c9c2-4788-a1f0-7de3abaa74f6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.173378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e146af997fd4e5c92af002ef8bbf301', 'request_id': 'req-3732b443-c9c2-4788-a1f0-7de3abaa74f6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.184739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e146af997fd4e5c92af002ef8bbf301', 'request_id': 'req-3732b443-c9c2-4788-a1f0-7de3abaa74f6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.213114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] The token cache is being invalidate because project 4e146af997fd4e5c92af002ef8bbf301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:57.213350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.214157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.220886 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3961/7887] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:56 2022] DELETE /identity/v3/projects/4e146af997fd4e5c92af002ef8bbf301 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:18:57.222229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:57.223390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a040022cdce4aaa855fbd8e1211e762', 'request_id': 'req-5bbd5388-c692-443e-b70c-3db115783bff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.223908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:57.233909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a040022cdce4aaa855fbd8e1211e762', 'request_id': 'req-5bbd5388-c692-443e-b70c-3db115783bff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.250585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a040022cdce4aaa855fbd8e1211e762', 'request_id': 'req-5bbd5388-c692-443e-b70c-3db115783bff'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.293514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] The token cache is being invalidate because project 4a040022cdce4aaa855fbd8e1211e762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:57.293888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.295075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.297146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3927/7888] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:56 2022] DELETE /identity/v3/projects/4a040022cdce4aaa855fbd8e1211e762 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:57.301843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:57.303779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:57.376593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732b443-c9c2-4788-a1f0-7de3abaa74f6 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:57.471652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:57.472704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:57.472869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:57.474047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/2e13289108c54f8c89c1bf43ab4b98ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:57.475668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd5388-c692-443e-b70c-3db115783bff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:57.479418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:update_user(user_id=2e13289108c54f8c89c1bf43ab4b98ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:57.482496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:57.489861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:57.490489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:57.490613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:57.490891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] PATH_INFO: `/v3/users/1a785e56b49741388e2c9e2593a9c4cb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:57.494575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a785e56b49741388e2c9e2593a9c4cb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:57.496961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:57.539708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.550296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invalidating the token cache because user 2e13289108c54f8c89c1bf43ab4b98ab was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:57.551548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:57.555198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.556200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6f29e49b-6696-47a8-835c-1a79c080ca74 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.559632 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3962/7889] 10.222.0.33 () {60 vars in 1243 bytes} [Sat Sep 10 21:18:57 2022] PATCH /identity/v3/users/2e13289108c54f8c89c1bf43ab4b98ab => generated 296 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:18:57.567933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:57.568752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:57.569058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:57.589564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a785e56b49741388e2c9e2593a9c4cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.602284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a785e56b49741388e2c9e2593a9c4cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.604129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a785e56b49741388e2c9e2593a9c4cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.616420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a785e56b49741388e2c9e2593a9c4cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.617240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] MFA Rules not processed for user `2e13289108c54f8c89c1bf43ab4b98ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:57.618050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a785e56b49741388e2c9e2593a9c4cb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.619516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:57.622543 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3963/7890] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:57 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:57.628480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:57.632298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:57.655847 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3928/7891] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:18:57 2022] DELETE /identity/v3/users/1a785e56b49741388e2c9e2593a9c4cb => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:57.659357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36ee68-7543-49a6-b8b1-7ba25f5778e5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:57.733692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:57.776715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:57.778351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:57.778808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:57.779213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/2e13289108c54f8c89c1bf43ab4b98ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:57.781052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:update_user(user_id=2e13289108c54f8c89c1bf43ab4b98ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:57.784173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:57.853365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.870025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invalidating the token cache because user 2e13289108c54f8c89c1bf43ab4b98ab was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:57.870479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.872065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9fa075d-a7f2-434e-b5a8-bf8c74f7acf4 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:57.883057 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3964/7892] 10.222.0.33 () {60 vars in 1243 bytes} [Sat Sep 10 21:18:57 2022] PATCH /identity/v3/users/2e13289108c54f8c89c1bf43ab4b98ab => generated 296 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:57.886629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:57.906604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662267-651e-4a7b-bac2-c985ca89f45b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:57.916769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:57.917525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:57.918162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:57.918523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/2e13289108c54f8c89c1bf43ab4b98ab` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:57.986399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:update_user(user_id=2e13289108c54f8c89c1bf43ab4b98ab)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:57.987627 ncdv-master-1 devstack@keystone.service[93528]: WARNING py.warnings [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:18:57.987627 ncdv-master-1 devstack@keystone.service[93528]: warnings.warn(msg) Sep 10 21:18:57.987627 ncdv-master-1 devstack@keystone.service[93528]: Sep 10 21:18:57.998126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:58.059085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.072190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invalidating the token cache because user 2e13289108c54f8c89c1bf43ab4b98ab was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:58.072809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.073776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.080284 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3929/7893] 10.222.0.33 () {60 vars in 1243 bytes} [Sat Sep 10 21:18:57 2022] PATCH /identity/v3/users/2e13289108c54f8c89c1bf43ab4b98ab => generated 296 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:58.081226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.081696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.082668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.145188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:58.146371 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] MFA Rules not processed for user `2e13289108c54f8c89c1bf43ab4b98ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:58.148360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:58.156091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:58.168582 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3965/7894] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:58 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:58.173951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:58.176654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:58.202133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5375d-60d2-4eba-936b-742504ec710f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:58.213176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:58.215500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.215500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.215755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/2e13289108c54f8c89c1bf43ab4b98ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.218612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:update_user(user_id=2e13289108c54f8c89c1bf43ab4b98ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:58.225111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:58.277028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.305052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invalidating the token cache because user 2e13289108c54f8c89c1bf43ab4b98ab was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:58.305052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.305879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.311438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3966/7895] 10.222.0.33 () {60 vars in 1243 bytes} [Sat Sep 10 21:18:58 2022] PATCH /identity/v3/users/2e13289108c54f8c89c1bf43ab4b98ab => generated 296 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:58.314611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:58.322983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:58.328808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb38e0-98d9-48fb-b252-6579e9958a00 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:58.339626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:58.340891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.340891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.340891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] PATH_INFO: `/v3/projects/113b912cf2664f5c8a8f5b9504621401` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.345287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=113b912cf2664f5c8a8f5b9504621401)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:58.348218 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:58.350929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c9985-d5cc-433f-809e-1be7c2c805a0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:58.373043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:58.376043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.376630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.379092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.380106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:58.386622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:58.461782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '113b912cf2664f5c8a8f5b9504621401', 'request_id': 'req-cacee51b-873c-48c1-a27c-e533ac75b9bf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.472547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '113b912cf2664f5c8a8f5b9504621401', 'request_id': 'req-cacee51b-873c-48c1-a27c-e533ac75b9bf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.474124 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3967/7896] 10.222.0.33 () {60 vars in 1143 bytes} [Sat Sep 10 21:18:58 2022] POST /identity/v3/users => generated 265 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:18:58.480024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:58.485497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '113b912cf2664f5c8a8f5b9504621401', 'request_id': 'req-cacee51b-873c-48c1-a27c-e533ac75b9bf'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.493256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=13,keystone:INSERT=4,keystone:UPDATE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:58.506916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] The token cache is being invalidate because project 113b912cf2664f5c8a8f5b9504621401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:18:58.507849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.508832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-cacee51b-873c-48c1-a27c-e533ac75b9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.511111 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3930/7897] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:18:58 2022] DELETE /identity/v3/projects/113b912cf2664f5c8a8f5b9504621401 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:58.512598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf34184-87f9-4ebb-ad5b-fb5717f419b5 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:58.516520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d7d42-fb26-4368-a697-2bc33572b5eb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.517235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d7d42-fb26-4368-a697-2bc33572b5eb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.518262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d7d42-fb26-4368-a697-2bc33572b5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.545515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7c4d7d42-fb26-4368-a697-2bc33572b5eb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:58.547630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7c4d7d42-fb26-4368-a697-2bc33572b5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:58.671140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e13845-229e-4368-a2c2-5f0fd7771d0a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:58.671828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e13845-229e-4368-a2c2-5f0fd7771d0a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.675567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e13845-229e-4368-a2c2-5f0fd7771d0a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.676501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e13845-229e-4368-a2c2-5f0fd7771d0a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/e7ade6f4998d4a9dac730e28c38ed3b6/password` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.686018 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3931/7898] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:58.687857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2df0f-8142-41ed-980a-298aea836a97 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.689011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2df0f-8142-41ed-980a-298aea836a97 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.689365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2df0f-8142-41ed-980a-298aea836a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.736147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7ee2df0f-8142-41ed-980a-298aea836a97 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:58.737606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7ee2df0f-8142-41ed-980a-298aea836a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:58.748833 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3932/7899] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:58.753762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2df0f-8142-41ed-980a-298aea836a97 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:58.779591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2df0f-8142-41ed-980a-298aea836a97 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:58.791903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:58.792445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.792753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.793555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.794870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:18:58.796847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:58.798562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:58.840678 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3933/7900] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:18:58.845140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:58.866321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d7e13845-229e-4368-a2c2-5f0fd7771d0a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e7ade6f4998d4a9dac730e28c38ed3b6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:18:58.877796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6c235-2af8-410f-8ab0-0e93d9227fc8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:58.889424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:58.890113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:58.890378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:58.890659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:58.891369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:58.892201 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3968/7901] 10.222.0.33 () {60 vars in 1269 bytes} [Sat Sep 10 21:18:58 2022] POST /identity/v3/users/e7ade6f4998d4a9dac730e28c38ed3b6/password => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:58.897425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:58.953475 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3934/7902] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:18:58 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:58.958457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e13845-229e-4368-a2c2-5f0fd7771d0a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:58.977134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e13845-229e-4368-a2c2-5f0fd7771d0a tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:58.981290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:UPDATE=3,keystone:INSERT=9,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:18:59.076948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:59.077853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:59.078055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:59.078335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:59.079002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:59.081291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:59.108098 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3969/7903] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:18:58 2022] POST /identity/v3/users => generated 279 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:18:59.109241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:59.120572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f176fa-b115-46f7-832c-70b4d78fcb49 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:59.127461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:59.128364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:59.130509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:59.130823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:59.131505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:59.133661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:59.149219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3935/7904] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:18:59.150517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:59.162438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d91bcc-781d-4197-85a7-4bbeb53dae5a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:59.167933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082c2a-f882-4b2f-89ea-ceda6818caf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:59.168853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082c2a-f882-4b2f-89ea-ceda6818caf8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:59.168853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082c2a-f882-4b2f-89ea-ceda6818caf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:59.169156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082c2a-f882-4b2f-89ea-ceda6818caf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:59.169804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082c2a-f882-4b2f-89ea-ceda6818caf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:59.171756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082c2a-f882-4b2f-89ea-ceda6818caf8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:59.185734 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3970/7905] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:18:59.189803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:59.202152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3774b-923e-4f79-a3fd-423dd262ac28 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:59.208424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:59.209089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:59.209863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:59.209863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] PATH_INFO: `/v3/projects/584e0ad9edd24d67a68ecd56f3b7e0d6/users/b8d0a762edaf45f89684220812587fd4/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:59.227668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=584e0ad9edd24d67a68ecd56f3b7e0d6, user_id=b8d0a762edaf45f89684220812587fd4, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:59.231449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:59.255344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3936/7906] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:18:59 2022] PUT /identity/v3/projects/584e0ad9edd24d67a68ecd56f3b7e0d6/users/b8d0a762edaf45f89684220812587fd4/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:18:59.266156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf8a9-0766-4abd-a3f0-ac6fc5e24b60 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:59.266459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf8a9-0766-4abd-a3f0-ac6fc5e24b60 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:59.266687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf8a9-0766-4abd-a3f0-ac6fc5e24b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:59.302734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-773cf8a9-0766-4abd-a3f0-ac6fc5e24b60 None None] MFA Rules not processed for user `b8d0a762edaf45f89684220812587fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:59.305329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-773cf8a9-0766-4abd-a3f0-ac6fc5e24b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:59.386205 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3971/7907] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:18:59.404471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:59.406659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:59.483602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43290a5-4cfd-4752-8853-49a5ce04acd8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:18:59.488055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:18:59.488695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:59.489014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:59.489225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:59.500518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:18:59.501998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:18:59.512497 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3937/7908] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:18:59.897360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9720c-b7eb-4b1e-817f-4eb685503b72 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:18:59.897891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9720c-b7eb-4b1e-817f-4eb685503b72 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:18:59.898414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9720c-b7eb-4b1e-817f-4eb685503b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:18:59.944364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7db9720c-b7eb-4b1e-817f-4eb685503b72 None None] MFA Rules not processed for user `e7ade6f4998d4a9dac730e28c38ed3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:18:59.945862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7db9720c-b7eb-4b1e-817f-4eb685503b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:59.952033 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3972/7909] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:18:59 2022] POST /identity/v3/auth/tokens => generated 330 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 21:18:59.956438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:18:59.958920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:18:59.995764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:00.070607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910df320-4b04-4aef-b234-c1651bcb6396 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.076168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.076786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.077069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.077346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.089016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.091839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:00.110536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9720c-b7eb-4b1e-817f-4eb685503b72 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:00.121882 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3938/7910] 10.222.0.33 () {56 vars in 1263 bytes} [Sat Sep 10 21:18:59 2022] GET /identity/v3/auth/tokens => generated 330 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 21:19:00.131493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9720c-b7eb-4b1e-817f-4eb685503b72 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.144579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.145345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.145703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.146091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/e7ade6f4998d4a9dac730e28c38ed3b6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.147466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_user(user_id=e7ade6f4998d4a9dac730e28c38ed3b6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.149432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:00.217604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ade6f4998d4a9dac730e28c38ed3b6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.247452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:00.251166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ade6f4998d4a9dac730e28c38ed3b6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.274670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ade6f4998d4a9dac730e28c38ed3b6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.291794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ade6f4998d4a9dac730e28c38ed3b6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.293139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ade6f4998d4a9dac730e28c38ed3b6'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.306127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3973/7911] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:00 2022] DELETE /identity/v3/users/e7ade6f4998d4a9dac730e28c38ed3b6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:00.310878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:00.334655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadc31e-7da2-4837-a68f-bf97ab10e59c tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.414902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.415977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.416267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.416555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.417203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.420500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:00.485242 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3939/7912] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:00 2022] POST /identity/v3/users => generated 364 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:00.489664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:00.491677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:00.508550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4930bc0-55e8-43a6-8c37-221476aba986 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.514883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.516543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.516882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.517161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.518609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.522842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:00.580619 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3974/7913] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:00 2022] POST /identity/v3/projects => generated 360 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:00.583021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:00.594442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34649136-43dc-4501-865d-3f619688ac8b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.658864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.659620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `PATCH` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.660035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.660435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/05418e324589498fa3b22d8ccdce4d95` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.675430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:update_user(user_id=05418e324589498fa3b22d8ccdce4d95)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.678288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:00.717385 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3940/7914] 10.222.0.33 () {60 vars in 1244 bytes} [Sat Sep 10 21:19:00 2022] PATCH /identity/v3/users/05418e324589498fa3b22d8ccdce4d95 => generated 521 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:00.719048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:00.730427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dffca-a395-4355-9695-686467988f9b tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.735701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.736356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.736645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.736929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/05418e324589498fa3b22d8ccdce4d95` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.756216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:get_user(user_id=05418e324589498fa3b22d8ccdce4d95)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.758590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:00.760380 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3975/7915] 10.222.0.33 () {58 vars in 1221 bytes} [Sat Sep 10 21:19:00 2022] GET /identity/v3/users/05418e324589498fa3b22d8ccdce4d95 => generated 388 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:00.764696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:00.776093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82fc89-18e4-4ea8-91c1-950858ab34b1 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.781141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.781786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.782103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.782377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/projects/2e7b9766907a44e5af11402ea445a6b8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.783533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2e7b9766907a44e5af11402ea445a6b8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.785773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:00.796169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:00.842290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7b9766907a44e5af11402ea445a6b8', 'request_id': 'req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.850882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7b9766907a44e5af11402ea445a6b8', 'request_id': 'req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.859876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7b9766907a44e5af11402ea445a6b8', 'request_id': 'req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.872827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] The token cache is being invalidate because project 2e7b9766907a44e5af11402ea445a6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:00.873121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.873957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:00.875970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3941/7916] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:00 2022] DELETE /identity/v3/projects/2e7b9766907a44e5af11402ea445a6b8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:00.880735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:00.882804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:00.897278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783d5b0-d2ef-4ec6-a506-e0f2a6abb07e tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:00.990043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e4d7fa14359481cac6c2ca9a0ae853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58f5895f8b2f4cf585b86dda9ab4fd76', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:00.990932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:00.991280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:00.991621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] PATH_INFO: `/v3/users/05418e324589498fa3b22d8ccdce4d95` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:00.993009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorizing `identity:delete_user(user_id=05418e324589498fa3b22d8ccdce4d95)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:00.995282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:01.045176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05418e324589498fa3b22d8ccdce4d95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.061401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05418e324589498fa3b22d8ccdce4d95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.069687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05418e324589498fa3b22d8ccdce4d95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.077427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05418e324589498fa3b22d8ccdce4d95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.078317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-167df625-97fa-4da6-8f38-559172b25eb0 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05418e324589498fa3b22d8ccdce4d95'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.085150 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3976/7917] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:00 2022] DELETE /identity/v3/users/05418e324589498fa3b22d8ccdce4d95 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:01.090331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:01.092191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:01.168179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfea0f7-ef4f-4b7c-be61-c08ff6c8eaa3 tempest-UsersV3TestJSON-1629828486 tempest-UsersV3TestJSON-1629828486-project-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:01.230155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:01.230807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:01.231090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:01.231373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] PATH_INFO: `/v3/users/2e13289108c54f8c89c1bf43ab4b98ab` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:01.247300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e13289108c54f8c89c1bf43ab4b98ab)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:01.249256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:01.292825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:01.306382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.313661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.314398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.322527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.323273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e13289108c54f8c89c1bf43ab4b98ab'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:01.342314 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3942/7918] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:01 2022] DELETE /identity/v3/users/2e13289108c54f8c89c1bf43ab4b98ab => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:01.346980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:01.349751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:01.349751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:01.376333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:01.377575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:01.399856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:01.463767 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3977/7919] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:01.490047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:01.492330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:01.578514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb9925-f817-4ab9-bc0a-63bc8d03e229 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:01.583160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:01.583775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:01.584061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:01.584338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:01.594901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:01.596450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:01.607147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3943/7920] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:02.320374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:02.328223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:02.372355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c98f1-ceb9-4562-98b3-1fcb5221fe29 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:02.386197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:02.387128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:02.387872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:02.389772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] PATH_INFO: `/v3/projects/cab5bdf63abd4e8ca2b8d7b3c7b3fe7f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:02.392765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab5bdf63abd4e8ca2b8d7b3c7b3fe7f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:02.400526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:02.524465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab5bdf63abd4e8ca2b8d7b3c7b3fe7f', 'request_id': 'req-28fc7285-7e41-41d6-bfa3-6c60006f600b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.532622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab5bdf63abd4e8ca2b8d7b3c7b3fe7f', 'request_id': 'req-28fc7285-7e41-41d6-bfa3-6c60006f600b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.545458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab5bdf63abd4e8ca2b8d7b3c7b3fe7f', 'request_id': 'req-28fc7285-7e41-41d6-bfa3-6c60006f600b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.562978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] The token cache is being invalidate because project cab5bdf63abd4e8ca2b8d7b3c7b3fe7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:02.563277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.563972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.569188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3978/7921] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:02 2022] DELETE /identity/v3/projects/cab5bdf63abd4e8ca2b8d7b3c7b3fe7f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:02.570521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:02.584097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0f042-8262-4dea-8a4c-c876429eabc1 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:02.716526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:02.721736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:02.722900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:02.723713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] PATH_INFO: `/v3/users/8e4d7fa14359481cac6c2ca9a0ae853f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:02.727311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e4d7fa14359481cac6c2ca9a0ae853f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:02.734708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:02.887907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e4d7fa14359481cac6c2ca9a0ae853f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.914777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e4d7fa14359481cac6c2ca9a0ae853f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.916557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e4d7fa14359481cac6c2ca9a0ae853f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.919483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:02.944188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e4d7fa14359481cac6c2ca9a0ae853f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.945097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e4d7fa14359481cac6c2ca9a0ae853f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:02.989177 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3944/7922] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:02 2022] DELETE /identity/v3/users/8e4d7fa14359481cac6c2ca9a0ae853f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:03.001713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:03.003304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:03.042482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:03.045190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:03.148636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7285-7e41-41d6-bfa3-6c60006f600b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:03.252630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14e0ea-ff2b-4424-b323-7b76ae5aa19a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:03.276362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e019d-4fd6-41a5-8c9c-4b9b36bf03c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:03.277015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e019d-4fd6-41a5-8c9c-4b9b36bf03c6 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:03.277299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e019d-4fd6-41a5-8c9c-4b9b36bf03c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:03.277574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e019d-4fd6-41a5-8c9c-4b9b36bf03c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:03.281373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-be3e019d-4fd6-41a5-8c9c-4b9b36bf03c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:03.308754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:03.312509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:03.312509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:03.312509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:03.313988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:03.336711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:03.338651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:03.356212 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3979/7923] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:03.419493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e019d-4fd6-41a5-8c9c-4b9b36bf03c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:03.421173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e019d-4fd6-41a5-8c9c-4b9b36bf03c6 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:03.455243 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3945/7924] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:03.733324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:03.734813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:03.741976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:03.881733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327866e-a96f-418c-954e-dd346e34b1dd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:03.890578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:03.891968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:03.892412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:03.893322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] PATH_INFO: `/v3/projects/58f5895f8b2f4cf585b86dda9ab4fd76` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:03.895817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f5895f8b2f4cf585b86dda9ab4fd76)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:03.898902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:03.985290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f5895f8b2f4cf585b86dda9ab4fd76', 'request_id': 'req-81e04b59-0eaa-4bac-b19d-eed459c8817c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:03.995152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f5895f8b2f4cf585b86dda9ab4fd76', 'request_id': 'req-81e04b59-0eaa-4bac-b19d-eed459c8817c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:04.014895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f5895f8b2f4cf585b86dda9ab4fd76', 'request_id': 'req-81e04b59-0eaa-4bac-b19d-eed459c8817c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:04.040935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] The token cache is being invalidate because project 58f5895f8b2f4cf585b86dda9ab4fd76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:04.041579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:04.043369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-81e04b59-0eaa-4bac-b19d-eed459c8817c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:04.045906 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3980/7925] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:03 2022] DELETE /identity/v3/projects/58f5895f8b2f4cf585b86dda9ab4fd76 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:04.057555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cdded-4635-4e49-99bf-322f8eeb564d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:04.058142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cdded-4635-4e49-99bf-322f8eeb564d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:04.058443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cdded-4635-4e49-99bf-322f8eeb564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:04.118213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-2b5cdded-4635-4e49-99bf-322f8eeb564d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:04.119905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2b5cdded-4635-4e49-99bf-322f8eeb564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:04.270510 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3946/7926] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:04.278843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fbcaa-4d7f-4cd2-8664-b1d5c7b9e874 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:04.278843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fbcaa-4d7f-4cd2-8664-b1d5c7b9e874 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:04.278843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fbcaa-4d7f-4cd2-8664-b1d5c7b9e874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:04.319081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fa1fbcaa-4d7f-4cd2-8664-b1d5c7b9e874 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:04.319830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fa1fbcaa-4d7f-4cd2-8664-b1d5c7b9e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:04.339791 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3981/7927] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:04.345774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cdded-4635-4e49-99bf-322f8eeb564d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:04.364091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cdded-4635-4e49-99bf-322f8eeb564d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:04.369996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:04.370726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:04.371080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:04.371464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:04.372137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:04.372443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:04.376180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:04.398687 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3947/7928] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:04.399837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1fbcaa-4d7f-4cd2-8664-b1d5c7b9e874 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:04.413541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1fbcaa-4d7f-4cd2-8664-b1d5c7b9e874 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:04.419860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:04.420774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:04.421285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:04.421712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:04.422553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:04.424662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:04.492011 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3982/7929] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:04 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:04.496479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:04.515184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6f920-ff6c-488d-9d0e-91aaf7079b39 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:04.551865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:04.723777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:04.724671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:04.725116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:04.728095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:04.728963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:04.732134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:04.799218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3948/7930] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:04 2022] POST /identity/v3/users => generated 300 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:04.805013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:04.833907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99463b55-fc95-4c8f-b713-420edfc76ce7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:04.849016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:04.849780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:04.850205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:04.850486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:04.853316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:04.856457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:04.883264 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3983/7931] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:04.893464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:04.941989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765279d-87b8-4676-829d-121d5004d954 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:04.953424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:04.955169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:04.958526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:04.959112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] PATH_INFO: `/v3/projects/71346055b04f4dde9ba3b33ccc19c10d/users/7783af337df449da866e37af9aa0ab2b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.016022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71346055b04f4dde9ba3b33ccc19c10d, user_id=7783af337df449da866e37af9aa0ab2b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:05.020400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:05.051671 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3949/7932] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:04 2022] PUT /identity/v3/projects/71346055b04f4dde9ba3b33ccc19c10d/users/7783af337df449da866e37af9aa0ab2b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:05.056319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:05.075491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f857-48a3-4c5e-bbdd-b6365d62c3b0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:05.195899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:05.210770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8be91d-d86a-4c4a-bc07-a34ada0042f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:05.211155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8be91d-d86a-4c4a-bc07-a34ada0042f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:05.211414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8be91d-d86a-4c4a-bc07-a34ada0042f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:05.211703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8be91d-d86a-4c4a-bc07-a34ada0042f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.212361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8be91d-d86a-4c4a-bc07-a34ada0042f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:05.214065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8be91d-d86a-4c4a-bc07-a34ada0042f1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:05.237585 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3984/7933] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:05.242216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:05.256354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e1822-ebf9-4fc4-964c-917f3672bce5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:05.261629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:05.262446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:05.262927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:05.263100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] PATH_INFO: `/v3/projects/71346055b04f4dde9ba3b33ccc19c10d/users/7783af337df449da866e37af9aa0ab2b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.265337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71346055b04f4dde9ba3b33ccc19c10d, user_id=7783af337df449da866e37af9aa0ab2b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:05.272267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:05.302474 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3950/7934] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:05 2022] PUT /identity/v3/projects/71346055b04f4dde9ba3b33ccc19c10d/users/7783af337df449da866e37af9aa0ab2b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:05.303526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23db3-946b-4b2c-b92a-6c25c7c0c21b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:05.303781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23db3-946b-4b2c-b92a-6c25c7c0c21b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:05.304063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23db3-946b-4b2c-b92a-6c25c7c0c21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.367969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4cf23db3-946b-4b2c-b92a-6c25c7c0c21b None None] MFA Rules not processed for user `7783af337df449da866e37af9aa0ab2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:05.369236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4cf23db3-946b-4b2c-b92a-6c25c7c0c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:05.531055 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3985/7935] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:05.539158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:05.617468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf147ed3-8a50-4f49-ac04-3e6bd0a19a8d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:05.621702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:05.622437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:05.622792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:05.623134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.623845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:05.626058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:05.663269 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3951/7936] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:05 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:05.664145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23db3-946b-4b2c-b92a-6c25c7c0c21b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:05.679681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23db3-946b-4b2c-b92a-6c25c7c0c21b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:05.815866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:05.816570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:05.816854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:05.817133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.817765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:05.819747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:05.876435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3986/7937] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:05 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:05.881029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:05.911032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721a48f-6a0a-4773-ad9a-cc278b849360 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:05.927355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:05.928237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:05.928705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:05.929127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.929960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:05.931755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:05.953769 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3952/7938] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:05.955715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:05.974065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec244956-9195-4096-90c4-115a43afc645 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:05.983585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4df17-7631-472e-a7d2-2c0ecf3b106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:05.984208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4df17-7631-472e-a7d2-2c0ecf3b106d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:05.984494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4df17-7631-472e-a7d2-2c0ecf3b106d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:05.984774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4df17-7631-472e-a7d2-2c0ecf3b106d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:05.985422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4df17-7631-472e-a7d2-2c0ecf3b106d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:05.987244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4df17-7631-472e-a7d2-2c0ecf3b106d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:06.002118 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3987/7939] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:06.012562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:06.027646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4726-2080-4837-80bd-96f8c991f189 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:06.034299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:06.036067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:06.036487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:06.036847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] PATH_INFO: `/v3/projects/887758e19dd140bda1329efd0637c393/users/b0f08fb7fb0c4bbbae7a5bf9fad858be/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:06.060598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=887758e19dd140bda1329efd0637c393, user_id=b0f08fb7fb0c4bbbae7a5bf9fad858be, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:06.064583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:06.091606 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3953/7940] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:06 2022] PUT /identity/v3/projects/887758e19dd140bda1329efd0637c393/users/b0f08fb7fb0c4bbbae7a5bf9fad858be/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:06.104421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f567591-7ce4-4a7c-b603-48747014b598 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:06.104700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f567591-7ce4-4a7c-b603-48747014b598 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:06.104975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f567591-7ce4-4a7c-b603-48747014b598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:06.143241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3f567591-7ce4-4a7c-b603-48747014b598 None None] MFA Rules not processed for user `b0f08fb7fb0c4bbbae7a5bf9fad858be`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:06.145437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3f567591-7ce4-4a7c-b603-48747014b598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:06.238274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:06.267084 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3988/7941] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:06.283374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:06.288877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:06.312569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:06.438755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52593b0c-95c6-49fb-a345-120b8145cdcc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:06.444532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fcfef-e84d-4a14-b9f5-c1f66baf7161 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:06.445499 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fcfef-e84d-4a14-b9f5-c1f66baf7161 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:06.449456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fcfef-e84d-4a14-b9f5-c1f66baf7161 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:06.450488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fcfef-e84d-4a14-b9f5-c1f66baf7161 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:06.630765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fcfef-e84d-4a14-b9f5-c1f66baf7161 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:06.632411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fcfef-e84d-4a14-b9f5-c1f66baf7161 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:06.655656 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3954/7942] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:07.150197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f567591-7ce4-4a7c-b603-48747014b598 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:07.241037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f567591-7ce4-4a7c-b603-48747014b598 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:07.245814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:07.246477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:07.246916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:07.247200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] PATH_INFO: `/v3/users/7783af337df449da866e37af9aa0ab2b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:07.248460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7783af337df449da866e37af9aa0ab2b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:07.250594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:07.347258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7783af337df449da866e37af9aa0ab2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:07.372168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7783af337df449da866e37af9aa0ab2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:07.385632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7783af337df449da866e37af9aa0ab2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:07.397763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7783af337df449da866e37af9aa0ab2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:07.398751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7783af337df449da866e37af9aa0ab2b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:07.409414 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3989/7943] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:07 2022] DELETE /identity/v3/users/7783af337df449da866e37af9aa0ab2b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:07.411497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:07.411981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:07.412106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:07.438213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:07.439592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:07.535123 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3955/7944] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:07.552066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:07.553683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:07.639089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b7dc4-067c-4972-b0f3-a0cba845894f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:07.648276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:07.648910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:07.667481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:07.667481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:07.682597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:07.684135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:07.695521 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3990/7945] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:08.131713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:08.133162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:08.165023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe9d5a-c8fd-4768-8810-ef98bcef91bc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:08.173714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:08.174616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:08.174782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:08.175050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] PATH_INFO: `/v3/projects/71346055b04f4dde9ba3b33ccc19c10d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:08.176650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71346055b04f4dde9ba3b33ccc19c10d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:08.179641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:08.247621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71346055b04f4dde9ba3b33ccc19c10d', 'request_id': 'req-a275f309-471e-4062-80d2-70731782b6b8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.248744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:08.255438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71346055b04f4dde9ba3b33ccc19c10d', 'request_id': 'req-a275f309-471e-4062-80d2-70731782b6b8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.264788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71346055b04f4dde9ba3b33ccc19c10d', 'request_id': 'req-a275f309-471e-4062-80d2-70731782b6b8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.277995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] The token cache is being invalidate because project 71346055b04f4dde9ba3b33ccc19c10d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:08.278374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.279010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.280838 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3956/7946] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:08 2022] DELETE /identity/v3/projects/71346055b04f4dde9ba3b33ccc19c10d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:08.284826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:08.296257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdd16b-47ab-4359-b7af-8f44b1d351a3 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:08.302808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:08.375439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:08.376007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:08.376310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:08.376791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] PATH_INFO: `/v3/users/b0f08fb7fb0c4bbbae7a5bf9fad858be` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:08.377989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f08fb7fb0c4bbbae7a5bf9fad858be)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:08.380613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:08.436799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f08fb7fb0c4bbbae7a5bf9fad858be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.455357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f08fb7fb0c4bbbae7a5bf9fad858be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.466116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f08fb7fb0c4bbbae7a5bf9fad858be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.477327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f08fb7fb0c4bbbae7a5bf9fad858be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.478569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f08fb7fb0c4bbbae7a5bf9fad858be'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:08.492237 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3991/7947] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:08 2022] DELETE /identity/v3/users/b0f08fb7fb0c4bbbae7a5bf9fad858be => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:09.084539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.087492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:09.123375 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.125388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:09.228334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275f309-471e-4062-80d2-70731782b6b8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.249638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16202a7-4c56-48a7-bd58-50a035592864 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.259707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.260416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.260713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.260993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] PATH_INFO: `/v3/projects/887758e19dd140bda1329efd0637c393` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.262333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] RBAC: Authorizing `identity:delete_project(project_id=887758e19dd140bda1329efd0637c393)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.264428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.305511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:09.328785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887758e19dd140bda1329efd0637c393', 'request_id': 'req-d33711c7-8e17-4709-9ba8-863e2abe936d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:09.335432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887758e19dd140bda1329efd0637c393', 'request_id': 'req-d33711c7-8e17-4709-9ba8-863e2abe936d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:09.342983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887758e19dd140bda1329efd0637c393', 'request_id': 'req-d33711c7-8e17-4709-9ba8-863e2abe936d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:09.348925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86f602-3287-42f2-b4df-c772af447b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.353487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f602-3287-42f2-b4df-c772af447b73 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.353804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f602-3287-42f2-b4df-c772af447b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.354121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86f602-3287-42f2-b4df-c772af447b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.359012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af86f602-3287-42f2-b4df-c772af447b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:09.362196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] The token cache is being invalidate because project 887758e19dd140bda1329efd0637c393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:09.362592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:09.364022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d33711c7-8e17-4709-9ba8-863e2abe936d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:09.366143 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3992/7948] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:09 2022] DELETE /identity/v3/projects/887758e19dd140bda1329efd0637c393 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:09.370992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767fbf7-3919-4ce7-8127-7e101ce543d2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.371349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767fbf7-3919-4ce7-8127-7e101ce543d2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.371633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767fbf7-3919-4ce7-8127-7e101ce543d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.399803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b767fbf7-3919-4ce7-8127-7e101ce543d2 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:09.401677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b767fbf7-3919-4ce7-8127-7e101ce543d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:09.454326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86f602-3287-42f2-b4df-c772af447b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.455863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86f602-3287-42f2-b4df-c772af447b73 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.465856 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3957/7949] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:09.494160 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3993/7950] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:09.502726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac31099-5c1f-4aa6-b2ae-69476fc910f4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.502726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac31099-5c1f-4aa6-b2ae-69476fc910f4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.503036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac31099-5c1f-4aa6-b2ae-69476fc910f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.526666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4ac31099-5c1f-4aa6-b2ae-69476fc910f4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:09.527765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4ac31099-5c1f-4aa6-b2ae-69476fc910f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:09.536803 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3958/7951] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:09.540905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767fbf7-3919-4ce7-8127-7e101ce543d2 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.551477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767fbf7-3919-4ce7-8127-7e101ce543d2 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.557132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.557743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.558069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.558349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.559003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:09.559309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.561070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.579110 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3994/7952] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:09.580257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac31099-5c1f-4aa6-b2ae-69476fc910f4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.591434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac31099-5c1f-4aa6-b2ae-69476fc910f4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.596382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.597239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.597532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.597805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.598493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.601006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.636430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3959/7953] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:09 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:09.637676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.650594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d0d6-f9a1-4a2b-9810-1117fcd2530a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.725644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.726298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.726585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.726863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.727506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.729489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.762008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3995/7954] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:09 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:09.764439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.791906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce368d85-0b64-4fa8-85af-f8a78843bc6a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.802051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.802778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.803139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.803487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.804219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.806012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.819110 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3960/7955] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:09.822908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.834456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dd29d-183c-4294-9b43-33e14bc0a309 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.839698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb092d24-775b-404e-bb93-334d3566fcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.840407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb092d24-775b-404e-bb93-334d3566fcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.840765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb092d24-775b-404e-bb93-334d3566fcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.841113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb092d24-775b-404e-bb93-334d3566fcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.842086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb092d24-775b-404e-bb93-334d3566fcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.846866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb092d24-775b-404e-bb93-334d3566fcd3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.866493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3996/7956] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:19:09.868490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:09.889533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2c058-02c6-4eac-94d9-8aa0bad2c277 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:09.899495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:09.900519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.900855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.901134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] PATH_INFO: `/v3/projects/72d3d7ca3cb44c06b39a951eb198b8b3/users/b431d8b003ab4e35bbd2a382c6c49745/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:09.906053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:09.923756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d3d7ca3cb44c06b39a951eb198b8b3, user_id=b431d8b003ab4e35bbd2a382c6c49745, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:09.928714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:09.955441 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3961/7957] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:09 2022] PUT /identity/v3/projects/72d3d7ca3cb44c06b39a951eb198b8b3/users/b431d8b003ab4e35bbd2a382c6c49745/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:09.966612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aad99a-dbb5-425b-a5cc-b3016ee70d14 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:09.967056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aad99a-dbb5-425b-a5cc-b3016ee70d14 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:09.967466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aad99a-dbb5-425b-a5cc-b3016ee70d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.008366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d0aad99a-dbb5-425b-a5cc-b3016ee70d14 None None] MFA Rules not processed for user `b431d8b003ab4e35bbd2a382c6c49745`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:10.009723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d0aad99a-dbb5-425b-a5cc-b3016ee70d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:10.025913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:10.148108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3997/7958] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:10.152785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:10.271773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a01c0a-21ec-44a0-96ac-df30a2c262e7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:10.277061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:10.277766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:10.278283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:10.278634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.279358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:10.281564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:10.327999 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3962/7959] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:10 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:10.329068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aad99a-dbb5-425b-a5cc-b3016ee70d14 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:10.342532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aad99a-dbb5-425b-a5cc-b3016ee70d14 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:10.439989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:10.441220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:10.443249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:10.443249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.443937 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:10.448516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:10.513036 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3998/7960] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:10 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:10.516917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:10.529863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f7434-eb11-4381-b7bd-4ca407ac028f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:10.538701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:10.539458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:10.539813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:10.540164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.540883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:10.542816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:10.561915 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3963/7961] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:10.563872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:10.576524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7bb1-d285-42b4-bbc0-e0d087be801b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:10.581376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309a68b-c819-4d0f-9d66-363a3cd207ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:10.582757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309a68b-c819-4d0f-9d66-363a3cd207ab admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:10.583175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309a68b-c819-4d0f-9d66-363a3cd207ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:10.583516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309a68b-c819-4d0f-9d66-363a3cd207ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.584236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309a68b-c819-4d0f-9d66-363a3cd207ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:10.586055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309a68b-c819-4d0f-9d66-363a3cd207ab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:10.597951 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 3999/7962] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:10.603317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:10.614945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c434157-a955-4dcd-8376-79398d760c8b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:10.619792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:10.620541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:10.620900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:10.621328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] PATH_INFO: `/v3/projects/12844a973af949af999e6ea733f5da40/users/746f76909fd9426789759227f2690ee9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.638465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12844a973af949af999e6ea733f5da40, user_id=746f76909fd9426789759227f2690ee9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:10.641236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:10.658452 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3964/7963] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:10 2022] PUT /identity/v3/projects/12844a973af949af999e6ea733f5da40/users/746f76909fd9426789759227f2690ee9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:10.671260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bbe8e-bf96-47ed-bc1f-28e9ab0c2c4e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:10.671703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bbe8e-bf96-47ed-bc1f-28e9ab0c2c4e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:10.672055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bbe8e-bf96-47ed-bc1f-28e9ab0c2c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.720432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-691bbe8e-bf96-47ed-bc1f-28e9ab0c2c4e None None] MFA Rules not processed for user `746f76909fd9426789759227f2690ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:10.720432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-691bbe8e-bf96-47ed-bc1f-28e9ab0c2c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:10.791024 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4000/7964] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:10.807789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:10.809540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:10.888596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66916fe-c9ab-4301-aab2-a5cf8aae719e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:10.893504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:10.893504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:10.893783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:10.895401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:10.938755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:10.960823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:10.963448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:10.981009 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3965/7965] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:11.127815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:11.128286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:11.128744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:11.162647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:11.167989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:11.192450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:11.256419 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4001/7966] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:11.325114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:11.327442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:11.509919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3abdf-32a9-4b9d-9d96-5e96714617c3 service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:11.517468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf7db7-3f1f-4a11-84e0-cc730e9fb9f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:11.518133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf7db7-3f1f-4a11-84e0-cc730e9fb9f9 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:11.518514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf7db7-3f1f-4a11-84e0-cc730e9fb9f9 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:11.522229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf7db7-3f1f-4a11-84e0-cc730e9fb9f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:11.536290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf7db7-3f1f-4a11-84e0-cc730e9fb9f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:11.538607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf7db7-3f1f-4a11-84e0-cc730e9fb9f9 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:11.549956 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3966/7967] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:19:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:19:11.812605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:11.814190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:11.830420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f007b-da4e-4428-965f-b7919a0c8c18 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:11.835393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:11.836453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:11.836895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:11.837323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:11.850065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:11.852013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:11.870147 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4002/7968] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:11.925343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8df097-b35a-4d62-96bc-49f39339f857 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:11.925641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8df097-b35a-4d62-96bc-49f39339f857 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:11.925947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8df097-b35a-4d62-96bc-49f39339f857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:11.946350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ff8df097-b35a-4d62-96bc-49f39339f857 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:11.947737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ff8df097-b35a-4d62-96bc-49f39339f857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:11.955951 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3967/7969] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:11.975967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:11.997412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efbd7c-949b-4fa6-b9a3-0928be88680f service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:12.003789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eafda-b6a5-4dc5-9fbe-0f2f5a489f57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:12.004682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eafda-b6a5-4dc5-9fbe-0f2f5a489f57 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:12.005039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eafda-b6a5-4dc5-9fbe-0f2f5a489f57 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:12.005388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eafda-b6a5-4dc5-9fbe-0f2f5a489f57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:12.015154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3eafda-b6a5-4dc5-9fbe-0f2f5a489f57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:12.016790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3eafda-b6a5-4dc5-9fbe-0f2f5a489f57 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:12.027323 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4003/7970] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:19:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:19:12.183272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8df097-b35a-4d62-96bc-49f39339f857 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:12.297249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8df097-b35a-4d62-96bc-49f39339f857 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:12.303557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:12.304244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:12.304587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:12.304956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] PATH_INFO: `/v3/users/b431d8b003ab4e35bbd2a382c6c49745` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:12.306401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b431d8b003ab4e35bbd2a382c6c49745)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:12.308641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:12.410576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b431d8b003ab4e35bbd2a382c6c49745'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:12.416607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:12.418870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b431d8b003ab4e35bbd2a382c6c49745'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:12.419808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b431d8b003ab4e35bbd2a382c6c49745'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:12.435134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b431d8b003ab4e35bbd2a382c6c49745'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:12.437664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b431d8b003ab4e35bbd2a382c6c49745'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:12.485229 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3968/7971] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:12 2022] DELETE /identity/v3/users/b431d8b003ab4e35bbd2a382c6c49745 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:12.490396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:12.490693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:12.490998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:12.534474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:12.536130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:12.639680 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4004/7972] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:12.669364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:12.671139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:12.775859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbac45-e8e0-4bd7-8065-cf5918eefcf9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:12.779695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedc7e1-4585-4da7-9310-7f61f215fb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:12.780316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedc7e1-4585-4da7-9310-7f61f215fb8c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:12.780601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedc7e1-4585-4da7-9310-7f61f215fb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:12.780877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedc7e1-4585-4da7-9310-7f61f215fb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:12.791421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedc7e1-4585-4da7-9310-7f61f215fb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:12.792913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedc7e1-4585-4da7-9310-7f61f215fb8c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:12.805023 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3969/7973] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:13.085637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:13.087271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:13.100623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc1b78-a6e7-4122-aa9c-e1a41ca26bda None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:13.105553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:13.106232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:13.106520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:13.107118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] PATH_INFO: `/v3/users/b8d0a762edaf45f89684220812587fd4` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:13.108225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d0a762edaf45f89684220812587fd4)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:13.110201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:13.162529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d0a762edaf45f89684220812587fd4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:13.176626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d0a762edaf45f89684220812587fd4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:13.184358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d0a762edaf45f89684220812587fd4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:13.192736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d0a762edaf45f89684220812587fd4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:13.193632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d0a762edaf45f89684220812587fd4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:13.200634 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4005/7974] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:13 2022] DELETE /identity/v3/users/b8d0a762edaf45f89684220812587fd4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:13.205128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:13.205416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:13.205696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:13.227384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:13.228834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:13.321198 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3970/7975] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:13.331290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:13.333417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:13.339206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:13.403980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f677-9640-41aa-9776-8112ebdad616 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:13.408986 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:13.409706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:13.410029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:13.410309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:13.421158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:13.422964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:13.437679 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4006/7976] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:13.665479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:13.667185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:13.804468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:13.806119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:13.807908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:13.826244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098177a-646d-4653-af2f-033156ca55e4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:13.845732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09549923-befb-4fe1-a58a-117606f91a82 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:13.859089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:13.861166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:13.862203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:13.862818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:13.863276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] PATH_INFO: `/v3/projects/72d3d7ca3cb44c06b39a951eb198b8b3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:13.865794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d3d7ca3cb44c06b39a951eb198b8b3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:13.866706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:13.868801 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:13.869543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:13.871189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:13.876969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:13.916353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:13.916353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:13.939840 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3971/7977] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:13.986542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d3d7ca3cb44c06b39a951eb198b8b3', 'request_id': 'req-15e9ba94-2048-4b04-8d4f-7b751001d82d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:13.999036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d3d7ca3cb44c06b39a951eb198b8b3', 'request_id': 'req-15e9ba94-2048-4b04-8d4f-7b751001d82d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.000576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.002550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.016453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d3d7ca3cb44c06b39a951eb198b8b3', 'request_id': 'req-15e9ba94-2048-4b04-8d4f-7b751001d82d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.022402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6932c9-c152-4b4e-827e-b2b3da5ac31e service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.039864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] The token cache is being invalidate because project 72d3d7ca3cb44c06b39a951eb198b8b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:14.040476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.041325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.045156 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4007/7978] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:13 2022] DELETE /identity/v3/projects/72d3d7ca3cb44c06b39a951eb198b8b3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:14.048462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.052525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.110849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.112032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.112326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.113171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] PATH_INFO: `/v3/projects/584e0ad9edd24d67a68ecd56f3b7e0d6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.115124 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=584e0ad9edd24d67a68ecd56f3b7e0d6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.117412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.191254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584e0ad9edd24d67a68ecd56f3b7e0d6', 'request_id': 'req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.200377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584e0ad9edd24d67a68ecd56f3b7e0d6', 'request_id': 'req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.210965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ba94-2048-4b04-8d4f-7b751001d82d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.216204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584e0ad9edd24d67a68ecd56f3b7e0d6', 'request_id': 'req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.234680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] The token cache is being invalidate because project 584e0ad9edd24d67a68ecd56f3b7e0d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:14.234979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.235677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed4db4bb-bdf1-4cec-bd41-c3015f148f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.242775 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3972/7979] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:13 2022] DELETE /identity/v3/projects/584e0ad9edd24d67a68ecd56f3b7e0d6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:14.244774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9792-45dd-4418-91fd-99ef1833c81f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.244774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9792-45dd-4418-91fd-99ef1833c81f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.245009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9792-45dd-4418-91fd-99ef1833c81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.275246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-81ff9792-45dd-4418-91fd-99ef1833c81f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:14.277616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-81ff9792-45dd-4418-91fd-99ef1833c81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.294734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:14.374983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.376440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.376727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.377003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] PATH_INFO: `/v3/users/746f76909fd9426789759227f2690ee9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.380806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746f76909fd9426789759227f2690ee9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.382945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.397857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3973/7980] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:14.404252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd831d0-f5f7-4817-ad20-e58065f1dbb0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.404252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd831d0-f5f7-4817-ad20-e58065f1dbb0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.404252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd831d0-f5f7-4817-ad20-e58065f1dbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.458967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cbd831d0-f5f7-4817-ad20-e58065f1dbb0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:14.464982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cbd831d0-f5f7-4817-ad20-e58065f1dbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.480721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746f76909fd9426789759227f2690ee9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.488438 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3974/7981] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:14.490610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd831d0-f5f7-4817-ad20-e58065f1dbb0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.516812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746f76909fd9426789759227f2690ee9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.516930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd831d0-f5f7-4817-ad20-e58065f1dbb0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.521807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.522765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.523256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.523773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.525107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:14.525642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.526299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746f76909fd9426789759227f2690ee9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.528216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.539302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746f76909fd9426789759227f2690ee9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.540325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746f76909fd9426789759227f2690ee9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:14.548341 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4008/7982] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:14 2022] DELETE /identity/v3/users/746f76909fd9426789759227f2690ee9 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:14.558657 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3975/7983] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:19:14.562852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.564684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.683641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504347e-1202-45ef-b864-1f6f7e27ad53 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.695453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.696512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.697050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.701082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.702724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.705718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.750034 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4009/7984] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:14 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:14.754727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.761623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:14.771306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49aae6-17fe-4bf3-88ad-111d1a878b5c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.863106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.864043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.864460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.864901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.865539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.869017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.905657 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3976/7985] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:14 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:14.908639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.920745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaddb91-5600-46b1-97c6-92ea19c917e3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.927992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.928963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.928963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.929267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.930356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.932088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.944420 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4010/7986] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:14.950700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.967305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:14.969211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:14.970114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7579f6-85e8-48d7-ba69-7df9a53abb78 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.978341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.979315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.980561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.981566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.981982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:14.983970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:14.990410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f373f8-ef15-4eed-8775-4857a2636820 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:14.996259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:14.996997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:14.997368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:14.997718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] PATH_INFO: `/v3/projects/12844a973af949af999e6ea733f5da40` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:14.998870 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3977/7987] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:14.999349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12844a973af949af999e6ea733f5da40)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.002015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.002906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.026774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d79aa0-3ff0-4d9a-948f-47036d6af9a8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.032917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4aaf9-20f8-4a37-9985-e5c613a0ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.034085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4aaf9-20f8-4a37-9985-e5c613a0ebc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.034085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4aaf9-20f8-4a37-9985-e5c613a0ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.034337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4aaf9-20f8-4a37-9985-e5c613a0ebc8 admin admin] PATH_INFO: `/v3/projects/a12c8f1460bd4b93990ce09297d0ea53/users/2cacc5411a7f4771b8f4ef7d5eb652e3/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.041138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:15.064570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4aaf9-20f8-4a37-9985-e5c613a0ebc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12c8f1460bd4b93990ce09297d0ea53, user_id=2cacc5411a7f4771b8f4ef7d5eb652e3, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.068447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4aaf9-20f8-4a37-9985-e5c613a0ebc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.080162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12844a973af949af999e6ea733f5da40', 'request_id': 'req-3542da92-393c-434e-a70a-e553ea4c1963'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:15.101195 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3978/7988] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:15 2022] PUT /identity/v3/projects/a12c8f1460bd4b93990ce09297d0ea53/users/2cacc5411a7f4771b8f4ef7d5eb652e3/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:15.101427 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12844a973af949af999e6ea733f5da40', 'request_id': 'req-3542da92-393c-434e-a70a-e553ea4c1963'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:15.101716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12844a973af949af999e6ea733f5da40', 'request_id': 'req-3542da92-393c-434e-a70a-e553ea4c1963'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:15.103012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.103302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.104226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.123776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] The token cache is being invalidate because project 12844a973af949af999e6ea733f5da40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:15.125363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:15.126101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3542da92-393c-434e-a70a-e553ea4c1963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:15.127920 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4011/7989] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:14 2022] DELETE /identity/v3/projects/12844a973af949af999e6ea733f5da40 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:15.134684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dce0a-c70a-4666-b51b-fdacc8043f7b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.134940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dce0a-c70a-4666-b51b-fdacc8043f7b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.135217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dce0a-c70a-4666-b51b-fdacc8043f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.149207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] MFA Rules not processed for user `2cacc5411a7f4771b8f4ef7d5eb652e3`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:15.150854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.171189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f62dce0a-c70a-4666-b51b-fdacc8043f7b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:15.172698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f62dce0a-c70a-4666-b51b-fdacc8043f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.264763 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3979/7990] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:15.269974 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4012/7991] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:15.274651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.275540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff979-165f-4e5a-b65d-7e547c4caa51 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.275947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff979-165f-4e5a-b65d-7e547c4caa51 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.276074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.276389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ff979-165f-4e5a-b65d-7e547c4caa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.297989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda2c32-2c00-4262-8eff-cd6b8813377f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.309250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.309925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.310209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.310488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.311132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.313235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.328363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5f9ff979-165f-4e5a-b65d-7e547c4caa51 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:15.339875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f9ff979-165f-4e5a-b65d-7e547c4caa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:15.353045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4013/7992] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:15.357712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ff979-165f-4e5a-b65d-7e547c4caa51 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.364085 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3980/7993] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:15.369112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.372161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ff979-165f-4e5a-b65d-7e547c4caa51 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.415636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:15.483396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcd7be-376f-41d8-8ee3-f808e33c0b8e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.484273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.485552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.485891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.486379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.487228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:15.487727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.490919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.498966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.499961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.500587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.501080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.502080 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.505422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.527906 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4014/7994] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:15.529959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.543177 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3981/7995] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:15.551340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.552122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef82f4-03b3-453b-b5ff-d9424d41004c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.564290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c854f-dd0e-42aa-8f26-0749327b3573 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.565334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.566684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.567010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.567560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.568632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.570747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.587347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.588891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.589249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.589535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.590324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.592040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.608868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3982/7996] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:15.615281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.619642 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4015/7997] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:15.622815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.640641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9314c8-8046-44b3-89c8-4374873126d4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.659618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:15.723565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f49f-61ab-43ed-bd51-561140fa1048 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.730570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.732770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.732770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.733374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.734452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.740003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.744999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.745976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.746492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.746884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] PATH_INFO: `/v3/projects/55665d6a7f5f4a8faea85c4f8c1c547d/users/4001d51354dc4117ab7f740d7aecf9b3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.767755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55665d6a7f5f4a8faea85c4f8c1c547d, user_id=4001d51354dc4117ab7f740d7aecf9b3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.771770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.781843 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4016/7998] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:15.791102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.802783 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3983/7999] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:15 2022] PUT /identity/v3/projects/55665d6a7f5f4a8faea85c4f8c1c547d/users/4001d51354dc4117ab7f740d7aecf9b3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:15.804447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.808731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14904a9d-93a1-4c84-819c-bbb09d4ee3ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.899103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.899727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.900008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.900283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.900925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.901948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62df4-544c-41e0-8162-3a5bba3cb6b0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.902613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.907147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.907763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.908050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.908332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.909009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.911214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.915263 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4017/8000] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:19:15.918798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.922298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3984/8001] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:15.927082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.933309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786e59-e2f6-41e6-87cc-3aaedaf22e48 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.938864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8393ed-f6ad-4a4b-84a7-c3ff118197d7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.939646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.940443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.941456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.941850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.942816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.944535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0bfb5-a055-40c5-b7a5-a4f37038c35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:15.944702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.945192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0bfb5-a055-40c5-b7a5-a4f37038c35c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.945479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0bfb5-a055-40c5-b7a5-a4f37038c35c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.945762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0bfb5-a055-40c5-b7a5-a4f37038c35c admin admin] PATH_INFO: `/v3/projects/55665d6a7f5f4a8faea85c4f8c1c547d/users/4001d51354dc4117ab7f740d7aecf9b3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:15.947893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0bfb5-a055-40c5-b7a5-a4f37038c35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=55665d6a7f5f4a8faea85c4f8c1c547d, user_id=4001d51354dc4117ab7f740d7aecf9b3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:15.950637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0bfb5-a055-40c5-b7a5-a4f37038c35c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:15.956733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4018/8002] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:19:15.961033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:15.971284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d570f-dd6b-4d7e-8808-c0a32f9388c1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:15.976894 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3985/8003] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:15 2022] PUT /identity/v3/projects/55665d6a7f5f4a8faea85c4f8c1c547d/users/4001d51354dc4117ab7f740d7aecf9b3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:15.988703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:15.989489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:15.990211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:16.014934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:16.026548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] MFA Rules not processed for user `4001d51354dc4117ab7f740d7aecf9b3`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:16.028065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:16.040634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330db17-4f9b-445b-8965-a6968bbddb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:16.041580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330db17-4f9b-445b-8965-a6968bbddb53 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:16.041901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330db17-4f9b-445b-8965-a6968bbddb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:16.042185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330db17-4f9b-445b-8965-a6968bbddb53 admin admin] PATH_INFO: `/v3/projects/71d9908326754af3b47c0e3e26a4a04b/users/afe708fa9a294268b8a55ac0a7172900/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:16.059636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330db17-4f9b-445b-8965-a6968bbddb53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d9908326754af3b47c0e3e26a4a04b, user_id=afe708fa9a294268b8a55ac0a7172900, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:16.062446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330db17-4f9b-445b-8965-a6968bbddb53 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:16.084905 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4019/8004] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:15 2022] PUT /identity/v3/projects/71d9908326754af3b47c0e3e26a4a04b/users/afe708fa9a294268b8a55ac0a7172900/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:16.089285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:16.089684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:16.090098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:16.121851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] MFA Rules not processed for user `afe708fa9a294268b8a55ac0a7172900`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:16.123256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:16.127659 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3986/8005] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:16.132739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:16.141206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:16.145043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:16.224195 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4020/8006] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:16.235790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db44c-4588-4583-b0c3-34aba6a8ccf1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:16.237978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:16.240227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:16.294043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7905e-ccd3-47f8-b32d-6ebd1b7b570f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:16.294807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7905e-ccd3-47f8-b32d-6ebd1b7b570f service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:16.294998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7905e-ccd3-47f8-b32d-6ebd1b7b570f service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:16.295404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7905e-ccd3-47f8-b32d-6ebd1b7b570f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:16.309722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0881930-861d-4f43-bb1c-ac40f463c191 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:16.313928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:16.314557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:16.314868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:16.315120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:16.324653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:16.326175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:16.338199 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4021/8007] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:19:16.369367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:16.373158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7905e-ccd3-47f8-b32d-6ebd1b7b570f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:16.375039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7905e-ccd3-47f8-b32d-6ebd1b7b570f service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:16.386056 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3987/8008] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:19:17.451120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:17.517968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a767a-0123-44df-b21f-c5de1a9a25b6 service glance] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:17.522973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:17.523606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:17.523908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:17.524310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] PATH_INFO: `/v3/users/afe708fa9a294268b8a55ac0a7172900` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:17.525445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe708fa9a294268b8a55ac0a7172900)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:17.527401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:17.576646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe708fa9a294268b8a55ac0a7172900'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:17.579051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:17.594675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe708fa9a294268b8a55ac0a7172900'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:17.602064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe708fa9a294268b8a55ac0a7172900'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:17.620522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe708fa9a294268b8a55ac0a7172900'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:17.621266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe708fa9a294268b8a55ac0a7172900'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:17.631946 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4022/8009] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:17 2022] DELETE /identity/v3/users/afe708fa9a294268b8a55ac0a7172900 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:17.636318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:17.636787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:17.636901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:17.662957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:17.664189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:17.755678 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3988/8010] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:17.767524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:17.770327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:17.881155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cc99b-a28b-45e6-b24a-2ed7d7114bd0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:17.887060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640dc72-4446-431d-9f02-31ef7b033919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:17.887739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640dc72-4446-431d-9f02-31ef7b033919 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:17.888080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640dc72-4446-431d-9f02-31ef7b033919 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:17.889007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640dc72-4446-431d-9f02-31ef7b033919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:17.902607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640dc72-4446-431d-9f02-31ef7b033919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:17.904269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640dc72-4446-431d-9f02-31ef7b033919 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:17.929419 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4023/8011] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:18.643407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:18.646041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:18.662728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf0db5-70b1-46c9-9403-ccd3b2f6adfc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:18.668135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:18.668943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:18.669352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:18.669644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] PATH_INFO: `/v3/projects/71d9908326754af3b47c0e3e26a4a04b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:18.671664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d9908326754af3b47c0e3e26a4a04b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:18.674277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:18.729638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d9908326754af3b47c0e3e26a4a04b', 'request_id': 'req-1130a69f-3548-440e-bf6c-5a6661325a3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:18.737412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d9908326754af3b47c0e3e26a4a04b', 'request_id': 'req-1130a69f-3548-440e-bf6c-5a6661325a3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:18.745636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d9908326754af3b47c0e3e26a4a04b', 'request_id': 'req-1130a69f-3548-440e-bf6c-5a6661325a3e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:18.757045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] The token cache is being invalidate because project 71d9908326754af3b47c0e3e26a4a04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:18.757339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:18.758301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1130a69f-3548-440e-bf6c-5a6661325a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:18.760009 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3989/8012] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:18 2022] DELETE /identity/v3/projects/71d9908326754af3b47c0e3e26a4a04b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:18.765638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24cc7-6c1a-49cd-92c0-03d4d277e767 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:18.766658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24cc7-6c1a-49cd-92c0-03d4d277e767 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:18.766946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24cc7-6c1a-49cd-92c0-03d4d277e767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:18.787731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-98d24cc7-6c1a-49cd-92c0-03d4d277e767 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:18.788944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-98d24cc7-6c1a-49cd-92c0-03d4d277e767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:18.850644 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4024/8013] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:18.859945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e1f0-8cba-4df3-a4c4-1c7a9f1fcacb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:18.860232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e1f0-8cba-4df3-a4c4-1c7a9f1fcacb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:18.860510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e1f0-8cba-4df3-a4c4-1c7a9f1fcacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:18.882133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c9c3e1f0-8cba-4df3-a4c4-1c7a9f1fcacb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:18.883260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c9c3e1f0-8cba-4df3-a4c4-1c7a9f1fcacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:18.891270 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3990/8014] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:18.895591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d24cc7-6c1a-49cd-92c0-03d4d277e767 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:18.905359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d24cc7-6c1a-49cd-92c0-03d4d277e767 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:18.910436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:18.911051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:18.911315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:18.911589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:18.912227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:18.912531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:18.914165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:18.926684 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4025/8015] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:19:18.930851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e1f0-8cba-4df3-a4c4-1c7a9f1fcacb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:18.940536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e1f0-8cba-4df3-a4c4-1c7a9f1fcacb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:18.945803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:18.946443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:18.946727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:18.947003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:18.947643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:18.949553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:18.981092 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3991/8016] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:18 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:18.985548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:18.996545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64250ee7-e9ec-455f-b0f3-e5626c0eb6c6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:19.052972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:19.060624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:19.061312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:19.061867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:19.062366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:19.063412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:19.066254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:19.093710 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4026/8017] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:18 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:19.094779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:19.105626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b522b-e7fe-47bd-8442-cedf6c21261c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:19.110703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:19.111305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:19.111590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:19.111866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:19.112600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:19.114460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:19.124619 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3992/8018] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:19.129337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:19.139618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a367d-d72b-444e-b962-4d00ca4b0deb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:19.144809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48911e-3c50-4576-97d8-ba75e29db540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:19.145475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48911e-3c50-4576-97d8-ba75e29db540 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:19.145820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48911e-3c50-4576-97d8-ba75e29db540 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:19.146176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48911e-3c50-4576-97d8-ba75e29db540 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:19.146919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48911e-3c50-4576-97d8-ba75e29db540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:19.149684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48911e-3c50-4576-97d8-ba75e29db540 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:19.163308 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4027/8019] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:19.167978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:19.188744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db7d8d-4ba7-4260-89a3-14b07bf027b4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:19.196148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:19.196871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:19.197261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:19.198332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] PATH_INFO: `/v3/projects/6e4ce02058134f19802bb869ddf64e22/users/a3260e19b7e14ba784ffb4cf0a6d50e6/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:19.217316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4ce02058134f19802bb869ddf64e22, user_id=a3260e19b7e14ba784ffb4cf0a6d50e6, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:19.221125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:19.238218 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3993/8020] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:19 2022] PUT /identity/v3/projects/6e4ce02058134f19802bb869ddf64e22/users/a3260e19b7e14ba784ffb4cf0a6d50e6/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:19.250743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:19.251027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:19.251323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:19.283489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] MFA Rules not processed for user `a3260e19b7e14ba784ffb4cf0a6d50e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:19.284827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:19.503975 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4028/8021] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:19.514151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:19.515890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:19.536288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:19.587500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ec393-ebb9-4da1-b5f6-1a4045858a3b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:19.591442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:19.592141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:19.592536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:19.592811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:19.606237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:19.608024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:19.619903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3994/8022] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:19:20.296900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:20.298449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:20.381556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565c926-4050-47b7-9641-010df5997ec8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:20.385634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7dd76-f809-4fa9-873f-a421068c9662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:20.387257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7dd76-f809-4fa9-873f-a421068c9662 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:20.387257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7dd76-f809-4fa9-873f-a421068c9662 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:20.387257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7dd76-f809-4fa9-873f-a421068c9662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:20.389046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ffc7dd76-f809-4fa9-873f-a421068c9662 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:20.470258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7dd76-f809-4fa9-873f-a421068c9662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:20.472675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7dd76-f809-4fa9-873f-a421068c9662 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:20.488021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4029/8023] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:21.588629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:21.590217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:21.659622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae421-cae3-4024-8e26-685d92ee86f8 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:21.665300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:21.665962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:21.666245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:21.666536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] PATH_INFO: `/v3/users/c501424e97f642089f7c9da42d1c70c6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:21.667646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c501424e97f642089f7c9da42d1c70c6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:21.669563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:21.726329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c501424e97f642089f7c9da42d1c70c6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:21.733608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c501424e97f642089f7c9da42d1c70c6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:21.734349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c501424e97f642089f7c9da42d1c70c6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:21.744619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c501424e97f642089f7c9da42d1c70c6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:21.745234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c501424e97f642089f7c9da42d1c70c6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:21.780224 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3995/8024] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:21 2022] DELETE /identity/v3/users/c501424e97f642089f7c9da42d1c70c6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:21.782169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:21.782627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:21.782730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:21.818182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:21.819424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:21.842429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:21.923147 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4030/8025] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:21.934586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:21.936251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:22.014778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:22.042936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95584a41-612f-4b61-874f-394c764c2da3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:22.047492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:22.048906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:22.049416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:22.049904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:22.063295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:22.064953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:22.086280 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3996/8026] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:22.422997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:22.426571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:22.480050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92eaf4-aaf1-457b-b76a-e29f0ad37f64 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:22.493294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:22.494251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:22.494551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:22.494811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] PATH_INFO: `/v3/projects/7748a65ff6f248debe13c9cb6c91177c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:22.496167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7748a65ff6f248debe13c9cb6c91177c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:22.498395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:22.591780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7748a65ff6f248debe13c9cb6c91177c', 'request_id': 'req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.617678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7748a65ff6f248debe13c9cb6c91177c', 'request_id': 'req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.632903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7748a65ff6f248debe13c9cb6c91177c', 'request_id': 'req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.648507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] The token cache is being invalidate because project 7748a65ff6f248debe13c9cb6c91177c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:22.648946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.649865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.653351 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4031/8027] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:22 2022] DELETE /identity/v3/projects/7748a65ff6f248debe13c9cb6c91177c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:22.657473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:22.672216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badd2d5-e5bb-45ff-bb4a-89cc46ef1d26 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:22.749766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:22.750332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:22.750966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:22.751578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] PATH_INFO: `/v3/users/7d7cfedd7ff841e983f6f8f6ead327c4` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:22.753718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d7cfedd7ff841e983f6f8f6ead327c4)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:22.755852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:22.819059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d7cfedd7ff841e983f6f8f6ead327c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.826507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d7cfedd7ff841e983f6f8f6ead327c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.827222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d7cfedd7ff841e983f6f8f6ead327c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.839739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d7cfedd7ff841e983f6f8f6ead327c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.840900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d7cfedd7ff841e983f6f8f6ead327c4'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:22.867270 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3997/8028] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:22 2022] DELETE /identity/v3/users/7d7cfedd7ff841e983f6f8f6ead327c4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:23.269359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:23.272243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:23.307769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126b1f0-fd6e-4ca3-afaa-caf540edbcb3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:23.455040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:23.457635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:23.457851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:23.458334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] PATH_INFO: `/v3/projects/bbdb725754214f05af47df40f9722acb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:23.462693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbdb725754214f05af47df40f9722acb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:23.465177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:23.552530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:23.571930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbdb725754214f05af47df40f9722acb', 'request_id': 'req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.578864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbdb725754214f05af47df40f9722acb', 'request_id': 'req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.596388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbdb725754214f05af47df40f9722acb', 'request_id': 'req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.623432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] The token cache is being invalidate because project bbdb725754214f05af47df40f9722acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:23.623432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.623717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.625984 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4032/8029] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:23 2022] DELETE /identity/v3/projects/bbdb725754214f05af47df40f9722acb => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:23.630946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:23.632723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:23.652130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93613-65cd-4f6d-aca3-bca13496eb15 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:23.752290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:23.755189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:23.755776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:23.756381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] PATH_INFO: `/v3/users/895c0ca90f844e0db7feb668f7cc5a47` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:23.759012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=895c0ca90f844e0db7feb668f7cc5a47)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:23.761360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:23.767060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:23.867348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '895c0ca90f844e0db7feb668f7cc5a47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.883194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '895c0ca90f844e0db7feb668f7cc5a47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.884721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '895c0ca90f844e0db7feb668f7cc5a47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.909535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '895c0ca90f844e0db7feb668f7cc5a47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.911750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2e288b99-7454-415d-9f9c-d848a1bff507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '895c0ca90f844e0db7feb668f7cc5a47'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:23.954399 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3998/8030] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:23 2022] DELETE /identity/v3/users/895c0ca90f844e0db7feb668f7cc5a47 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:24.419457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:24.421458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:24.443035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b12743-4b60-4276-8f5e-ac8be84f4d4e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:24.560050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:24.561036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:24.561586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:24.562126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] PATH_INFO: `/v3/projects/67ae7b20dc114a978f792b5e9481bf9d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:24.564290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ae7b20dc114a978f792b5e9481bf9d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:24.568003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:24.666093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ae7b20dc114a978f792b5e9481bf9d', 'request_id': 'req-2c423336-1c9d-4f60-884c-d010e2c8d36e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:24.673441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ae7b20dc114a978f792b5e9481bf9d', 'request_id': 'req-2c423336-1c9d-4f60-884c-d010e2c8d36e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:24.683027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ae7b20dc114a978f792b5e9481bf9d', 'request_id': 'req-2c423336-1c9d-4f60-884c-d010e2c8d36e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:24.706660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] The token cache is being invalidate because project 67ae7b20dc114a978f792b5e9481bf9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:24.706964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:24.708251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2c423336-1c9d-4f60-884c-d010e2c8d36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:24.712460 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4033/8031] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:24 2022] DELETE /identity/v3/projects/67ae7b20dc114a978f792b5e9481bf9d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:24.718326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1a66-ae50-4a73-8250-639ea50866d0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:24.718765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1a66-ae50-4a73-8250-639ea50866d0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:24.719105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a1a66-ae50-4a73-8250-639ea50866d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:24.761404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a00a1a66-ae50-4a73-8250-639ea50866d0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:24.762977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a00a1a66-ae50-4a73-8250-639ea50866d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:24.952810 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 3999/8032] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:24.954848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f2aa1-27ad-4f0b-9be7-1e0b52dea030 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:24.956282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f2aa1-27ad-4f0b-9be7-1e0b52dea030 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:24.956831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f2aa1-27ad-4f0b-9be7-1e0b52dea030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:24.985998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e93f2aa1-27ad-4f0b-9be7-1e0b52dea030 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:24.988087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e93f2aa1-27ad-4f0b-9be7-1e0b52dea030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:25.009742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a1a66-ae50-4a73-8250-639ea50866d0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.027788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a1a66-ae50-4a73-8250-639ea50866d0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.029019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4034/8033] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:25.040936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.041591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.041911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.042213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.046295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:25.046659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.050876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.070172 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4000/8034] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:25.074710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f2aa1-27ad-4f0b-9be7-1e0b52dea030 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.090043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f2aa1-27ad-4f0b-9be7-1e0b52dea030 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.095938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.096907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.097384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.099521 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.100214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.102674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.138889 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4035/8035] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:25 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:25.142940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.163786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7a2a2-cdb9-4ec9-b34a-cc95f9b01d86 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.267600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.268201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.269143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.269342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.271206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.273340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.301758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:25.333652 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4001/8036] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:25 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:25.337555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.360131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602967-298f-4a88-92cd-5a1c108265fe admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.368731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.369596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.370060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.370638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.371753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.373605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.391387 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4036/8037] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:25.395478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.411730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3d7ed-02d6-45ad-83b2-ea01bbf9ae48 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.440043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420ac11-c941-480f-8104-8da2cebfe234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.441192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ac11-c941-480f-8104-8da2cebfe234 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.441488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ac11-c941-480f-8104-8da2cebfe234 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.441707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420ac11-c941-480f-8104-8da2cebfe234 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.442373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420ac11-c941-480f-8104-8da2cebfe234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.444010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420ac11-c941-480f-8104-8da2cebfe234 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.462608 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4002/8038] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:25.463115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.477684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eee03-e8e2-47d5-b0f9-99b9babba8d1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.493367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.494891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.495417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.495848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] PATH_INFO: `/v3/projects/269aa73599864b0ab0398f2efc01df71/users/6bb7e5d4c2124073b92162c961179540/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.527203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=269aa73599864b0ab0398f2efc01df71, user_id=6bb7e5d4c2124073b92162c961179540, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.530669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.537533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:25.562045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4037/8039] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:25 2022] PUT /identity/v3/projects/269aa73599864b0ab0398f2efc01df71/users/6bb7e5d4c2124073b92162c961179540/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:25.574283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169903-73b2-4143-9ad7-d43868262bac None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.575074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169903-73b2-4143-9ad7-d43868262bac None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.575360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169903-73b2-4143-9ad7-d43868262bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.611286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f3169903-73b2-4143-9ad7-d43868262bac None None] MFA Rules not processed for user `6bb7e5d4c2124073b92162c961179540`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:25.613312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f3169903-73b2-4143-9ad7-d43868262bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:25.694905 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4003/8040] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:25.699292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.776456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76edb6-1523-47dc-a403-e521ad2f6d56 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.782633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.783247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.783525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.783815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.785143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.787160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.822812 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4038/8041] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:25 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:25.827122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169903-73b2-4143-9ad7-d43868262bac None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.843955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169903-73b2-4143-9ad7-d43868262bac None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:25.923155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:25.923829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:25.924077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:25.924371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:25.926083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:25.928051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:25.957157 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4004/8042] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:25 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:25.964483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:25.989083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d6b7c-f8c9-4678-a663-9daee5b644f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:26.005109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:26.005109 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:26.005406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:26.005771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:26.006514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:26.011110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:26.034185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4039/8043] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:26.039274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:26.067203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0c7e2-740a-40a2-8c7a-de2da61f02b3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:26.075123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:26.075123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:26.075464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:26.075671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] PATH_INFO: `/v3/projects/18c02b2acf0e4489902b383eb30282a8/users/2c549a7d08d14f11b93e13ae3f54a59c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:26.118418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c02b2acf0e4489902b383eb30282a8, user_id=2c549a7d08d14f11b93e13ae3f54a59c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:26.121249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:26.136517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4005/8044] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:26 2022] PUT /identity/v3/projects/18c02b2acf0e4489902b383eb30282a8/users/2c549a7d08d14f11b93e13ae3f54a59c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:26.140345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:26.153052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b619f-23ca-4e0f-ba3f-41a2c390c7f0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:26.230268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42116b0b-c664-4de3-a002-77f55a5d7b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:26.230884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42116b0b-c664-4de3-a002-77f55a5d7b0f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:26.231164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42116b0b-c664-4de3-a002-77f55a5d7b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:26.231443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42116b0b-c664-4de3-a002-77f55a5d7b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:26.232345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42116b0b-c664-4de3-a002-77f55a5d7b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:26.233970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42116b0b-c664-4de3-a002-77f55a5d7b0f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:26.246483 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4040/8045] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:19:26.250643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:26.265323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6200be9-dc97-4cb8-8280-fc7bf738e668 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:26.278069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:26.278932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:26.279289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:26.279661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] PATH_INFO: `/v3/projects/18c02b2acf0e4489902b383eb30282a8/users/2c549a7d08d14f11b93e13ae3f54a59c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:26.281800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c02b2acf0e4489902b383eb30282a8, user_id=2c549a7d08d14f11b93e13ae3f54a59c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:26.284992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:26.298096 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4006/8046] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:26 2022] PUT /identity/v3/projects/18c02b2acf0e4489902b383eb30282a8/users/2c549a7d08d14f11b93e13ae3f54a59c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:26.307952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:26.308520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:26.308823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:26.374424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] MFA Rules not processed for user `2c549a7d08d14f11b93e13ae3f54a59c`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:26.376089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:26.476090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:26.510448 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4041/8047] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:26.536422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:26.538324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:26.593789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:26.642934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb302f91-cdd7-46b4-bede-43f19bc29164 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:26.648985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:26.649620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:26.649961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:26.650287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:26.663848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:26.665308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:26.677132 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4007/8048] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:19:27.572342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:27.574660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:27.606772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fa27f-9de7-4597-a589-d29d2d267476 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:27.611834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e91e4-b79e-497b-ad0d-73d437aec76c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:27.612574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e91e4-b79e-497b-ad0d-73d437aec76c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:27.612868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e91e4-b79e-497b-ad0d-73d437aec76c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:27.613452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e91e4-b79e-497b-ad0d-73d437aec76c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:27.615366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0f3e91e4-b79e-497b-ad0d-73d437aec76c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:27.628785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e91e4-b79e-497b-ad0d-73d437aec76c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:27.630651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e91e4-b79e-497b-ad0d-73d437aec76c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:27.642669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4042/8049] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:28.884875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:28.886546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:28.974700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a6747-a809-40f7-b01a-a5c6206ccd08 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:28.979910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:28.980590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:28.983627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:28.983627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] PATH_INFO: `/v3/users/eaf8e35a0a6845768bd0ad61c9b87e22` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:28.984877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf8e35a0a6845768bd0ad61c9b87e22)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:28.986993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:29.064176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf8e35a0a6845768bd0ad61c9b87e22'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.078478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf8e35a0a6845768bd0ad61c9b87e22'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.080299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf8e35a0a6845768bd0ad61c9b87e22'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.099252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf8e35a0a6845768bd0ad61c9b87e22'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.106562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf8e35a0a6845768bd0ad61c9b87e22'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.133409 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4008/8050] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:28 2022] DELETE /identity/v3/users/eaf8e35a0a6845768bd0ad61c9b87e22 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:29.135184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:29.135644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:29.135769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:29.151176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:29.152692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:29.158766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:29.160470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:29.221268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:29.247971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaa54a-9194-418f-8ddc-7dc3996122be admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:29.254747 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4043/8051] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:29.258115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:29.258778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:29.259098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:29.259321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] PATH_INFO: `/v3/users/6bb7e5d4c2124073b92162c961179540` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:29.260689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb7e5d4c2124073b92162c961179540)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:29.262482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:29.264948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:29.266462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:29.344517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8af28-31b3-466e-8460-9f195a610e6f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:29.350276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51535d79-c807-4420-913a-afabcb996b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:29.350894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535d79-c807-4420-913a-afabcb996b40 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:29.351352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535d79-c807-4420-913a-afabcb996b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:29.351487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb7e5d4c2124073b92162c961179540'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.351624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535d79-c807-4420-913a-afabcb996b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:29.362147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb7e5d4c2124073b92162c961179540'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.364088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb7e5d4c2124073b92162c961179540'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.364315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51535d79-c807-4420-913a-afabcb996b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:29.365860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51535d79-c807-4420-913a-afabcb996b40 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:29.374907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb7e5d4c2124073b92162c961179540'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.375564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb7e5d4c2124073b92162c961179540'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.379379 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4044/8052] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:19:29.402021 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4009/8053] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:29 2022] DELETE /identity/v3/users/6bb7e5d4c2124073b92162c961179540 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:29.407265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:29.407795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:29.408149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:29.432435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:29.433651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:29.497459 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4045/8054] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:29.507493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:29.508921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:29.572101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5430831-06da-44ec-b00b-7a7604e64496 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:29.575802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038db6c-fbab-498f-8a5a-713f60f0b7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:29.576445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038db6c-fbab-498f-8a5a-713f60f0b7ba service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:29.576727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038db6c-fbab-498f-8a5a-713f60f0b7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:29.577016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038db6c-fbab-498f-8a5a-713f60f0b7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:29.587138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038db6c-fbab-498f-8a5a-713f60f0b7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:29.588171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038db6c-fbab-498f-8a5a-713f60f0b7ba service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:29.599305 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4010/8055] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:29.733011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:29.734570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:29.748834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d6f05-06a3-4012-b402-e117d0eb88b9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:29.754305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:29.754951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:29.755234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:29.755512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] PATH_INFO: `/v3/projects/b5963e3c1ac34aa29de539734a0c6466` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:29.757012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5963e3c1ac34aa29de539734a0c6466)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:29.759735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:29.787890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:29.849311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5963e3c1ac34aa29de539734a0c6466', 'request_id': 'req-30b8c532-5b9c-4935-9611-61b6c4afa1ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.857388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5963e3c1ac34aa29de539734a0c6466', 'request_id': 'req-30b8c532-5b9c-4935-9611-61b6c4afa1ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.867410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5963e3c1ac34aa29de539734a0c6466', 'request_id': 'req-30b8c532-5b9c-4935-9611-61b6c4afa1ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.887022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] The token cache is being invalidate because project b5963e3c1ac34aa29de539734a0c6466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:29.887445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.888386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:29.891828 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4046/8056] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:29 2022] DELETE /identity/v3/projects/b5963e3c1ac34aa29de539734a0c6466 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:29.897791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a6f28-fe5c-4b68-9a6d-fcccfbe60d6c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:29.898252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a6f28-fe5c-4b68-9a6d-fcccfbe60d6c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:29.898876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a6f28-fe5c-4b68-9a6d-fcccfbe60d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:29.936010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8d0a6f28-fe5c-4b68-9a6d-fcccfbe60d6c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:29.937076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8d0a6f28-fe5c-4b68-9a6d-fcccfbe60d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.000533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.002650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.038907 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4011/8057] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:30.048594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee37ab-24eb-4a64-b6b5-fd5744661d4b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.048791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee37ab-24eb-4a64-b6b5-fd5744661d4b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.049069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee37ab-24eb-4a64-b6b5-fd5744661d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.082262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f5ee37ab-24eb-4a64-b6b5-fd5744661d4b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:30.085488 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f5ee37ab-24eb-4a64-b6b5-fd5744661d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.096258 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4012/8058] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:30.101874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee37ab-24eb-4a64-b6b5-fd5744661d4b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.107667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:30.116169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8c532-5b9c-4935-9611-61b6c4afa1ba admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.123122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee37ab-24eb-4a64-b6b5-fd5744661d4b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.124419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.124945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.125615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.126045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] PATH_INFO: `/v3/projects/269aa73599864b0ab0398f2efc01df71` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.127476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=269aa73599864b0ab0398f2efc01df71)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.131799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.134037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.134829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.135173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.135751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.136617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:30.137206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.139447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.154447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4013/8059] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:30.158682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.172104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27539ba3-eb4b-4fad-9dad-6be9035f8653 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.181175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.182289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.183103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.183635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.185089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.188129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.201574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '269aa73599864b0ab0398f2efc01df71', 'request_id': 'req-b67315c7-2312-4265-9fa6-7c2b932125d5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.210744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '269aa73599864b0ab0398f2efc01df71', 'request_id': 'req-b67315c7-2312-4265-9fa6-7c2b932125d5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.218486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '269aa73599864b0ab0398f2efc01df71', 'request_id': 'req-b67315c7-2312-4265-9fa6-7c2b932125d5'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.234412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] The token cache is being invalidate because project 269aa73599864b0ab0398f2efc01df71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:30.235214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.236414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.241680 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4014/8060] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:30 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:30.242758 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4047/8061] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:29 2022] DELETE /identity/v3/projects/269aa73599864b0ab0398f2efc01df71 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:30.245772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.246946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.247700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.248665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.303261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:30.347522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14c9f0-fa79-4ccf-b110-e40b71250dbc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.354918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67315c7-2312-4265-9fa6-7c2b932125d5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.361563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.362058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.362764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.363053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.363364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.364077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.366236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.366725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.367119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.367626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] PATH_INFO: `/v3/users/2c549a7d08d14f11b93e13ae3f54a59c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.369028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c549a7d08d14f11b93e13ae3f54a59c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.372590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.401566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4015/8062] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:30 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:30.405164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.416709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957043c-e36f-4c53-a2ba-529ad2dfae3b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.421885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.422508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.422791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.423065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.423713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.425365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.436653 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4016/8063] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:30.441799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.447398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c549a7d08d14f11b93e13ae3f54a59c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.458183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed1aa5-ae48-456d-a5a7-44b9ccc9954e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.464752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c549a7d08d14f11b93e13ae3f54a59c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.466223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.467002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.467279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.467563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.468211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.469990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.478872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c549a7d08d14f11b93e13ae3f54a59c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.492896 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4017/8064] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:30.493642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c549a7d08d14f11b93e13ae3f54a59c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.494612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f44db18f-0c66-4fcc-9f48-ec5901fa8735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c549a7d08d14f11b93e13ae3f54a59c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:30.494804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.497158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.504240 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4048/8065] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:30 2022] DELETE /identity/v3/users/2c549a7d08d14f11b93e13ae3f54a59c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:30.519610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed68c3-6ba0-41cd-a009-23ed44fbe72d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.596938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.597805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.598129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.598465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] PATH_INFO: `/v3/projects/6d1050bedc7e43f29cfc5aab839e75cb/users/8d2bcdaa6eda42b0bd561d657c1072ac/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.618186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1050bedc7e43f29cfc5aab839e75cb, user_id=8d2bcdaa6eda42b0bd561d657c1072ac, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.620918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.637461 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4018/8066] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:30 2022] PUT /identity/v3/projects/6d1050bedc7e43f29cfc5aab839e75cb/users/8d2bcdaa6eda42b0bd561d657c1072ac/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:30.649143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.649871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.649871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.680920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] MFA Rules not processed for user `8d2bcdaa6eda42b0bd561d657c1072ac`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:30.683344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.772471 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4049/8067] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:30.781739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.783869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.810760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:30.867312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:30.869140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:30.893384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c65085-9b66-4820-9435-faff9f2462a8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:30.904384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359447b-7241-453b-9ba7-deccc01b28f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:30.905085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359447b-7241-453b-9ba7-deccc01b28f4 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:30.905365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359447b-7241-453b-9ba7-deccc01b28f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:30.905634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359447b-7241-453b-9ba7-deccc01b28f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:30.928557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359447b-7241-453b-9ba7-deccc01b28f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:30.930447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359447b-7241-453b-9ba7-deccc01b28f4 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:30.945748 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4019/8068] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:30.948298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:30.995239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa855db-97e8-443e-b945-5cd2333c0fa8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.004097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.004828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.005208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.005483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] PATH_INFO: `/v3/projects/18c02b2acf0e4489902b383eb30282a8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.008227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=18c02b2acf0e4489902b383eb30282a8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.010646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.105249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18c02b2acf0e4489902b383eb30282a8', 'request_id': 'req-9cb74adf-0608-4c50-a5d8-56aab33d78ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:31.121789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18c02b2acf0e4489902b383eb30282a8', 'request_id': 'req-9cb74adf-0608-4c50-a5d8-56aab33d78ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:31.135362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18c02b2acf0e4489902b383eb30282a8', 'request_id': 'req-9cb74adf-0608-4c50-a5d8-56aab33d78ed'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:31.155408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] The token cache is being invalidate because project 18c02b2acf0e4489902b383eb30282a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:31.155689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:31.156699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-9cb74adf-0608-4c50-a5d8-56aab33d78ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:31.159886 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4050/8069] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:30 2022] DELETE /identity/v3/projects/18c02b2acf0e4489902b383eb30282a8 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:31.164315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c22c-985e-4c99-8b92-eb6f1d1af628 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.165496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c22c-985e-4c99-8b92-eb6f1d1af628 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.165872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c22c-985e-4c99-8b92-eb6f1d1af628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.194958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f5a8c22c-985e-4c99-8b92-eb6f1d1af628 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:31.196596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f5a8c22c-985e-4c99-8b92-eb6f1d1af628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:31.288545 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4020/8070] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:31.296265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0301f-c245-497e-874c-fb2bd53a8444 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.296823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0301f-c245-497e-874c-fb2bd53a8444 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.296943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0301f-c245-497e-874c-fb2bd53a8444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.346066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-86a0301f-c245-497e-874c-fb2bd53a8444 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:31.348045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-86a0301f-c245-497e-874c-fb2bd53a8444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:31.364194 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4051/8071] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:31.368047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c22c-985e-4c99-8b92-eb6f1d1af628 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:31.376165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c22c-985e-4c99-8b92-eb6f1d1af628 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.382192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.382811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.383283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.383570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.384650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:31.384966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.386906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.420813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4021/8072] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:31.422102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0301f-c245-497e-874c-fb2bd53a8444 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:31.432225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0301f-c245-497e-874c-fb2bd53a8444 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.437143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.437749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.438073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.438339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.438969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.441631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.473757 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4052/8073] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:31 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:31.477733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:31.489288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653cf36-681c-4e20-8461-3d8f2974f577 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.554388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.555003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.555281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.555551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.556177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.558125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.582380 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4022/8074] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:31 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:31.586411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:31.598009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5c189-bbc2-4763-b1d3-8ab5f5b007ae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.602785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.603515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.603697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.603973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.604643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.606342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.617930 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4053/8075] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:31.622302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:31.633953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ec565-775a-4883-86f4-45fc203be1f8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.638731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbc460-e081-49ff-86e4-969f7b3e56ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.639327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbc460-e081-49ff-86e4-969f7b3e56ad admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.639606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbc460-e081-49ff-86e4-969f7b3e56ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.639877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbc460-e081-49ff-86e4-969f7b3e56ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.640510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbc460-e081-49ff-86e4-969f7b3e56ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.642129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbc460-e081-49ff-86e4-969f7b3e56ad admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.652986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4023/8076] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:31.657310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:31.668024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0780d-cd34-438d-aef9-4b60390eb304 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.673135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.673782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.674097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.674372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] PATH_INFO: `/v3/projects/520c8d7744b44016bad9c8e81d778589/users/554f1c7ed01144808efe5657d94308fb/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.690678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=520c8d7744b44016bad9c8e81d778589, user_id=554f1c7ed01144808efe5657d94308fb, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.693366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.706487 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4054/8077] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:31 2022] PUT /identity/v3/projects/520c8d7744b44016bad9c8e81d778589/users/554f1c7ed01144808efe5657d94308fb/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:31.718304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.718609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.718871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.749707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] MFA Rules not processed for user `554f1c7ed01144808efe5657d94308fb`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:31.750971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:31.769138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:31.836941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4024/8078] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:31.854382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:31.856076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:31.930666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9a0a8-dce5-4fe7-b0a0-a4897748cc68 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:31.934731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec0186-ae88-4a38-b195-38124af17e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:31.935348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec0186-ae88-4a38-b195-38124af17e14 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:31.935630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec0186-ae88-4a38-b195-38124af17e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:31.935904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec0186-ae88-4a38-b195-38124af17e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:31.946367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec0186-ae88-4a38-b195-38124af17e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:31.947839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec0186-ae88-4a38-b195-38124af17e14 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:31.959214 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4055/8079] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:34.778608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:34.780110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:34.861863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2eb7ac-9c8d-4c34-bf22-cdf79b62f7d6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:34.866591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:34.867246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:34.867518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:34.867787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] PATH_INFO: `/v3/users/a3260e19b7e14ba784ffb4cf0a6d50e6` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:34.868907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3260e19b7e14ba784ffb4cf0a6d50e6)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:34.870873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:34.919280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3260e19b7e14ba784ffb4cf0a6d50e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:34.926569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3260e19b7e14ba784ffb4cf0a6d50e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:34.927261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3260e19b7e14ba784ffb4cf0a6d50e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:34.935641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3260e19b7e14ba784ffb4cf0a6d50e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:34.936279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3260e19b7e14ba784ffb4cf0a6d50e6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:34.959351 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4025/8080] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:34 2022] DELETE /identity/v3/users/a3260e19b7e14ba784ffb4cf0a6d50e6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:34.964081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:34.964565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:34.964659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:34.988675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:34.990048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:34.998382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:35.061457 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4056/8081] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:35.071688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:35.073137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:35.118657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:35.140063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed705256-2ba7-458d-a11c-8d2d9f52a99a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:35.144515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c46d8-4089-4b7a-a182-0f4b9fafb8da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:35.145114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c46d8-4089-4b7a-a182-0f4b9fafb8da service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.145384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c46d8-4089-4b7a-a182-0f4b9fafb8da service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.145687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c46d8-4089-4b7a-a182-0f4b9fafb8da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.155969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c46d8-4089-4b7a-a182-0f4b9fafb8da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:35.157443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c46d8-4089-4b7a-a182-0f4b9fafb8da service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:35.169713 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4026/8082] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:35.365488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:35.366980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:35.381856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1537a-b613-41a6-828c-f0658d5ed451 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:35.388115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:35.388773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.389200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.389626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] PATH_INFO: `/v3/projects/6e4ce02058134f19802bb869ddf64e22` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.390865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4ce02058134f19802bb869ddf64e22)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:35.393516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:35.456846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4ce02058134f19802bb869ddf64e22', 'request_id': 'req-962f37b2-5253-40ba-8e5f-e32578491bdc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:35.465879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4ce02058134f19802bb869ddf64e22', 'request_id': 'req-962f37b2-5253-40ba-8e5f-e32578491bdc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:35.473693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4ce02058134f19802bb869ddf64e22', 'request_id': 'req-962f37b2-5253-40ba-8e5f-e32578491bdc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:35.494918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] The token cache is being invalidate because project 6e4ce02058134f19802bb869ddf64e22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:35.495214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:35.495872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-962f37b2-5253-40ba-8e5f-e32578491bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:35.497379 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4057/8083] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:35 2022] DELETE /identity/v3/projects/6e4ce02058134f19802bb869ddf64e22 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:19:35.503319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761e239-2b29-4416-bcfe-3935e7c3914b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.504614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761e239-2b29-4416-bcfe-3935e7c3914b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.504614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761e239-2b29-4416-bcfe-3935e7c3914b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.525813 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1761e239-2b29-4416-bcfe-3935e7c3914b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:35.527620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1761e239-2b29-4416-bcfe-3935e7c3914b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:35.608430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4027/8084] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:19:35.609328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112770-0725-4557-aba5-024b74f5c72a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.609693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112770-0725-4557-aba5-024b74f5c72a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.610174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62112770-0725-4557-aba5-024b74f5c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.634367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-62112770-0725-4557-aba5-024b74f5c72a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:35.635833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-62112770-0725-4557-aba5-024b74f5c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:35.647848 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4058/8085] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:35.651262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761e239-2b29-4416-bcfe-3935e7c3914b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:35.662490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761e239-2b29-4416-bcfe-3935e7c3914b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:35.668329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:35.668938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.669215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.669484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.670276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:35.670718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:35.672948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:35.690006 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4028/8086] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:35.693638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62112770-0725-4557-aba5-024b74f5c72a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:35.703963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62112770-0725-4557-aba5-024b74f5c72a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:35.710370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:35.711051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.711538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.711911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.712547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:35.714487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:35.755891 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4059/8087] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:35 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:35.767553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:35.804658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57c2d1-d1ac-4a97-b4ef-e4021e934d59 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:35.930605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:35.931239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.931527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.931807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.932507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:35.935767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:35.962010 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4029/8088] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:35 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:19:35.965946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:35.979025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e49a-f417-4105-96bc-e0430707dd76 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:35.985570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:35.986557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:35.987131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:35.987754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:35.988618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:35.990829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:36.005418 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4060/8089] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:19:36.006900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:36.017305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331f43c-f9df-40eb-846c-ae54d5da2bc4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:36.024883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a6748-c0c6-4c52-8834-3171865de595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:36.025766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6748-c0c6-4c52-8834-3171865de595 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:36.026242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6748-c0c6-4c52-8834-3171865de595 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:36.026695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6748-c0c6-4c52-8834-3171865de595 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:36.027495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a6748-c0c6-4c52-8834-3171865de595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:36.029289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a6748-c0c6-4c52-8834-3171865de595 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:36.042407 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4030/8090] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:19:36.047431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:36.063786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e62c91-35c3-4cbf-a73c-7944ed54ce21 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:36.069600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:36.070400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:36.070783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:36.071120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] PATH_INFO: `/v3/projects/3339bce9ebc84932901bae8a4b14ae8e/users/d233813fc12e4875a526f0a6104e4257/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:36.089066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3339bce9ebc84932901bae8a4b14ae8e, user_id=d233813fc12e4875a526f0a6104e4257, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:36.091790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:36.112801 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4061/8091] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:36 2022] PUT /identity/v3/projects/3339bce9ebc84932901bae8a4b14ae8e/users/d233813fc12e4875a526f0a6104e4257/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:36.124126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45e042-6200-48a0-bd8b-720f44d72eca None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:36.124410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45e042-6200-48a0-bd8b-720f44d72eca None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:36.124694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45e042-6200-48a0-bd8b-720f44d72eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:36.155688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8f45e042-6200-48a0-bd8b-720f44d72eca None None] MFA Rules not processed for user `d233813fc12e4875a526f0a6104e4257`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:36.156993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8f45e042-6200-48a0-bd8b-720f44d72eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:36.197314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:36.234854 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4031/8092] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:36.253799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:36.255698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:36.298811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:36.327403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3489cc-bf0d-47fc-8637-15926af536e7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:36.331475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bbb67-701c-42d2-be0c-a0db27eda329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:36.332115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bbb67-701c-42d2-be0c-a0db27eda329 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:36.332374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bbb67-701c-42d2-be0c-a0db27eda329 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:36.332650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bbb67-701c-42d2-be0c-a0db27eda329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:36.342614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bbb67-701c-42d2-be0c-a0db27eda329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:36.344405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bbb67-701c-42d2-be0c-a0db27eda329 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:36.356078 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4062/8093] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:43.690215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45e042-6200-48a0-bd8b-720f44d72eca None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:43.773710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45e042-6200-48a0-bd8b-720f44d72eca None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:43.779492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:43.780279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:43.780665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:43.781558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] PATH_INFO: `/v3/users/d233813fc12e4875a526f0a6104e4257` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:43.782312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d233813fc12e4875a526f0a6104e4257)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:43.786874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:43.886201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd233813fc12e4875a526f0a6104e4257'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:43.899775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd233813fc12e4875a526f0a6104e4257'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:43.901003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd233813fc12e4875a526f0a6104e4257'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:43.929619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd233813fc12e4875a526f0a6104e4257'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:43.930598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd233813fc12e4875a526f0a6104e4257'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:43.956464 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4032/8094] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:19:43 2022] DELETE /identity/v3/users/d233813fc12e4875a526f0a6104e4257 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:43.960931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:43.961724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:43.962045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:43.986806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:43.989041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:44.070293 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4063/8095] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:44.089750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:44.092093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:44.166936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90cabf-ea88-40f4-a381-6f4710465060 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:44.171323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed569a-1b5a-40b6-8a2d-7249c2574c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:44.171971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed569a-1b5a-40b6-8a2d-7249c2574c7b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:44.172329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed569a-1b5a-40b6-8a2d-7249c2574c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:44.172698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed569a-1b5a-40b6-8a2d-7249c2574c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:44.183751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed569a-1b5a-40b6-8a2d-7249c2574c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:44.188901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed569a-1b5a-40b6-8a2d-7249c2574c7b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:44.193772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:44.201532 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4033/8096] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:44.557639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:44.560783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:44.595484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3149e6-085b-477d-8120-9bf7d42297cd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:44.601203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:44.601880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:44.602148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:44.602425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] PATH_INFO: `/v3/projects/3339bce9ebc84932901bae8a4b14ae8e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:44.603555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3339bce9ebc84932901bae8a4b14ae8e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:44.605438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:44.671098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3339bce9ebc84932901bae8a4b14ae8e', 'request_id': 'req-0b5a0e81-5dcc-4406-96df-879bb0d1253d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:44.685268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3339bce9ebc84932901bae8a4b14ae8e', 'request_id': 'req-0b5a0e81-5dcc-4406-96df-879bb0d1253d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:44.695032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3339bce9ebc84932901bae8a4b14ae8e', 'request_id': 'req-0b5a0e81-5dcc-4406-96df-879bb0d1253d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:44.711264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] The token cache is being invalidate because project 3339bce9ebc84932901bae8a4b14ae8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:19:44.711941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:44.712898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0b5a0e81-5dcc-4406-96df-879bb0d1253d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:19:44.715127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4064/8097] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:19:44 2022] DELETE /identity/v3/projects/3339bce9ebc84932901bae8a4b14ae8e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:44.720793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab9e6-2ab4-4715-973a-018823dd7563 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:44.721884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab9e6-2ab4-4715-973a-018823dd7563 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:44.722112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab9e6-2ab4-4715-973a-018823dd7563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:44.746311 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ea8ab9e6-2ab4-4715-973a-018823dd7563 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:44.748103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ea8ab9e6-2ab4-4715-973a-018823dd7563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:44.826808 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4034/8098] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:44.836043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe866-7980-4924-8a01-edc67d6d12a8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:44.836235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe866-7980-4924-8a01-edc67d6d12a8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:44.836796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe866-7980-4924-8a01-edc67d6d12a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:44.868328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-09dfe866-7980-4924-8a01-edc67d6d12a8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:44.869650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-09dfe866-7980-4924-8a01-edc67d6d12a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:44.881979 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4065/8099] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:44.883277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ab9e6-2ab4-4715-973a-018823dd7563 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:44.893999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ab9e6-2ab4-4715-973a-018823dd7563 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:44.901329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:44.902023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:44.902470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:44.902745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:44.903791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:19:44.904101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:44.906620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:44.927453 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4035/8100] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:19:44.931529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfe866-7980-4924-8a01-edc67d6d12a8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:44.942881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfe866-7980-4924-8a01-edc67d6d12a8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:44.949174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:44.949942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:44.951195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:44.951817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:44.952495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:44.954543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:45.000998 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4066/8101] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:19:44 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:45.002288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:45.013597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1ab3f-fb42-4b89-9fdb-a04a2ce1a97c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:45.077236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:45.077909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:45.078194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:45.078469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:45.079108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:45.082248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:45.105796 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4036/8102] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:19:45 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:19:45.110089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:45.120590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ae36e-b975-42f5-a942-1cbf7e9f4e55 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:45.125256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:45.125884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:45.126173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:45.126448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:45.127076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:45.129224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:45.140467 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4067/8103] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:45.144806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:45.155491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429537b3-7b11-40f2-ad3c-d52c15926c1e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:45.160777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b01fa-172c-4be8-b2d1-9c1dbede9415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:45.161378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b01fa-172c-4be8-b2d1-9c1dbede9415 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:45.161668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b01fa-172c-4be8-b2d1-9c1dbede9415 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:45.162057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b01fa-172c-4be8-b2d1-9c1dbede9415 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:45.162701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b01fa-172c-4be8-b2d1-9c1dbede9415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:45.164323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b01fa-172c-4be8-b2d1-9c1dbede9415 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:45.179457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:45.183096 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4037/8104] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:19:45.190941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296c1a7-a541-479b-9600-4c40c6a1b899 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:45.195849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:45.196511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:45.196795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:45.197072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] PATH_INFO: `/v3/projects/5657f9e3468940da9b58eb2f463a78ab/users/aa1dfe36ace24c7eae7610ba0890cc56/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:45.212711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5657f9e3468940da9b58eb2f463a78ab, user_id=aa1dfe36ace24c7eae7610ba0890cc56, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:45.215380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:45.218606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:45.232277 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4068/8105] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:19:45 2022] PUT /identity/v3/projects/5657f9e3468940da9b58eb2f463a78ab/users/aa1dfe36ace24c7eae7610ba0890cc56/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:19:45.244025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:45.244344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:45.244626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:45.276162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] MFA Rules not processed for user `aa1dfe36ace24c7eae7610ba0890cc56`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:19:45.277570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:45.334657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:19:45.350813 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4038/8106] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:19:45.359993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:45.362250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:45.432337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8894a9-c2c9-46e9-ab24-21051c2402d7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:45.436529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:45.437535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:45.438077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:45.438542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:45.450075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:45.451524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:45.461733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4069/8107] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:48.998229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:48.999733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:49.127843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7400b02-a4ce-442e-831b-831ffcfac66a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:49.133958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:49.134696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:49.135050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:49.135406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:49.137317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:49.153522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:49.155288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:49.180221 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4039/8108] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:50.388702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:50.390959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:50.411753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783536b-7abd-4ae3-b428-14c4bd88c432 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:50.421809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:50.422783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:50.425963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:50.426636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:50.431574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:50.457340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:50.459699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:50.474600 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4070/8109] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:50.653516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:50.655657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:50.675742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6001ca-028e-494a-bddc-a07901315d80 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:50.684309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:50.685247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:50.685729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:50.686342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:50.688971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:50.713841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:50.723346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:50.750382 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4040/8110] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:51.075656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:51.090836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b30f3-9a61-4fba-9806-4756ee955bbc service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:51.095322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:51.096258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:51.096744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:51.097213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:51.099602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:51.258849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:51.262811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:51.280057 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4071/8111] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:19:51.389154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:19:51.404632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101590a7-c17c-4380-84da-cfc1d631781a service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:19:51.415604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:19:51.416443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:19:51.416894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:19:51.417326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:19:51.419396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:19:51.436516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:19:51.438441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:19:51.457405 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4041/8112] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:19:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:01.274680 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=68 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:01.445863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=51 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:17.492703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:17.494409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:17.523644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d031c95-fdd9-4152-85e8-0b0f31aa6513 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:17.543850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:17.544559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:17.544912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:17.545264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:17.550671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:17.582725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:17.585145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:17.617471 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4072/8113] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:20.646560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:20.675247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf1f59-a265-4681-b46b-336b718a70a9 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:20.692141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:20.694279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:20.694747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:20.695176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:20.704045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:20.739431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:20.741702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:20.760457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4042/8114] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:27.557161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:27.573323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3652d-85ac-443e-ac8a-cd35443302b8 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:27.579387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fce03-b755-4916-9055-cb6d328a87f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:27.580329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fce03-b755-4916-9055-cb6d328a87f8 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:27.580664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fce03-b755-4916-9055-cb6d328a87f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:27.580945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fce03-b755-4916-9055-cb6d328a87f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:27.598700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cb6fce03-b755-4916-9055-cb6d328a87f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:27.728689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fce03-b755-4916-9055-cb6d328a87f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:27.730823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fce03-b755-4916-9055-cb6d328a87f8 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:27.769911 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4073/8115] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:30.750534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:31.213475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:31.215066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:31.296194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564049a-fecf-41a3-9ed1-870c981e4217 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:31.301619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:31.302299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:31.302580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:31.302855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] PATH_INFO: `/v3/users/93648515d8fb42a7aaf61e245db0e07d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:31.304150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93648515d8fb42a7aaf61e245db0e07d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:31.306190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:31.384361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93648515d8fb42a7aaf61e245db0e07d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:31.391597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93648515d8fb42a7aaf61e245db0e07d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:31.392879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93648515d8fb42a7aaf61e245db0e07d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:31.400992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93648515d8fb42a7aaf61e245db0e07d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:31.401713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93648515d8fb42a7aaf61e245db0e07d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:31.420758 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4043/8116] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:20:31 2022] DELETE /identity/v3/users/93648515d8fb42a7aaf61e245db0e07d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:31.425622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:31.426166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:31.426384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:31.446808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:31.448341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:31.516456 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4074/8117] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:31.526859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:31.528556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:31.592361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaba478-f380-4317-8f2b-692252c13f77 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:31.597417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:31.598055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:31.598337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:31.598609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:31.608407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:31.609855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:31.620936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4044/8118] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:31.943385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:31.948795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:31.988821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f4b02-ece9-493b-891d-caf271e81c45 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:31.998184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:31.999090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:32.004995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:32.005291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] PATH_INFO: `/v3/projects/fe0d9347a2d64323bb563b6f1f0b0cd6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:32.009175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe0d9347a2d64323bb563b6f1f0b0cd6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:32.016207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:32.120038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe0d9347a2d64323bb563b6f1f0b0cd6', 'request_id': 'req-40a65cbb-4bf9-47f8-9b4b-959f64923f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.137309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe0d9347a2d64323bb563b6f1f0b0cd6', 'request_id': 'req-40a65cbb-4bf9-47f8-9b4b-959f64923f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.153958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe0d9347a2d64323bb563b6f1f0b0cd6', 'request_id': 'req-40a65cbb-4bf9-47f8-9b4b-959f64923f21'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.178285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] The token cache is being invalidate because project fe0d9347a2d64323bb563b6f1f0b0cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:20:32.178859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.179698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.181790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4075/8119] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:20:31 2022] DELETE /identity/v3/projects/fe0d9347a2d64323bb563b6f1f0b0cd6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:32.185337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:32.197850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506631-6a95-40af-af6a-5b20d2740f6d service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:32.227695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:32.287000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:32.289254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:32.290624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:32.290624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] PATH_INFO: `/v3/users/9e2e31abcac04c0bb71fbed775b80d6f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:32.291814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e2e31abcac04c0bb71fbed775b80d6f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:32.293854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:32.401384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e2e31abcac04c0bb71fbed775b80d6f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.418662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e2e31abcac04c0bb71fbed775b80d6f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.424305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e2e31abcac04c0bb71fbed775b80d6f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.436250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e2e31abcac04c0bb71fbed775b80d6f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.436454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61812b87-c695-448a-8fdb-7a4a2ba4744a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e2e31abcac04c0bb71fbed775b80d6f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:32.479501 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4045/8120] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:20:32 2022] DELETE /identity/v3/users/9e2e31abcac04c0bb71fbed775b80d6f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:33.115591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:33.117308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:33.132530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a65cbb-4bf9-47f8-9b4b-959f64923f21 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:33.142731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:33.221788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:33.224478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.225247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.225363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] PATH_INFO: `/v3/projects/2ba991d7c6a946739e76f00bbbf420ba` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.226645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba991d7c6a946739e76f00bbbf420ba)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:33.229029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:33.325352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba991d7c6a946739e76f00bbbf420ba', 'request_id': 'req-18b02a5a-4a93-49ea-b740-09587892e655'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:33.334581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba991d7c6a946739e76f00bbbf420ba', 'request_id': 'req-18b02a5a-4a93-49ea-b740-09587892e655'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:33.344194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba991d7c6a946739e76f00bbbf420ba', 'request_id': 'req-18b02a5a-4a93-49ea-b740-09587892e655'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:33.388488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] The token cache is being invalidate because project 2ba991d7c6a946739e76f00bbbf420ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:20:33.389719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:33.392423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-18b02a5a-4a93-49ea-b740-09587892e655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:33.400851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc0f36-05a1-4cd8-9d4b-184ef4ac23cc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.401380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc0f36-05a1-4cd8-9d4b-184ef4ac23cc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.402011 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4076/8121] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:20:33 2022] DELETE /identity/v3/projects/2ba991d7c6a946739e76f00bbbf420ba => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:33.403346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc0f36-05a1-4cd8-9d4b-184ef4ac23cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.429507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e5dc0f36-05a1-4cd8-9d4b-184ef4ac23cc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:33.430993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e5dc0f36-05a1-4cd8-9d4b-184ef4ac23cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:33.549112 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4046/8122] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:33.557696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88b59b-4b90-4955-9f3f-44875bf85335 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.558016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88b59b-4b90-4955-9f3f-44875bf85335 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.558297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88b59b-4b90-4955-9f3f-44875bf85335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.600282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6e88b59b-4b90-4955-9f3f-44875bf85335 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:33.602626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6e88b59b-4b90-4955-9f3f-44875bf85335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:33.620758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc0f36-05a1-4cd8-9d4b-184ef4ac23cc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:33.621172 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4077/8123] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:33.635537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc0f36-05a1-4cd8-9d4b-184ef4ac23cc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:33.643434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:33.644635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.645487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.646031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.646923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:20:33.648011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:33.650089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:33.675804 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4047/8124] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:20:33.676476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88b59b-4b90-4955-9f3f-44875bf85335 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:33.688869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88b59b-4b90-4955-9f3f-44875bf85335 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:33.694946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:33.695573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.696170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.696620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.697331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:33.699525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:33.743704 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4078/8125] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:20:33 2022] POST /identity/v3/projects => generated 389 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:20:33.748663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:33.761065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae5425-ba5e-456e-85c2-6613e7babddb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:33.804613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:33.843977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:33.845512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.845809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.846131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.846769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:33.848914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:33.875689 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4048/8126] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:20:33 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:20:33.880395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:33.896186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21b921-3ce2-40a3-8bed-78156772821b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:33.912287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:33.912958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.913243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.913520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.914403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:33.921055 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:33.933884 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4079/8127] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:20:33.937112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:33.949662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ad34-2168-4aae-aa08-1bcb847cdaa3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:33.956235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc7d23-24c4-44a6-888a-c804cfa6d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:33.957223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc7d23-24c4-44a6-888a-c804cfa6d7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:33.957684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc7d23-24c4-44a6-888a-c804cfa6d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:33.958171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc7d23-24c4-44a6-888a-c804cfa6d7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:33.960562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc7d23-24c4-44a6-888a-c804cfa6d7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:33.960562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc7d23-24c4-44a6-888a-c804cfa6d7c7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:33.988226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:33.988688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4049/8128] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:20:34.000049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d1220-a310-4d55-8c90-cff4fb3a8d7e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:34.007932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:34.008632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.008907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.009186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] PATH_INFO: `/v3/projects/371635a1edd041828749d00c5526197c/users/f5afa360fc4c40288b65c437519c2a7f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.028269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=371635a1edd041828749d00c5526197c, user_id=f5afa360fc4c40288b65c437519c2a7f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:34.032817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:34.053621 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4080/8129] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:20:33 2022] PUT /identity/v3/projects/371635a1edd041828749d00c5526197c/users/f5afa360fc4c40288b65c437519c2a7f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:34.068055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de06b7-fe05-493d-88c6-76c43312703f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.068287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de06b7-fe05-493d-88c6-76c43312703f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.068399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de06b7-fe05-493d-88c6-76c43312703f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.117559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-87de06b7-fe05-493d-88c6-76c43312703f None None] MFA Rules not processed for user `f5afa360fc4c40288b65c437519c2a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:34.119084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-87de06b7-fe05-493d-88c6-76c43312703f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:34.229122 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4050/8130] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:34.234252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:34.295278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:34.364834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12178-448a-43b0-845b-f1b1847a4e3a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:34.371611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:34.372398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.372631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.372935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.373864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:34.383750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:34.434028 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4081/8131] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:20:34.437684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de06b7-fe05-493d-88c6-76c43312703f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:34.454686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de06b7-fe05-493d-88c6-76c43312703f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:34.558621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:34.559230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.560899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.561181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.561865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:34.568419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:34.620047 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4051/8132] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:20:34 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:20:34.621604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:34.636581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b291-91e0-43e0-8313-946415610902 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:34.652150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:34.653154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.653543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.653972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.654731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:34.657723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:34.675579 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4082/8133] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:20:34.680444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:34.700646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610e609-3a9d-4cca-9baf-6823084c01ff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:34.712761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64883ce6-a584-4f4b-bb3f-0fc3171762d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:34.713462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64883ce6-a584-4f4b-bb3f-0fc3171762d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.713681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64883ce6-a584-4f4b-bb3f-0fc3171762d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.714004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64883ce6-a584-4f4b-bb3f-0fc3171762d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.714668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64883ce6-a584-4f4b-bb3f-0fc3171762d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:34.716324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64883ce6-a584-4f4b-bb3f-0fc3171762d0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:34.728935 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4052/8134] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:20:34.733605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:34.747127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cdd18-ec6d-4f2c-86be-e196fa268532 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:34.757672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:34.758606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.759003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.759343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] PATH_INFO: `/v3/projects/52b6406d45f34d02a69acca219c02c4c/users/9fdd78391cf6435eb54ea4974ecd0e39/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.779910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b6406d45f34d02a69acca219c02c4c, user_id=9fdd78391cf6435eb54ea4974ecd0e39, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:34.783952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:34.803764 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4083/8135] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:20:34 2022] PUT /identity/v3/projects/52b6406d45f34d02a69acca219c02c4c/users/9fdd78391cf6435eb54ea4974ecd0e39/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:34.814649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa8c5b-c9a9-48ea-9d45-8fbff1e99436 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:34.814940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa8c5b-c9a9-48ea-9d45-8fbff1e99436 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:34.815224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa8c5b-c9a9-48ea-9d45-8fbff1e99436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:34.851624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d0fa8c5b-c9a9-48ea-9d45-8fbff1e99436 None None] MFA Rules not processed for user `9fdd78391cf6435eb54ea4974ecd0e39`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:34.853051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d0fa8c5b-c9a9-48ea-9d45-8fbff1e99436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:34.858441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:34.989044 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4053/8136] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:20:34.993479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:35.103443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b5c71-f4fe-468f-bef1-728aa9c3bfd8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:35.112149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:35.112826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.113117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.113395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:35.115755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:35.117555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:35.154526 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4084/8137] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:20:34 2022] POST /identity/v3/projects => generated 389 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:20:35.155753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa8c5b-c9a9-48ea-9d45-8fbff1e99436 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:35.171072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa8c5b-c9a9-48ea-9d45-8fbff1e99436 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:35.260432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:35.261164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.261525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.261812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:35.262478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:35.264453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:35.291864 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4054/8138] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:20:35 2022] POST /identity/v3/users => generated 301 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:20:35.296008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:35.312532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12145ed1-8a29-4b78-89ab-c182274de0b3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:35.319533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:35.320949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.321466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.322114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:35.322999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:35.324850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:35.345729 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4085/8139] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:20:35.349240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:35.376508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353fe12-a2f0-49fd-a95c-bcd1ef7fcfcd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:35.385001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:35.387434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.387727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.388012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] PATH_INFO: `/v3/projects/0d3b288cfec049f0ae3e8bbf69c6fe65/users/839c2e5c2fc9402e8a8ceaa32afabc41/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:35.410551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3b288cfec049f0ae3e8bbf69c6fe65, user_id=839c2e5c2fc9402e8a8ceaa32afabc41, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:35.413353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:35.451867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4055/8140] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:20:35 2022] PUT /identity/v3/projects/0d3b288cfec049f0ae3e8bbf69c6fe65/users/839c2e5c2fc9402e8a8ceaa32afabc41/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:35.456756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:35.468306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbab7f-c17c-4b7e-8cbb-c5e10d601c89 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:35.546627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cad43-bb80-41e9-a02e-2a1f482b1137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:35.547365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cad43-bb80-41e9-a02e-2a1f482b1137 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.547788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cad43-bb80-41e9-a02e-2a1f482b1137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.548286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cad43-bb80-41e9-a02e-2a1f482b1137 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:35.549157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cad43-bb80-41e9-a02e-2a1f482b1137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:35.551400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cad43-bb80-41e9-a02e-2a1f482b1137 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:35.553249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:35.578469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:35.589066 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4086/8141] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:20:35.597970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39b65b-3cad-4543-8032-3f46405e1771 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:35.613720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382325d-645e-4559-b415-88547de8768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:35.614891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382325d-645e-4559-b415-88547de8768c admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.615341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382325d-645e-4559-b415-88547de8768c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.615807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382325d-645e-4559-b415-88547de8768c admin admin] PATH_INFO: `/v3/projects/0d3b288cfec049f0ae3e8bbf69c6fe65/users/839c2e5c2fc9402e8a8ceaa32afabc41/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:35.619691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382325d-645e-4559-b415-88547de8768c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3b288cfec049f0ae3e8bbf69c6fe65, user_id=839c2e5c2fc9402e8a8ceaa32afabc41, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:35.622413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382325d-645e-4559-b415-88547de8768c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:35.645961 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4056/8142] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:20:35 2022] PUT /identity/v3/projects/0d3b288cfec049f0ae3e8bbf69c6fe65/users/839c2e5c2fc9402e8a8ceaa32afabc41/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:35.659858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.660289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.660675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:35.730927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] MFA Rules not processed for user `839c2e5c2fc9402e8a8ceaa32afabc41`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:35.731651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:35.825816 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4087/8143] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:35 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:35.851858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382325d-645e-4559-b415-88547de8768c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:35.854101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d382325d-645e-4559-b415-88547de8768c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:35.952513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:35.984065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382325d-645e-4559-b415-88547de8768c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:35.991809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbb23a-3310-415c-909d-110408d6c217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:35.992427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbb23a-3310-415c-909d-110408d6c217 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:35.992708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbb23a-3310-415c-909d-110408d6c217 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:35.992984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbb23a-3310-415c-909d-110408d6c217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:36.076159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbb23a-3310-415c-909d-110408d6c217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:36.077799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbb23a-3310-415c-909d-110408d6c217 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:36.089907 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4057/8144] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:38.743254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:38.744968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:38.818334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9db8b-53f2-4ba3-92ee-908ba34c5169 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:38.823108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:38.823997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:38.824139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:38.824419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:38.837076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:38.838677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:38.851118 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4088/8145] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:46.084427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=30 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:46.317409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:46.317733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:46.318050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:46.341070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] MFA Rules not processed for user `f5afa360fc4c40288b65c437519c2a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:46.342382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:46.354389 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4058/8146] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:47.256750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:47.260819 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:47.290739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36e0b-c195-474d-bada-7cd6ed00557c service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:47.299516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3dfe5-d925-4f6c-8376-ef2552259049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:47.301284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3dfe5-d925-4f6c-8376-ef2552259049 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:47.304161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3dfe5-d925-4f6c-8376-ef2552259049 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:47.304466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3dfe5-d925-4f6c-8376-ef2552259049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:47.310678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d6b3dfe5-d925-4f6c-8376-ef2552259049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:47.331826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3dfe5-d925-4f6c-8376-ef2552259049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:47.333657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3dfe5-d925-4f6c-8376-ef2552259049 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:47.347228 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4089/8147] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:48.796338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:48.798502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:48.959029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ac7d-697a-4148-b187-fd22bf298d09 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:48.978284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:48.981738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:48.982274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:48.982875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] PATH_INFO: `/v3/users/aa1dfe36ace24c7eae7610ba0890cc56` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:48.984642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1dfe36ace24c7eae7610ba0890cc56)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:48.987358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:49.075187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1dfe36ace24c7eae7610ba0890cc56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:49.093921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1dfe36ace24c7eae7610ba0890cc56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:49.095149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1dfe36ace24c7eae7610ba0890cc56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:49.110528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1dfe36ace24c7eae7610ba0890cc56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:49.111543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1dfe36ace24c7eae7610ba0890cc56'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:49.151481 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4059/8148] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:20:48 2022] DELETE /identity/v3/users/aa1dfe36ace24c7eae7610ba0890cc56 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:49.155808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:49.156714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:49.158060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:49.182583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:49.184478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:49.331805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4090/8149] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:20:49.359380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:49.361669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:49.447483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfa9b2-5101-4d41-b777-7ceceff8273f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:49.451610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aa6ca-1222-4cb5-92ad-5a9a78e522ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:49.452478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa6ca-1222-4cb5-92ad-5a9a78e522ef service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:49.452910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa6ca-1222-4cb5-92ad-5a9a78e522ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:49.453354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa6ca-1222-4cb5-92ad-5a9a78e522ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:49.464999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aa6ca-1222-4cb5-92ad-5a9a78e522ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:49.467646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aa6ca-1222-4cb5-92ad-5a9a78e522ef service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:49.480331 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4060/8150] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:20:49.980771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:49.983006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:50.011548 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4049ae9-ab89-4385-b674-ca55f5612bc8 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:50.026970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:50.027931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.028309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.028679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] PATH_INFO: `/v3/projects/5657f9e3468940da9b58eb2f463a78ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.029953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5657f9e3468940da9b58eb2f463a78ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:50.033121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:50.041146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:50.128053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5657f9e3468940da9b58eb2f463a78ab', 'request_id': 'req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:50.139671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5657f9e3468940da9b58eb2f463a78ab', 'request_id': 'req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:50.153136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5657f9e3468940da9b58eb2f463a78ab', 'request_id': 'req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:50.184871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] The token cache is being invalidate because project 5657f9e3468940da9b58eb2f463a78ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:20:50.185543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:50.186416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-bd285f06-ba43-4d29-ac29-8ba6a3b26e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:20:50.189669 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4091/8151] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:20:49 2022] DELETE /identity/v3/projects/5657f9e3468940da9b58eb2f463a78ab => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:50.195071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8d848-e2a5-4093-acf3-f86a4b20ba1c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.195602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8d848-e2a5-4093-acf3-f86a4b20ba1c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.196151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8d848-e2a5-4093-acf3-f86a4b20ba1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.243322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-02b8d848-e2a5-4093-acf3-f86a4b20ba1c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:50.245469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-02b8d848-e2a5-4093-acf3-f86a4b20ba1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:50.263862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:20:50.336733 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4061/8152] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:20:50.343969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56267ce6-52b7-4018-9135-b5190cbba231 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.344497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56267ce6-52b7-4018-9135-b5190cbba231 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.344994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56267ce6-52b7-4018-9135-b5190cbba231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.380154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-56267ce6-52b7-4018-9135-b5190cbba231 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:50.382308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-56267ce6-52b7-4018-9135-b5190cbba231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:50.397266 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4092/8153] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:50.401257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8d848-e2a5-4093-acf3-f86a4b20ba1c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:50.416438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8d848-e2a5-4093-acf3-f86a4b20ba1c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:50.429875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:50.430622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.432238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.432755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.433796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:20:50.434321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:50.436213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:50.463692 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4062/8154] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:20:50.465094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56267ce6-52b7-4018-9135-b5190cbba231 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:50.476839 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56267ce6-52b7-4018-9135-b5190cbba231 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:50.486302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:50.487861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.488589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.488872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.489734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:50.491920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:50.540234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:50.552163 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4093/8155] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:20:50 2022] POST /identity/v3/projects => generated 389 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:20:50.557001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732b0c-b2cd-4394-ae42-2bfaf4e8ada5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:50.709614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:50.709614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.709614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.709614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.710585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:50.710585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:50.743728 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4063/8156] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:20:50 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:20:50.747678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:50.762014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8f324-e331-4f6a-8f78-439a4a24aa61 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:50.769482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:50.769617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.769956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.770368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.771270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:50.780428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:50.794968 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4094/8157] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:20:50.799734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:50.810693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48536cbe-2539-4b53-8e52-4dfdd1669101 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:50.818115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe6f6-a4fa-4372-a21a-61b32ab19264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:50.818862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe6f6-a4fa-4372-a21a-61b32ab19264 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.819160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe6f6-a4fa-4372-a21a-61b32ab19264 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.820057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe6f6-a4fa-4372-a21a-61b32ab19264 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.820057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fe6f6-a4fa-4372-a21a-61b32ab19264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:50.822064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fe6f6-a4fa-4372-a21a-61b32ab19264 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:50.836771 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4064/8158] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:20:50.840804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:50.862983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd21529-0192-4bd8-9c49-9261eed85dfd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:50.870165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:50.872441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.872892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.873578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] PATH_INFO: `/v3/projects/22f64891769f4bf288a75ea3c167bbce/users/c13aa4386c124c629be81bbf2843472d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:50.903280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f64891769f4bf288a75ea3c167bbce, user_id=c13aa4386c124c629be81bbf2843472d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:50.906280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:50.933941 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4095/8159] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:20:50 2022] PUT /identity/v3/projects/22f64891769f4bf288a75ea3c167bbce/users/c13aa4386c124c629be81bbf2843472d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:20:50.944346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edd3f3-2688-4a90-86ca-e00116eba400 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:50.945021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edd3f3-2688-4a90-86ca-e00116eba400 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:50.945463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edd3f3-2688-4a90-86ca-e00116eba400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:51.018883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e7edd3f3-2688-4a90-86ca-e00116eba400 None None] MFA Rules not processed for user `c13aa4386c124c629be81bbf2843472d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:20:51.020249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e7edd3f3-2688-4a90-86ca-e00116eba400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:51.199143 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4065/8160] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:20:51.228775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:20:51.230946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:20:51.365914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d23f3b-7505-4c7e-8cfb-ed2958c7d9cf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:20:51.369796 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:20:51.370517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:20:51.371260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:20:51.371644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:20:51.383379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:20:51.385163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:20:51.399245 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4096/8161] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:20:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:01.188951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:01.392508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:14.876088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edd3f3-2688-4a90-86ca-e00116eba400 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:14.937361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:14.939321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:15.056237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edd3f3-2688-4a90-86ca-e00116eba400 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:15.067277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:15.067998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:15.068287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:15.068570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] PATH_INFO: `/v3/users/c13aa4386c124c629be81bbf2843472d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:15.070220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13aa4386c124c629be81bbf2843472d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:15.072707 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:15.121409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5994b-13e5-4cb3-b702-81a5011c70d0 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:15.139207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f6763-9457-416e-b9fc-65e15a8b69f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:15.139984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f6763-9457-416e-b9fc-65e15a8b69f4 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:15.140198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f6763-9457-416e-b9fc-65e15a8b69f4 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:15.140570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f6763-9457-416e-b9fc-65e15a8b69f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:15.152598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-282f6763-9457-416e-b9fc-65e15a8b69f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:15.168112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13aa4386c124c629be81bbf2843472d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:15.189209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13aa4386c124c629be81bbf2843472d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:15.194332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13aa4386c124c629be81bbf2843472d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:15.197805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f6763-9457-416e-b9fc-65e15a8b69f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:15.202107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f6763-9457-416e-b9fc-65e15a8b69f4 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:15.221305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13aa4386c124c629be81bbf2843472d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:15.221990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13aa4386c124c629be81bbf2843472d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:15.268749 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4097/8162] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:21:15.310388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc64d0-ea64-44fe-a49b-4dd7de86e569 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:15.310818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc64d0-ea64-44fe-a49b-4dd7de86e569 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:15.310961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc64d0-ea64-44fe-a49b-4dd7de86e569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:15.320949 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4066/8163] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:14 2022] DELETE /identity/v3/users/c13aa4386c124c629be81bbf2843472d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:15.337167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5cbc64d0-ea64-44fe-a49b-4dd7de86e569 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:15.338906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5cbc64d0-ea64-44fe-a49b-4dd7de86e569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:15.504533 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4098/8164] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:15.526413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:15.528815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:15.581274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65487-a7d6-4665-94be-c17680e2ca80 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:15.581593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65487-a7d6-4665-94be-c17680e2ca80 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:15.581889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65487-a7d6-4665-94be-c17680e2ca80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:15.612517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ecd65487-a7d6-4665-94be-c17680e2ca80 None None] MFA Rules not processed for user `b4791d13ab1c45bf8fabec70191426da`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:15.616277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ecd65487-a7d6-4665-94be-c17680e2ca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:15.637356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb34b-1673-4445-8833-ddd61ebe459e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:15.643668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:15.644548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:15.646259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:15.646628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:15.659165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:15.660729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:15.675085 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4067/8165] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:15.772840 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4099/8166] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:21:16.065577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:16.067024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:16.091848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df415f3-4a6e-4598-a001-fdb51c46c92f service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:16.098207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:16.099249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.099760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.100447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] PATH_INFO: `/v3/projects/22f64891769f4bf288a75ea3c167bbce` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.101956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f64891769f4bf288a75ea3c167bbce)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:16.104680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:16.155360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:16.211761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f64891769f4bf288a75ea3c167bbce', 'request_id': 'req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:16.220537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f64891769f4bf288a75ea3c167bbce', 'request_id': 'req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:16.233472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f64891769f4bf288a75ea3c167bbce', 'request_id': 'req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:16.258527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] The token cache is being invalidate because project 22f64891769f4bf288a75ea3c167bbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:16.259089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:16.260087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9cbf13ca-13dc-4d1a-8cb9-73fa273016e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:16.267592 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4068/8167] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:16 2022] DELETE /identity/v3/projects/22f64891769f4bf288a75ea3c167bbce => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:16.273479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3135-dabe-43c2-94fd-9cbeadc5203b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.274426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3135-dabe-43c2-94fd-9cbeadc5203b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.274955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3135-dabe-43c2-94fd-9cbeadc5203b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.304692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-91ff3135-dabe-43c2-94fd-9cbeadc5203b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:16.305951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-91ff3135-dabe-43c2-94fd-9cbeadc5203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:16.330191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:16.412016 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4100/8168] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:16.422474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194674-908f-44c3-931f-b5da46b94394 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.423872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194674-908f-44c3-931f-b5da46b94394 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.424438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194674-908f-44c3-931f-b5da46b94394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.473878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-43194674-908f-44c3-931f-b5da46b94394 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:16.475037 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-43194674-908f-44c3-931f-b5da46b94394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:16.491381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4069/8169] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:16.492460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff3135-dabe-43c2-94fd-9cbeadc5203b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:16.505786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff3135-dabe-43c2-94fd-9cbeadc5203b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:16.511624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:16.512310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.512593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.512868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.513522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:21:16.513836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:16.517076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:16.532747 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4101/8170] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:21:16.538898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43194674-908f-44c3-931f-b5da46b94394 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:16.556719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43194674-908f-44c3-931f-b5da46b94394 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:16.568526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:16.569953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.570494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.570942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.571775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:16.574103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:16.653976 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4070/8171] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:16 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:16.655904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:16.677358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a2086-af06-4865-8775-9c445638f23a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:16.791380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:16.792365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.792656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.793218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.794166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:16.797397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:16.831727 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4102/8172] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:16 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:16.835456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:16.863207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97337a4a-5315-4f5c-8799-f64178943037 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:16.870066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:16.870686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.870975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.871253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.872358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:16.874007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:16.897954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:16.898563 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4071/8173] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:16.909508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14754c-efd7-4c4c-94b0-67b578b9b319 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:16.915002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb45b-52aa-471f-af52-8e5e00a2c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:16.915835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb45b-52aa-471f-af52-8e5e00a2c525 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.916279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb45b-52aa-471f-af52-8e5e00a2c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.916711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb45b-52aa-471f-af52-8e5e00a2c525 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:16.917524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb45b-52aa-471f-af52-8e5e00a2c525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:16.919636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb45b-52aa-471f-af52-8e5e00a2c525 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:16.934688 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4103/8174] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:21:16.939935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:16.955275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a642-ed7e-4c1f-a12d-3bbee26c3bd0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:16.962535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:16.963365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:16.963815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:16.963984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] PATH_INFO: `/v3/projects/3aa1c538b98b4df389feb984fd908888/users/ef2da4748ba2458c9ab8758d24ca53ba/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:17.019217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa1c538b98b4df389feb984fd908888, user_id=ef2da4748ba2458c9ab8758d24ca53ba, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:17.022452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:17.046388 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4072/8175] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:16 2022] PUT /identity/v3/projects/3aa1c538b98b4df389feb984fd908888/users/ef2da4748ba2458c9ab8758d24ca53ba/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:17.057222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c9165-da55-4041-9fbc-fbb7be4b7188 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:17.057556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c9165-da55-4041-9fbc-fbb7be4b7188 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:17.058022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c9165-da55-4041-9fbc-fbb7be4b7188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:17.105249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bf9c9165-da55-4041-9fbc-fbb7be4b7188 None None] MFA Rules not processed for user `ef2da4748ba2458c9ab8758d24ca53ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:17.106902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bf9c9165-da55-4041-9fbc-fbb7be4b7188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:17.215209 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4104/8176] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:17.219482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:17.341198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb8701-e48c-4f7a-80b3-69971f9f12af admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:17.349009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:17.349923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:17.350449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:17.351889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:17.352807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:17.355828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:17.404542 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4073/8177] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:17 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:21:17.409114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c9165-da55-4041-9fbc-fbb7be4b7188 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:17.421501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c9165-da55-4041-9fbc-fbb7be4b7188 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:17.464245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:17.556173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:17.558547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:17.558941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:17.559322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:17.560193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:17.562869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:17.615130 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4105/8178] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:17 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:17.621551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:17.670691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e7ea3-e4f1-45e6-9917-0654cfbcdebf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:17.677239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:17.678147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:17.678726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:17.679100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:17.679849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:17.681566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:17.712541 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4074/8179] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:17.716956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:17.731814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419dbd7-cf56-41b5-a1f3-0ddca95fb41b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:17.746252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:17.746950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:17.747240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:17.747691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] PATH_INFO: `/v3/projects/e17e5bb076834605afaadcfd58f7c633/users/18efcb0d138443409dfb59ccbb480197/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:17.772205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17e5bb076834605afaadcfd58f7c633, user_id=18efcb0d138443409dfb59ccbb480197, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:17.775508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:17.802340 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4106/8180] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:17 2022] PUT /identity/v3/projects/e17e5bb076834605afaadcfd58f7c633/users/18efcb0d138443409dfb59ccbb480197/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:21:17.803343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:17.820389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801369eb-9df2-4695-8a81-e3a9ee15a34d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:17.854634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:17.944314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f689a-2c82-4399-ada5-8f5c4c44a7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:17.945185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f689a-2c82-4399-ada5-8f5c4c44a7bf admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:17.945543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f689a-2c82-4399-ada5-8f5c4c44a7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:17.945923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f689a-2c82-4399-ada5-8f5c4c44a7bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:17.946646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f689a-2c82-4399-ada5-8f5c4c44a7bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:17.962098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f689a-2c82-4399-ada5-8f5c4c44a7bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:17.993070 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4075/8181] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:17.997308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:18.008733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773786b-a7f2-4f8f-80a3-08a9e0808d49 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:18.021445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:18.022169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:18.022514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:18.022733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] PATH_INFO: `/v3/projects/e17e5bb076834605afaadcfd58f7c633/users/18efcb0d138443409dfb59ccbb480197/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:18.039014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17e5bb076834605afaadcfd58f7c633, user_id=18efcb0d138443409dfb59ccbb480197, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:18.041709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:18.083702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a25b2-dbfd-49b7-ab19-174dfd9373c4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:18.083995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a25b2-dbfd-49b7-ab19-174dfd9373c4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:18.084275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a25b2-dbfd-49b7-ab19-174dfd9373c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:18.091868 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4107/8182] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:17 2022] PUT /identity/v3/projects/e17e5bb076834605afaadcfd58f7c633/users/18efcb0d138443409dfb59ccbb480197/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:18.123799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-415a25b2-dbfd-49b7-ab19-174dfd9373c4 None None] MFA Rules not processed for user `18efcb0d138443409dfb59ccbb480197`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:18.125664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-415a25b2-dbfd-49b7-ab19-174dfd9373c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:18.291387 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4076/8183] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:18.303347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:18.305173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:18.474194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11950400-155b-42cb-bc5b-413f6bbceabb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:18.484659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:18.485384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:18.485670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:18.485985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:18.515194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:18.518278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:18.539544 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4108/8184] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:21:18.717379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a25b2-dbfd-49b7-ab19-174dfd9373c4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:18.732447 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a25b2-dbfd-49b7-ab19-174dfd9373c4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:18.740644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a1697-0afb-42f3-bccb-2cb8873f5798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:18.741363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a1697-0afb-42f3-bccb-2cb8873f5798 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:18.741818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a1697-0afb-42f3-bccb-2cb8873f5798 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:18.742231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a1697-0afb-42f3-bccb-2cb8873f5798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:18.875767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a1697-0afb-42f3-bccb-2cb8873f5798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:18.877250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a1697-0afb-42f3-bccb-2cb8873f5798 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:18.895395 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4077/8185] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:23.914403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:24.025418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57caf2a6-e45f-4760-b052-9285a98cf90b service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:24.031776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:24.032710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:24.033159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:24.033624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] PATH_INFO: `/v3/users/ef2da4748ba2458c9ab8758d24ca53ba` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:24.035054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2da4748ba2458c9ab8758d24ca53ba)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:24.037251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:24.063087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:24.114139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2da4748ba2458c9ab8758d24ca53ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:24.132759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2da4748ba2458c9ab8758d24ca53ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:24.143635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2da4748ba2458c9ab8758d24ca53ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:24.159757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2da4748ba2458c9ab8758d24ca53ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:24.160549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2da4748ba2458c9ab8758d24ca53ba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:24.189602 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4109/8186] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:23 2022] DELETE /identity/v3/users/ef2da4748ba2458c9ab8758d24ca53ba => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:24.191702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:24.192767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:24.193465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:24.226096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:24.227591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:24.274139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:24.331672 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4078/8187] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:21:24.347330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:24.349051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:24.453506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108a8a-89e6-419d-b8c7-131bb678a064 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:24.462233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:24.463278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:24.463793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:24.464291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:24.476396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:24.477916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:24.491717 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4110/8188] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:24.976125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:24.977586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:24.991003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c02671-a4ca-49ff-82dd-1134726ec4d3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:24.998773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:25.000685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:25.001139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:25.001576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] PATH_INFO: `/v3/projects/3aa1c538b98b4df389feb984fd908888` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:25.003814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa1c538b98b4df389feb984fd908888)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:25.006104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:25.108873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa1c538b98b4df389feb984fd908888', 'request_id': 'req-25c3d90f-5b17-4fb9-820c-54ca226469d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.116783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa1c538b98b4df389feb984fd908888', 'request_id': 'req-25c3d90f-5b17-4fb9-820c-54ca226469d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.127033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa1c538b98b4df389feb984fd908888', 'request_id': 'req-25c3d90f-5b17-4fb9-820c-54ca226469d5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.143612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] The token cache is being invalidate because project 3aa1c538b98b4df389feb984fd908888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:25.143907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.144664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.150808 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4079/8189] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:24 2022] DELETE /identity/v3/projects/3aa1c538b98b4df389feb984fd908888 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:25.152221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:25.166908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf1aec-6c00-43ca-91ec-658299d8caf4 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:25.257555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:25.258356 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:25.258645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:25.259077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] PATH_INFO: `/v3/users/18efcb0d138443409dfb59ccbb480197` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:25.260311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18efcb0d138443409dfb59ccbb480197)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:25.262381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:25.317845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:25.332551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18efcb0d138443409dfb59ccbb480197'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.348664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18efcb0d138443409dfb59ccbb480197'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.363751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18efcb0d138443409dfb59ccbb480197'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.372193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18efcb0d138443409dfb59ccbb480197'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.373077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-efbe843d-b443-4638-8c4e-82e1fe583886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18efcb0d138443409dfb59ccbb480197'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.383666 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4111/8190] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:25 2022] DELETE /identity/v3/users/18efcb0d138443409dfb59ccbb480197 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:25.758554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:25.760155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:25.773948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3d90f-5b17-4fb9-820c-54ca226469d5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:25.868959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:25.869601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:25.869926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:25.870209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] PATH_INFO: `/v3/projects/e17e5bb076834605afaadcfd58f7c633` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:25.871341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17e5bb076834605afaadcfd58f7c633)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:25.873259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:25.923337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17e5bb076834605afaadcfd58f7c633', 'request_id': 'req-11a5cfb3-c3db-457e-80f5-0d9027955cf5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.929688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17e5bb076834605afaadcfd58f7c633', 'request_id': 'req-11a5cfb3-c3db-457e-80f5-0d9027955cf5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.937760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17e5bb076834605afaadcfd58f7c633', 'request_id': 'req-11a5cfb3-c3db-457e-80f5-0d9027955cf5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.939207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:25.951794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] The token cache is being invalidate because project e17e5bb076834605afaadcfd58f7c633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:25.952304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.953321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-11a5cfb3-c3db-457e-80f5-0d9027955cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:25.954947 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4080/8191] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:25 2022] DELETE /identity/v3/projects/e17e5bb076834605afaadcfd58f7c633 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:25.959860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961393-8c55-4e5e-99ad-cb8a71fdc3b0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:25.960153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961393-8c55-4e5e-99ad-cb8a71fdc3b0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:25.960442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961393-8c55-4e5e-99ad-cb8a71fdc3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:25.982110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d1961393-8c55-4e5e-99ad-cb8a71fdc3b0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:25.983608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d1961393-8c55-4e5e-99ad-cb8a71fdc3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:26.050390 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4112/8192] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:26.055539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a18aa-9175-4b34-8354-5193f9b3a624 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.055818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a18aa-9175-4b34-8354-5193f9b3a624 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.056098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a18aa-9175-4b34-8354-5193f9b3a624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.078172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d79a18aa-9175-4b34-8354-5193f9b3a624 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:26.079405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d79a18aa-9175-4b34-8354-5193f9b3a624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:26.087611 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4081/8193] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:26.091818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961393-8c55-4e5e-99ad-cb8a71fdc3b0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.101913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961393-8c55-4e5e-99ad-cb8a71fdc3b0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.106798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.107411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.107695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.108233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.109440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:21:26.109750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.111381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.123500 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4113/8194] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:21:26.129108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a18aa-9175-4b34-8354-5193f9b3a624 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.138791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a18aa-9175-4b34-8354-5193f9b3a624 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.143644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.144251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.144533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.144810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.145502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.147988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.179566 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4082/8195] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:26 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:26.183583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.195220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a503b-b38f-43dc-b285-e344a962c095 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.258808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.259424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.259708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.259985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.260615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.262816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.286855 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4114/8196] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:26 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:21:26.290788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.303640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0cb7-fd2a-41f4-8b24-fe1e6c36da69 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.308732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.309519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.309849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.310225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.310935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.312731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.323868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4083/8197] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:26.327351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.338845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d9a5c-0f45-438f-9fc3-a57db1b7a41b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.344096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf842ad-6309-458f-a7f2-aa13184f7ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.344892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf842ad-6309-458f-a7f2-aa13184f7ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.345357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf842ad-6309-458f-a7f2-aa13184f7ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.345666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf842ad-6309-458f-a7f2-aa13184f7ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.346419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf842ad-6309-458f-a7f2-aa13184f7ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.348016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf842ad-6309-458f-a7f2-aa13184f7ea0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.366079 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4115/8198] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:26.366896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.386883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee338f2-28d6-49b9-8ef0-403a10a3bf4a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.391919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.392829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.393279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.393745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] PATH_INFO: `/v3/projects/039063a8349a4aa9ace85047b83deb39/users/9bd2040aa86b4625bee4100a1015b9bd/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.412485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039063a8349a4aa9ace85047b83deb39, user_id=9bd2040aa86b4625bee4100a1015b9bd, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.415342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.431570 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4084/8199] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:26 2022] PUT /identity/v3/projects/039063a8349a4aa9ace85047b83deb39/users/9bd2040aa86b4625bee4100a1015b9bd/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:21:26.443144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39e66-6e93-4cf0-b338-0e534b075415 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.443144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39e66-6e93-4cf0-b338-0e534b075415 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.443144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39e66-6e93-4cf0-b338-0e534b075415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.473254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e8f39e66-6e93-4cf0-b338-0e534b075415 None None] MFA Rules not processed for user `9bd2040aa86b4625bee4100a1015b9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:26.474536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e8f39e66-6e93-4cf0-b338-0e534b075415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:26.489084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:26.554023 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4116/8200] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:26.558525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.623535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056ea8-bf3e-4d46-9f74-f6dd47a45bf6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.630105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.630723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.631007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.631281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.631918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.633848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.668984 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4085/8201] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:26 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:26.670233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39e66-6e93-4cf0-b338-0e534b075415 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.682207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39e66-6e93-4cf0-b338-0e534b075415 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.747880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.748507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.748807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.749663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.749805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.752558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.780615 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4117/8202] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:26 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:26.783225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.795341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc70468-2ad0-4087-ba7e-d944f6e57748 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.802284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.803054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.803440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.803821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.804654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.806955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.820442 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4086/8203] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:26.824512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.836365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf7d2b-b159-4fe6-b3bc-f1a3d264f703 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:26.842281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:26.843125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:26.844144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:26.844413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] PATH_INFO: `/v3/projects/f3b493a34e8f476296d07ca374669df6/users/1bde4c3f08054c16b6d5c6a229fea4bd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:26.867959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b493a34e8f476296d07ca374669df6, user_id=1bde4c3f08054c16b6d5c6a229fea4bd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:26.870847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:26.891488 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4118/8204] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:26 2022] PUT /identity/v3/projects/f3b493a34e8f476296d07ca374669df6/users/1bde4c3f08054c16b6d5c6a229fea4bd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:26.895106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:26.913263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b15e52-e699-4a7b-a42d-46818fe409b4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:27.006907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ad3d8-36de-4de9-a636-b9f5355a8aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:27.007645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ad3d8-36de-4de9-a636-b9f5355a8aba admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.008032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ad3d8-36de-4de9-a636-b9f5355a8aba admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.008382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ad3d8-36de-4de9-a636-b9f5355a8aba admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.009111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ad3d8-36de-4de9-a636-b9f5355a8aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:27.011089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ad3d8-36de-4de9-a636-b9f5355a8aba admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:27.013189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:27.027489 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4087/8205] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:27.033904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:27.055867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106612-d3fd-49d0-a87b-993f86e877ef admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:27.061750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:27.062655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.062947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.063485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] PATH_INFO: `/v3/projects/f3b493a34e8f476296d07ca374669df6/users/1bde4c3f08054c16b6d5c6a229fea4bd/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.066366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b493a34e8f476296d07ca374669df6, user_id=1bde4c3f08054c16b6d5c6a229fea4bd, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:27.069709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:27.083190 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4119/8206] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:27 2022] PUT /identity/v3/projects/f3b493a34e8f476296d07ca374669df6/users/1bde4c3f08054c16b6d5c6a229fea4bd/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:27.095836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.096180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.096441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.135375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] MFA Rules not processed for user `1bde4c3f08054c16b6d5c6a229fea4bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:27.136697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:27.211243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:27.213058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:27.237468 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4088/8207] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:27.247800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:27.250936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:27.335089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78bfd9-cf8c-460e-a6a8-be5405541c68 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:27.347263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:27.347904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.348182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.348460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.355186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:27.365594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32517d5-1c8e-4862-8e8f-9b063ef6f1f9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:27.370130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:27.371293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.371783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.372230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.386121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:27.387596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:27.393460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:27.399154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:27.402459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:27.422086 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4089/8208] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:21:27.423633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:27.425190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:27.446662 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4120/8209] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:21:27.574804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:27.577394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab381460-d468-4d8b-8924-738fe48a87ac service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:27.582624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:27.583324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.583610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.583890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] PATH_INFO: `/v3/users/554f1c7ed01144808efe5657d94308fb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.585164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=554f1c7ed01144808efe5657d94308fb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:27.587540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:27.599637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9746ea-611d-4b43-a8ad-38ced5a8fdd5 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:27.608289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6441c4-ab9c-4e06-a02a-4771e17b0913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:27.609456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6441c4-ab9c-4e06-a02a-4771e17b0913 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.609910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6441c4-ab9c-4e06-a02a-4771e17b0913 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.610420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6441c4-ab9c-4e06-a02a-4771e17b0913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.623492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:27.659304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554f1c7ed01144808efe5657d94308fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:27.667894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554f1c7ed01144808efe5657d94308fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:27.668875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554f1c7ed01144808efe5657d94308fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:27.679976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554f1c7ed01144808efe5657d94308fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:27.680719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554f1c7ed01144808efe5657d94308fb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:27.685240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6441c4-ab9c-4e06-a02a-4771e17b0913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:27.686775 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6441c4-ab9c-4e06-a02a-4771e17b0913 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:27.698981 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4121/8210] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:27.703966 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4090/8211] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:27 2022] DELETE /identity/v3/users/554f1c7ed01144808efe5657d94308fb => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:27.708264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.708556 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.708920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.729914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:27.731151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:27.805848 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4122/8212] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:21:27.814346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:27.815848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:27.893098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b380d3-0b26-4586-8495-a9344e7ace4b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:27.896786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1424327-4488-4684-850a-d97250d311cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:27.897388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1424327-4488-4684-850a-d97250d311cf service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:27.897669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1424327-4488-4684-850a-d97250d311cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:27.897987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1424327-4488-4684-850a-d97250d311cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:27.907672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1424327-4488-4684-850a-d97250d311cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:27.909090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1424327-4488-4684-850a-d97250d311cf service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:27.918493 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4091/8213] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:28.095502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.096933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:28.112333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5339341-cac7-4562-8b40-8cc506e53d9f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.117135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.117770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.118096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.118370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] PATH_INFO: `/v3/projects/520c8d7744b44016bad9c8e81d778589` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.119472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=520c8d7744b44016bad9c8e81d778589)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.121470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.170329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520c8d7744b44016bad9c8e81d778589', 'request_id': 'req-e4a76258-927f-4b9a-8a7e-b11e63ddf336'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:28.176844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520c8d7744b44016bad9c8e81d778589', 'request_id': 'req-e4a76258-927f-4b9a-8a7e-b11e63ddf336'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:28.183089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520c8d7744b44016bad9c8e81d778589', 'request_id': 'req-e4a76258-927f-4b9a-8a7e-b11e63ddf336'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:28.195826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] The token cache is being invalidate because project 520c8d7744b44016bad9c8e81d778589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:28.196119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:28.196752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e4a76258-927f-4b9a-8a7e-b11e63ddf336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:28.198304 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4123/8214] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:28 2022] DELETE /identity/v3/projects/520c8d7744b44016bad9c8e81d778589 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:28.203462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad0d01-f1a7-4ad5-abcf-aec75877531c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.203746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad0d01-f1a7-4ad5-abcf-aec75877531c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.204023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad0d01-f1a7-4ad5-abcf-aec75877531c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.224286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-63ad0d01-f1a7-4ad5-abcf-aec75877531c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:28.225480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-63ad0d01-f1a7-4ad5-abcf-aec75877531c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:28.285388 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4092/8215] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:28.293755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf4d1-3d64-42c2-9297-cfae3c7579d5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.294302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf4d1-3d64-42c2-9297-cfae3c7579d5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.294413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf4d1-3d64-42c2-9297-cfae3c7579d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.313947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-759cf4d1-3d64-42c2-9297-cfae3c7579d5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:28.315160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-759cf4d1-3d64-42c2-9297-cfae3c7579d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:28.323039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4124/8216] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:28.327276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad0d01-f1a7-4ad5-abcf-aec75877531c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.338519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad0d01-f1a7-4ad5-abcf-aec75877531c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.343026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.344279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.344557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.344835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.345479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:21:28.345782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.347402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.359383 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4093/8217] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:21:28.363779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf4d1-3d64-42c2-9297-cfae3c7579d5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.373414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf4d1-3d64-42c2-9297-cfae3c7579d5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.378398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.378994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.379303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.379575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.380201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.382128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.392948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:28.412077 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4125/8218] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:28 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:28.416419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.426956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a27ee-0d93-4301-9b28-8dd5fc41234b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.459065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:28.489130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.489749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.490069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.490347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.490988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.492923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.519867 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4094/8219] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:28 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:28.521132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.531958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aac28e-712e-4a3c-b538-97e3fe549946 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.536713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.537250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.537534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.537809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.538506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.540121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.550883 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4126/8220] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:28.554983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.566242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1e945-7b01-48fd-8002-cc4b89b23385 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.571001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724289fe-17c1-4b36-bf77-570a5937538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.571612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724289fe-17c1-4b36-bf77-570a5937538b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.571907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724289fe-17c1-4b36-bf77-570a5937538b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.572214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724289fe-17c1-4b36-bf77-570a5937538b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.572855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724289fe-17c1-4b36-bf77-570a5937538b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.574580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724289fe-17c1-4b36-bf77-570a5937538b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.585146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4095/8221] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:21:28.589244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.600098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8e7e-0ec8-4882-bb60-da42546a67c2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.604923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.605585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.606002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.606223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] PATH_INFO: `/v3/projects/f85ee96781e645c893dc1d07e5bfb1ef/users/2a43ec49435e4aa890899e2c538bec7e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.622183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85ee96781e645c893dc1d07e5bfb1ef, user_id=2a43ec49435e4aa890899e2c538bec7e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.625532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.644369 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4127/8222] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:28 2022] PUT /identity/v3/projects/f85ee96781e645c893dc1d07e5bfb1ef/users/2a43ec49435e4aa890899e2c538bec7e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:28.654829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.655121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.655451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.685335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] MFA Rules not processed for user `2a43ec49435e4aa890899e2c538bec7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:28.686602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:28.779560 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4096/8223] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:28.788532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:28.791280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:28.948004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a0b6b-2250-4cba-bc9d-86bc2dddbcb2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:28.957630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d3ad9-fc53-4956-af08-070aa642e46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:28.958515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3ad9-fc53-4956-af08-070aa642e46e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:28.958954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3ad9-fc53-4956-af08-070aa642e46e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:28.959406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3ad9-fc53-4956-af08-070aa642e46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:28.973880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d3ad9-fc53-4956-af08-070aa642e46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:28.976067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d3ad9-fc53-4956-af08-070aa642e46e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:28.987899 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4128/8224] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:29.325014 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:29.327358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:29.400709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab98f5-215d-426e-9539-eccd06313589 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:29.406637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:29.407339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:29.407620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:29.407892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] PATH_INFO: `/v3/users/9bd2040aa86b4625bee4100a1015b9bd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:29.409026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd2040aa86b4625bee4100a1015b9bd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:29.411056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:29.467628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:29.470285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd2040aa86b4625bee4100a1015b9bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:29.483986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd2040aa86b4625bee4100a1015b9bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:29.486799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd2040aa86b4625bee4100a1015b9bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:29.501675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd2040aa86b4625bee4100a1015b9bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:29.503648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd2040aa86b4625bee4100a1015b9bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:29.534804 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4097/8225] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:29 2022] DELETE /identity/v3/users/9bd2040aa86b4625bee4100a1015b9bd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:29.539274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:29.539735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:29.539827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:29.560677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:29.562243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:29.627681 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4129/8226] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:21:29.637509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:29.638972 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:29.703774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f831-4170-4772-b9e6-eba088dc0dff admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:29.707610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:29.709369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:29.709686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:29.710032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:29.720214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:29.721715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:29.732282 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4098/8227] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:30.078649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:30.080320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:30.094635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb0043-6c46-42f9-85fe-a52df84fd13e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:30.132159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:30.133337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:30.133780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:30.134246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] PATH_INFO: `/v3/projects/039063a8349a4aa9ace85047b83deb39` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:30.136278 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=039063a8349a4aa9ace85047b83deb39)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:30.141464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:30.211330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039063a8349a4aa9ace85047b83deb39', 'request_id': 'req-4745bd07-be80-418e-89bc-f646698c1778'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.213170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:30.218642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039063a8349a4aa9ace85047b83deb39', 'request_id': 'req-4745bd07-be80-418e-89bc-f646698c1778'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.226526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039063a8349a4aa9ace85047b83deb39', 'request_id': 'req-4745bd07-be80-418e-89bc-f646698c1778'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.244997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] The token cache is being invalidate because project 039063a8349a4aa9ace85047b83deb39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:30.245436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.246200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.248061 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4130/8228] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:30 2022] DELETE /identity/v3/projects/039063a8349a4aa9ace85047b83deb39 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:30.252473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:30.263068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95ab2a-1ee9-4b88-8301-3dd1caae6037 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:30.324884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:30.325919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:30.326537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:30.326537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] PATH_INFO: `/v3/users/1bde4c3f08054c16b6d5c6a229fea4bd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:30.328965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bde4c3f08054c16b6d5c6a229fea4bd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:30.331164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:30.382261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bde4c3f08054c16b6d5c6a229fea4bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.390430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bde4c3f08054c16b6d5c6a229fea4bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.391295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bde4c3f08054c16b6d5c6a229fea4bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.400549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bde4c3f08054c16b6d5c6a229fea4bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.401312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-98bdfeb6-ec5a-42a4-b855-7664e798c3bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bde4c3f08054c16b6d5c6a229fea4bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:30.404426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:30.439229 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4099/8229] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:30 2022] DELETE /identity/v3/users/1bde4c3f08054c16b6d5c6a229fea4bd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:30.806648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:30.808320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:30.837649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bd07-be80-418e-89bc-f646698c1778 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:30.937923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:30.938561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:30.939066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:30.939345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] PATH_INFO: `/v3/projects/f3b493a34e8f476296d07ca374669df6` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:30.940491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3b493a34e8f476296d07ca374669df6)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:30.942445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.009640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3b493a34e8f476296d07ca374669df6', 'request_id': 'req-68233eeb-3a10-4a50-a187-8e51f06f38de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:31.016145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3b493a34e8f476296d07ca374669df6', 'request_id': 'req-68233eeb-3a10-4a50-a187-8e51f06f38de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:31.023482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3b493a34e8f476296d07ca374669df6', 'request_id': 'req-68233eeb-3a10-4a50-a187-8e51f06f38de'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:31.038650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] The token cache is being invalidate because project f3b493a34e8f476296d07ca374669df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:31.039295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:31.039718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-68233eeb-3a10-4a50-a187-8e51f06f38de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:31.050460 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4131/8230] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:30 2022] DELETE /identity/v3/projects/f3b493a34e8f476296d07ca374669df6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:31.054988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ea0e9-171d-4090-929f-5399e7aeb166 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.055283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ea0e9-171d-4090-929f-5399e7aeb166 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.055873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ea0e9-171d-4090-929f-5399e7aeb166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.079197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e18ea0e9-171d-4090-929f-5399e7aeb166 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:31.080430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e18ea0e9-171d-4090-929f-5399e7aeb166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:31.143825 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4100/8231] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:31.151623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc1d68-7896-43c9-88c7-edf01529a40b None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.151912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc1d68-7896-43c9-88c7-edf01529a40b None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.152191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc1d68-7896-43c9-88c7-edf01529a40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.173997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1cdc1d68-7896-43c9-88c7-edf01529a40b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:31.175215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1cdc1d68-7896-43c9-88c7-edf01529a40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:31.186446 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4132/8232] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:31.194658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ea0e9-171d-4090-929f-5399e7aeb166 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:31.205862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ea0e9-171d-4090-929f-5399e7aeb166 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:31.211761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:31.213006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.213493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.213657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.214753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:21:31.215703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:31.217902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.237752 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4101/8233] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:21:31.242969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc1d68-7896-43c9-88c7-edf01529a40b None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:31.255198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc1d68-7896-43c9-88c7-edf01529a40b None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:31.262941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:31.263575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.263920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.264199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.264840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:31.268561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.310756 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4133/8234] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:31 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:31.314829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:31.331910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f75be9-30c8-4e90-bfc8-08a17427ce0b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:31.409701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:31.410301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.410591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.410873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.411522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:31.413673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.450022 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4102/8235] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:31 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:21:31.452252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:31.466373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122546d5-5955-42c6-84b1-3eb80f8e3f8f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:31.473708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:31.474898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.475283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.475789 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.476540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:31.478539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.496076 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4134/8236] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:31.496563 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:31.527166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b199b5a-9281-4a11-a799-da553d521d27 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:31.533078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da8a7f-83a9-4004-a4e7-7db24a75617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:31.534150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da8a7f-83a9-4004-a4e7-7db24a75617f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.534150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da8a7f-83a9-4004-a4e7-7db24a75617f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.534532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da8a7f-83a9-4004-a4e7-7db24a75617f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.535313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da8a7f-83a9-4004-a4e7-7db24a75617f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:31.537077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da8a7f-83a9-4004-a4e7-7db24a75617f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.553777 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4103/8237] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:31.558041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:31.572397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0e98b-c94a-40ff-9700-f6f08a1d9643 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:31.584865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:31.585637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.586045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.586336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] PATH_INFO: `/v3/projects/b889222bf143423e9b0862ba22ee98ac/users/1131386be42643b199d977cc9863f1da/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.606412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b889222bf143423e9b0862ba22ee98ac, user_id=1131386be42643b199d977cc9863f1da, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:31.611632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.644039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4135/8238] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:31 2022] PUT /identity/v3/projects/b889222bf143423e9b0862ba22ee98ac/users/1131386be42643b199d977cc9863f1da/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:31.646357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b79b50-e5e4-4f35-8641-516ee265ceaf None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.647274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b79b50-e5e4-4f35-8641-516ee265ceaf None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.647274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b79b50-e5e4-4f35-8641-516ee265ceaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.695385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d4b79b50-e5e4-4f35-8641-516ee265ceaf None None] MFA Rules not processed for user `1131386be42643b199d977cc9863f1da`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:31.696858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d4b79b50-e5e4-4f35-8641-516ee265ceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:31.778184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:31.831508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4104/8239] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:31.836028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:31.847396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:31.929266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe91bc2-558c-46d0-8d97-54973c453b68 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:31.939749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:31.940822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:31.941263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:31.941643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:31.942658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:31.945206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:31.990316 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4136/8240] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:31 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:31.994035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b79b50-e5e4-4f35-8641-516ee265ceaf None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:32.012557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b79b50-e5e4-4f35-8641-516ee265ceaf None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:32.106190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:32.106953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:32.107297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:32.108253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:32.110406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:32.112436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:32.143906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4105/8241] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:31 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:32.147907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:32.165766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248b49-8f94-4bfd-9adc-da4225bc6bdb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:32.181271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:32.181942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:32.183051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:32.183313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:32.184039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:32.186311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:32.209912 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4137/8242] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:32.214765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:32.228392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bebc-c04b-4dcb-b1b4-ffa00ab10b11 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:32.241164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:32.242348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:32.243595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:32.244094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] PATH_INFO: `/v3/projects/938231ac78e5477f85f79a0a7d5f2031/users/4af669a5304d4a608fb804c21404b8cc/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:32.283482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938231ac78e5477f85f79a0a7d5f2031, user_id=4af669a5304d4a608fb804c21404b8cc, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:32.289053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:32.320481 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4106/8243] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:32 2022] PUT /identity/v3/projects/938231ac78e5477f85f79a0a7d5f2031/users/4af669a5304d4a608fb804c21404b8cc/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:32.322269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:32.340762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e6b29-16c5-418c-b0fe-3dd03cbf4455 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:32.436061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cb7c2-b3fb-45f1-9047-968027d45ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:32.437344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cb7c2-b3fb-45f1-9047-968027d45ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:32.438877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cb7c2-b3fb-45f1-9047-968027d45ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:32.439263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cb7c2-b3fb-45f1-9047-968027d45ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:32.440046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cb7c2-b3fb-45f1-9047-968027d45ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:32.442120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cb7c2-b3fb-45f1-9047-968027d45ce8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:32.463516 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4138/8244] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:21:32.467098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:32.502038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b45609-ead8-4dcf-b971-fbfd5c16b567 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:32.512342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:32.513353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:32.513766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:32.514142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] PATH_INFO: `/v3/projects/938231ac78e5477f85f79a0a7d5f2031/users/4af669a5304d4a608fb804c21404b8cc/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:32.520911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938231ac78e5477f85f79a0a7d5f2031, user_id=4af669a5304d4a608fb804c21404b8cc, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:32.551116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:32.614655 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4107/8245] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:32 2022] PUT /identity/v3/projects/938231ac78e5477f85f79a0a7d5f2031/users/4af669a5304d4a608fb804c21404b8cc/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:32.616884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb12ee-2b0e-4424-b309-5ad43e8e53fb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:32.617176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb12ee-2b0e-4424-b309-5ad43e8e53fb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:32.617459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb12ee-2b0e-4424-b309-5ad43e8e53fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:32.699245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-92fb12ee-2b0e-4424-b309-5ad43e8e53fb None None] MFA Rules not processed for user `4af669a5304d4a608fb804c21404b8cc`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:32.700869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-92fb12ee-2b0e-4424-b309-5ad43e8e53fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:32.835827 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4139/8246] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:32.849562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:32.851853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:32.997540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618a560-96ca-4d0d-9eeb-92c0f763de93 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:33.003704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:33.004456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:33.004811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:33.005169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:33.027804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:33.029847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:33.051469 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4108/8247] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:33.156765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb12ee-2b0e-4424-b309-5ad43e8e53fb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:33.163276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:33.177639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb12ee-2b0e-4424-b309-5ad43e8e53fb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:33.187204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b277c-b624-4980-a085-80a91b5f8b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:33.192721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b277c-b624-4980-a085-80a91b5f8b1e service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:33.193308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b277c-b624-4980-a085-80a91b5f8b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:33.194240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b277c-b624-4980-a085-80a91b5f8b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:33.301900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b277c-b624-4980-a085-80a91b5f8b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:33.303752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b277c-b624-4980-a085-80a91b5f8b1e service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:33.315861 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4140/8248] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:35.185390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:35.188373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:35.226346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:35.298830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca162f-ddba-4b6a-aedc-278f78cba066 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:35.304433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:35.305078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:35.305365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:35.305646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] PATH_INFO: `/v3/users/2a43ec49435e4aa890899e2c538bec7e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:35.307032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a43ec49435e4aa890899e2c538bec7e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:35.308976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:35.372783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a43ec49435e4aa890899e2c538bec7e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:35.381597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a43ec49435e4aa890899e2c538bec7e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:35.383376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a43ec49435e4aa890899e2c538bec7e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:35.402601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a43ec49435e4aa890899e2c538bec7e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:35.410977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a43ec49435e4aa890899e2c538bec7e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:35.442567 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4109/8249] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:35 2022] DELETE /identity/v3/users/2a43ec49435e4aa890899e2c538bec7e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:35.446869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:35.447399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:35.447399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:35.468780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:35.470365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:35.556478 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4141/8250] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:35.567518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:35.569761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:35.642781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70217ec-320b-45b8-a404-5b4c8e8dec59 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:35.646604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0744476-c101-42b3-bcf2-af3daa202cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:35.647459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0744476-c101-42b3-bcf2-af3daa202cbc service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:35.647774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0744476-c101-42b3-bcf2-af3daa202cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:35.648680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0744476-c101-42b3-bcf2-af3daa202cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:35.658624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0744476-c101-42b3-bcf2-af3daa202cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:35.660490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0744476-c101-42b3-bcf2-af3daa202cbc service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:35.672021 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4110/8251] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:35.878897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:35.881067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:35.894933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955aedc7-23bb-4021-b605-a07b2814e26d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:35.904585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:35.905504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:35.905995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:35.906275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] PATH_INFO: `/v3/projects/f85ee96781e645c893dc1d07e5bfb1ef` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:35.907460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85ee96781e645c893dc1d07e5bfb1ef)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:35.909401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:35.978979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85ee96781e645c893dc1d07e5bfb1ef', 'request_id': 'req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:35.987223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85ee96781e645c893dc1d07e5bfb1ef', 'request_id': 'req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:35.994263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85ee96781e645c893dc1d07e5bfb1ef', 'request_id': 'req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.009115 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] The token cache is being invalidate because project f85ee96781e645c893dc1d07e5bfb1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:36.009558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.010426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f490fe5a-9bc1-49b3-8e3c-001a56f7fb09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.011988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4142/8252] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:35 2022] DELETE /identity/v3/projects/f85ee96781e645c893dc1d07e5bfb1ef => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:36.017476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d58591-4b39-4e8d-97bd-e4fb6bfbf90d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.017767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d58591-4b39-4e8d-97bd-e4fb6bfbf90d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.018213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d58591-4b39-4e8d-97bd-e4fb6bfbf90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.042752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d5d58591-4b39-4e8d-97bd-e4fb6bfbf90d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:36.044097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d5d58591-4b39-4e8d-97bd-e4fb6bfbf90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:36.104929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:36.134674 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4111/8253] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:36.143262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1c3b8-9b7e-4bf8-a76e-958f75795067 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.143555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1c3b8-9b7e-4bf8-a76e-958f75795067 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.143833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1c3b8-9b7e-4bf8-a76e-958f75795067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.168496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ccd1c3b8-9b7e-4bf8-a76e-958f75795067 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:36.169893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ccd1c3b8-9b7e-4bf8-a76e-958f75795067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:36.180318 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4143/8254] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:36.186780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d58591-4b39-4e8d-97bd-e4fb6bfbf90d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.202760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d58591-4b39-4e8d-97bd-e4fb6bfbf90d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.213660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.214473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.214990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.215308 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.216083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:21:36.216452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.218821 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.234285 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4112/8255] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:21:36.240297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1c3b8-9b7e-4bf8-a76e-958f75795067 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.255418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1c3b8-9b7e-4bf8-a76e-958f75795067 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.261422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.262065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.262347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.262616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.263249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.265790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.288056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:36.304699 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4144/8256] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:36 2022] POST /identity/v3/projects => generated 359 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:36.309860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.328906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7938b-303f-41dd-886f-31e5f4bd2ee4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.349631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.352380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:36.400805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.401428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.401703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.402060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.402807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.404669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.431500 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4113/8257] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:36 2022] POST /identity/v3/users => generated 280 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:36.433140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.435111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7bae2-7b89-4c31-833c-966067f37781 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.440247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.442858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01374ed-4320-48e3-acbd-f08288484b56 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.445049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.445296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.445567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] PATH_INFO: `/v3/users/1131386be42643b199d977cc9863f1da` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.446751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1131386be42643b199d977cc9863f1da)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.447505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.448094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.448378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.448664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.448783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.449278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.450886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.466170 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4114/8258] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:36.467212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.478744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964b7f7-e21f-4e36-858f-5dd5a0da8066 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.485029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.485599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.485923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.486397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.487112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.488774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.500146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4115/8259] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:36.503918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.511586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1131386be42643b199d977cc9863f1da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.514194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b796011-13a7-4998-a10d-c487d073ce2b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.520032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a2d46-ab83-4e5e-add6-21e7fe44edc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.520671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a2d46-ab83-4e5e-add6-21e7fe44edc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.520956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a2d46-ab83-4e5e-add6-21e7fe44edc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.521233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a2d46-ab83-4e5e-add6-21e7fe44edc8 admin admin] PATH_INFO: `/v3/projects/0f99852f30db42d69b79be295dc8e5f7/users/a328d7cf3df0456d89d36e28b5a3c42a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.526063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1131386be42643b199d977cc9863f1da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.534589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1131386be42643b199d977cc9863f1da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.536686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a2d46-ab83-4e5e-add6-21e7fe44edc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f99852f30db42d69b79be295dc8e5f7, user_id=a328d7cf3df0456d89d36e28b5a3c42a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.539461 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a2d46-ab83-4e5e-add6-21e7fe44edc8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.541999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1131386be42643b199d977cc9863f1da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.542734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-17f34b31-68dd-495f-a6df-14acf6336c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1131386be42643b199d977cc9863f1da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:36.548853 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4145/8260] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:36 2022] DELETE /identity/v3/users/1131386be42643b199d977cc9863f1da => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:21:36.552471 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4116/8261] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:36 2022] PUT /identity/v3/projects/0f99852f30db42d69b79be295dc8e5f7/users/a328d7cf3df0456d89d36e28b5a3c42a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:21:36.553100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099c120-57df-418b-bf69-58917528ce9a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.553396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099c120-57df-418b-bf69-58917528ce9a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.553678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099c120-57df-418b-bf69-58917528ce9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.564212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.564497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.564770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.573447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1099c120-57df-418b-bf69-58917528ce9a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:36.574695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1099c120-57df-418b-bf69-58917528ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:36.594965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] MFA Rules not processed for user `a328d7cf3df0456d89d36e28b5a3c42a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:36.596180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:36.637477 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4146/8262] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:21:36.648229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099c120-57df-418b-bf69-58917528ce9a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.649700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1099c120-57df-418b-bf69-58917528ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:36.663341 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4117/8263] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:21:36.667812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:36.673925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:36.675302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:36.681856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:36.713768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099c120-57df-418b-bf69-58917528ce9a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.717461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.718917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.719288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.719569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.729148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.730671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.740624 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4147/8264] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:21:36.740904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175acd65-2874-483e-ab7f-f35d2537ed57 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:36.744618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:36.745231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:36.745545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:36.745819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:36.754837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:36.756284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:36.765941 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4118/8265] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:21:37.292413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:37.294250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:37.310448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f70940-e259-4592-a10f-48d113724c6d service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:37.315958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:37.316678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:37.316874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:37.317135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] PATH_INFO: `/v3/projects/b889222bf143423e9b0862ba22ee98ac` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:37.318317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b889222bf143423e9b0862ba22ee98ac)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:37.320342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:37.376733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b889222bf143423e9b0862ba22ee98ac', 'request_id': 'req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.388071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b889222bf143423e9b0862ba22ee98ac', 'request_id': 'req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.396543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b889222bf143423e9b0862ba22ee98ac', 'request_id': 'req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.409880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] The token cache is being invalidate because project b889222bf143423e9b0862ba22ee98ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:37.410178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.410808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.412532 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4148/8266] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:37 2022] DELETE /identity/v3/projects/b889222bf143423e9b0862ba22ee98ac => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:37.416389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:37.428852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5933fa-c07f-4993-9434-2b7ddd996569 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:37.490138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:37.490697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:37.490987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:37.491264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] PATH_INFO: `/v3/users/4af669a5304d4a608fb804c21404b8cc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:37.492395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af669a5304d4a608fb804c21404b8cc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:37.494425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:37.543902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af669a5304d4a608fb804c21404b8cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.551197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af669a5304d4a608fb804c21404b8cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.551950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af669a5304d4a608fb804c21404b8cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.559505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af669a5304d4a608fb804c21404b8cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.560269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5f03e7c6-d0bf-4544-a619-a468addecb50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af669a5304d4a608fb804c21404b8cc'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:37.580546 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4119/8267] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:37 2022] DELETE /identity/v3/users/4af669a5304d4a608fb804c21404b8cc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:37.901099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:37.902696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:37.917209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8fa8-5e7e-4a8a-ba1a-15d66cd61c92 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.009072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.009810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.010470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.010785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] PATH_INFO: `/v3/projects/938231ac78e5477f85f79a0a7d5f2031` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.012097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=938231ac78e5477f85f79a0a7d5f2031)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.015070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:38.076915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:38.083767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938231ac78e5477f85f79a0a7d5f2031', 'request_id': 'req-f53dda8f-d391-408a-8f5e-404ce5b45fe9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:38.095380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938231ac78e5477f85f79a0a7d5f2031', 'request_id': 'req-f53dda8f-d391-408a-8f5e-404ce5b45fe9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:38.110485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938231ac78e5477f85f79a0a7d5f2031', 'request_id': 'req-f53dda8f-d391-408a-8f5e-404ce5b45fe9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:38.133252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] The token cache is being invalidate because project 938231ac78e5477f85f79a0a7d5f2031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:38.133618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:38.137167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f53dda8f-d391-408a-8f5e-404ce5b45fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:38.144900 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4149/8268] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:37 2022] DELETE /identity/v3/projects/938231ac78e5477f85f79a0a7d5f2031 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:38.146735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055eb2-dc65-44d1-9e00-940d495fb64f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.147034 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055eb2-dc65-44d1-9e00-940d495fb64f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.147317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055eb2-dc65-44d1-9e00-940d495fb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.173213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8c055eb2-dc65-44d1-9e00-940d495fb64f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:38.174633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8c055eb2-dc65-44d1-9e00-940d495fb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:38.200091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:38.257366 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4120/8269] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:38.267228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245ec0-471a-4478-b3b2-d415218e0902 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.267761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245ec0-471a-4478-b3b2-d415218e0902 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.268286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245ec0-471a-4478-b3b2-d415218e0902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.300724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-e5245ec0-471a-4478-b3b2-d415218e0902 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:38.302321 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5245ec0-471a-4478-b3b2-d415218e0902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:38.313089 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4150/8270] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:38.317437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055eb2-dc65-44d1-9e00-940d495fb64f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:38.328870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055eb2-dc65-44d1-9e00-940d495fb64f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.334462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.335418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.335730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.336017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.336736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:21:38.337363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.339001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:38.354886 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4121/8271] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:21:38.360319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245ec0-471a-4478-b3b2-d415218e0902 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:38.374145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245ec0-471a-4478-b3b2-d415218e0902 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.383530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.384866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.385151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.385425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.386219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.388673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:38.427223 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4151/8272] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:21:38 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:38.428948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:38.440540 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6039d-fe51-4a53-aa12-37a6a6c21f36 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.522582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.523508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.523940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.524370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.525279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.527826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:38.556093 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4122/8273] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:21:38 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:21:38.559927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:38.575618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651121af-0f26-4e50-a137-e7c3e16c0868 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.581362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.582217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.582507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.582831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.583878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.586340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:38.599989 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4152/8274] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:21:38.603153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:38.618990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f40416-a851-432b-ab48-956d266532a0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.626600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf088e7-581e-48e1-8351-ddb9647cb6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.627315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf088e7-581e-48e1-8351-ddb9647cb6ea admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.627671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf088e7-581e-48e1-8351-ddb9647cb6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.628520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf088e7-581e-48e1-8351-ddb9647cb6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.628883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf088e7-581e-48e1-8351-ddb9647cb6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.630608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf088e7-581e-48e1-8351-ddb9647cb6ea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:38.648667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4123/8275] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:21:38.652648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:38.663254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb98d61-ddf3-49de-87e9-2bf86f51f053 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.669196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.670021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.670307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.670584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] PATH_INFO: `/v3/projects/4ae1d7150fa743188cc864665d705e7d/users/08f7d28b893d4ba093e78bc2231b4f79/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.687670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae1d7150fa743188cc864665d705e7d, user_id=08f7d28b893d4ba093e78bc2231b4f79, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.690557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:38.706008 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4153/8276] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:21:38 2022] PUT /identity/v3/projects/4ae1d7150fa743188cc864665d705e7d/users/08f7d28b893d4ba093e78bc2231b4f79/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:38.718443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfc9f0-1188-4909-9116-f4689c3b8b33 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.718768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfc9f0-1188-4909-9116-f4689c3b8b33 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.719607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfc9f0-1188-4909-9116-f4689c3b8b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.751324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cabfc9f0-1188-4909-9116-f4689c3b8b33 None None] MFA Rules not processed for user `08f7d28b893d4ba093e78bc2231b4f79`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:38.752593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cabfc9f0-1188-4909-9116-f4689c3b8b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:38.847011 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4124/8277] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:38.855581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:38.857642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:38.969748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c49c1d-747a-4555-bc26-00a5c6999c2e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:38.973727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633737d-f1da-4d61-bd94-e7f61da63011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:38.974383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633737d-f1da-4d61-bd94-e7f61da63011 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:38.974678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633737d-f1da-4d61-bd94-e7f61da63011 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:38.975202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633737d-f1da-4d61-bd94-e7f61da63011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:38.985652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633737d-f1da-4d61-bd94-e7f61da63011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:38.987954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633737d-f1da-4d61-bd94-e7f61da63011 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:39.001030 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4154/8278] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:39.028108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfc9f0-1188-4909-9116-f4689c3b8b33 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:39.065569 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:39.126656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfc9f0-1188-4909-9116-f4689c3b8b33 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:39.136032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:39.136697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:39.136977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:39.137251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] PATH_INFO: `/v3/users/08f7d28b893d4ba093e78bc2231b4f79` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:39.140919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08f7d28b893d4ba093e78bc2231b4f79)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:39.142981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:39.219321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08f7d28b893d4ba093e78bc2231b4f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.228099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08f7d28b893d4ba093e78bc2231b4f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.228954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08f7d28b893d4ba093e78bc2231b4f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.239310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08f7d28b893d4ba093e78bc2231b4f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.240875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08f7d28b893d4ba093e78bc2231b4f79'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.275170 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4125/8279] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:21:39 2022] DELETE /identity/v3/users/08f7d28b893d4ba093e78bc2231b4f79 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:39.281099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:39.282172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:39.282715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:39.310439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:21:39.311754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:39.376324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4155/8280] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:21:39.386186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:39.387925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:39.455419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebef0c-b7e1-419b-af1c-ceefbd105da2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:39.459901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:39.460646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:39.460935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:39.461213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:39.471968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:39.473340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:39.484413 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4126/8281] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:39.819715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:39.821484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:39.852565 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e15ca-200e-4af8-ac24-8572645ba4d1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:39.860712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:39.861510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:39.861999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:39.862353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] PATH_INFO: `/v3/projects/4ae1d7150fa743188cc864665d705e7d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:39.864360 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae1d7150fa743188cc864665d705e7d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:39.866842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:39.876449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:39.931550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae1d7150fa743188cc864665d705e7d', 'request_id': 'req-6b2828c5-53a1-4a22-a1ac-1acf25456696'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.938490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae1d7150fa743188cc864665d705e7d', 'request_id': 'req-6b2828c5-53a1-4a22-a1ac-1acf25456696'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.947895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae1d7150fa743188cc864665d705e7d', 'request_id': 'req-6b2828c5-53a1-4a22-a1ac-1acf25456696'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.971579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] The token cache is being invalidate because project 4ae1d7150fa743188cc864665d705e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:21:39.972563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.974226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:21:39.976090 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4156/8282] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:21:39 2022] DELETE /identity/v3/projects/4ae1d7150fa743188cc864665d705e7d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:21:40.834888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:21:40.836504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:40.886855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:40.916640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe09b8-0822-4992-bb84-eeccca51fffa service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:21:40.988888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bd420-41e3-4b82-9e27-2e3a8742394f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:21:40.989600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bd420-41e3-4b82-9e27-2e3a8742394f service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:21:40.990581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bd420-41e3-4b82-9e27-2e3a8742394f service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:21:40.990939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bd420-41e3-4b82-9e27-2e3a8742394f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:21:40.993767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-908bd420-41e3-4b82-9e27-2e3a8742394f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:21:41.009816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bd420-41e3-4b82-9e27-2e3a8742394f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:21:41.011545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bd420-41e3-4b82-9e27-2e3a8742394f service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:21:41.027430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4127/8283] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:21:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:21:49.964776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:21:51.019624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=38 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:02.751081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:02.752628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:02.822211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2828c5-53a1-4a22-a1ac-1acf25456696 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:02.827451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:02.828161 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:02.828445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:02.828720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] PATH_INFO: `/v3/users/a328d7cf3df0456d89d36e28b5a3c42a` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:02.829898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a328d7cf3df0456d89d36e28b5a3c42a)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:02.831815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:02.880693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a328d7cf3df0456d89d36e28b5a3c42a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:02.892647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a328d7cf3df0456d89d36e28b5a3c42a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:02.900482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a328d7cf3df0456d89d36e28b5a3c42a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:02.908792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a328d7cf3df0456d89d36e28b5a3c42a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:02.909501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a328d7cf3df0456d89d36e28b5a3c42a'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:02.915783 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4157/8284] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:22:02 2022] DELETE /identity/v3/users/a328d7cf3df0456d89d36e28b5a3c42a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:22:02.920219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:02.921263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:02.921263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:02.940869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:02.942097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:03.003108 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4128/8285] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:22:03.012755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.014192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:03.075678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8943021-272a-445e-8a8f-f7c91208c4c0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.079577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc38c9-ec14-40b1-954d-a5abc8cf2e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.080177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc38c9-ec14-40b1-954d-a5abc8cf2e42 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.080459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc38c9-ec14-40b1-954d-a5abc8cf2e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.080733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc38c9-ec14-40b1-954d-a5abc8cf2e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.089951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc38c9-ec14-40b1-954d-a5abc8cf2e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.091383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc38c9-ec14-40b1-954d-a5abc8cf2e42 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.101299 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4158/8286] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:22:03.272872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.274432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:03.290661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a64bf-d69e-4284-848a-0b5d3a0ac519 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.297104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.297744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.298119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.298324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] PATH_INFO: `/v3/projects/0f99852f30db42d69b79be295dc8e5f7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.299445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f99852f30db42d69b79be295dc8e5f7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.301892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.351720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f99852f30db42d69b79be295dc8e5f7', 'request_id': 'req-5a921a47-cad3-4634-be59-a7b0287b5a96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:03.358458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f99852f30db42d69b79be295dc8e5f7', 'request_id': 'req-5a921a47-cad3-4634-be59-a7b0287b5a96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:03.365845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f99852f30db42d69b79be295dc8e5f7', 'request_id': 'req-5a921a47-cad3-4634-be59-a7b0287b5a96'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:03.380027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] The token cache is being invalidate because project 0f99852f30db42d69b79be295dc8e5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:22:03.380315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:03.380950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-5a921a47-cad3-4634-be59-a7b0287b5a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:03.382459 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4129/8287] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:22:03 2022] DELETE /identity/v3/projects/0f99852f30db42d69b79be295dc8e5f7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:22:03.387306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d7808-c0c6-4865-b5d0-6fdf43c4e5cb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.387669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d7808-c0c6-4865-b5d0-6fdf43c4e5cb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.387946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d7808-c0c6-4865-b5d0-6fdf43c4e5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.408712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-268d7808-c0c6-4865-b5d0-6fdf43c4e5cb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:03.410155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-268d7808-c0c6-4865-b5d0-6fdf43c4e5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:03.426927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:03.479233 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4159/8288] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:22:03.580323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b5241-e632-4c6b-ab91-bfaaa57558d5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.580627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b5241-e632-4c6b-ab91-bfaaa57558d5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.580901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b5241-e632-4c6b-ab91-bfaaa57558d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.600699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-975b5241-e632-4c6b-ab91-bfaaa57558d5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:03.602002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-975b5241-e632-4c6b-ab91-bfaaa57558d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:03.610584 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4130/8289] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:22:03.615006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d7808-c0c6-4865-b5d0-6fdf43c4e5cb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.625569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d7808-c0c6-4865-b5d0-6fdf43c4e5cb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.630272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.630895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.631181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.631486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.632127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:22:03.632432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.634051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.649859 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4160/8290] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:22:03.651153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b5241-e632-4c6b-ab91-bfaaa57558d5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.661207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b5241-e632-4c6b-ab91-bfaaa57558d5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.665984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.666587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.666869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.667141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.667767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.669663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.705555 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4131/8291] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:22:03 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:22:03.706734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.716704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c75e-871d-46dc-937c-a75da2c22749 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.774063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.774755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.774934 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.775203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.775818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.777712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.799814 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4161/8292] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:22:03 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:22:03.803963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.815379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5a6df-4731-4fcc-8917-9aff70c29b03 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.821071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.821669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.822007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.822285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.822927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.824561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.834906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4132/8293] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:22:03.838832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.849208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8953d-2496-4818-9785-4531bc136202 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.853994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7253d43-56f1-4d9d-9bb4-7ea3ecd4dad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.854590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7253d43-56f1-4d9d-9bb4-7ea3ecd4dad2 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.854875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7253d43-56f1-4d9d-9bb4-7ea3ecd4dad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.855152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7253d43-56f1-4d9d-9bb4-7ea3ecd4dad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.855787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7253d43-56f1-4d9d-9bb4-7ea3ecd4dad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.857377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7253d43-56f1-4d9d-9bb4-7ea3ecd4dad2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.868841 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4162/8294] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:22:03.872989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:03.883883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f282-a5bf-4e87-8498-4d621a5ebec6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:03.888566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:03.889238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.889538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.889850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] PATH_INFO: `/v3/projects/dc118ce311a943d084a0761eb3583d10/users/c08a4bf7c3184c55b29d90f370476fcd/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.904092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc118ce311a943d084a0761eb3583d10, user_id=c08a4bf7c3184c55b29d90f370476fcd, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:03.907588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:03.921635 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4133/8295] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:22:03 2022] PUT /identity/v3/projects/dc118ce311a943d084a0761eb3583d10/users/c08a4bf7c3184c55b29d90f370476fcd/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:22:03.931486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:03.931778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:03.932058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:03.959750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] MFA Rules not processed for user `c08a4bf7c3184c55b29d90f370476fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:03.961006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:04.026903 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4163/8296] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:22:04.037966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:04.039705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:04.062682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:04.102704 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff6c5f-7c0b-4a04-80e1-bf1d4931659f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:04.106371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:04.107041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:04.107320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:04.107587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:04.117150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:04.118595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:04.127423 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4134/8297] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:22:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:22:09.527111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:09.528745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:09.547184 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24f7b3-0b88-485a-a3e0-fe6c5b9574c9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:09.551149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6359c9-9155-4772-b3f6-e823e0508e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:09.551858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6359c9-9155-4772-b3f6-e823e0508e36 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:09.552213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6359c9-9155-4772-b3f6-e823e0508e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:09.552558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6359c9-9155-4772-b3f6-e823e0508e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:09.554514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fa6359c9-9155-4772-b3f6-e823e0508e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:09.567728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6359c9-9155-4772-b3f6-e823e0508e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:09.569301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6359c9-9155-4772-b3f6-e823e0508e36 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:09.582501 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4164/8298] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:22:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:22:09.967146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:09.984817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea8df-8a01-41e9-acfd-391bb5621ba2 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:09.988722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:09.989642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:09.989968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:09.990409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:09.992326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:10.081444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:10.083327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:10.101328 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4135/8299] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:22:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:22:19.576763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:20.092589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=51 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:23.516172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:23.516752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:23.516752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:23.552314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] MFA Rules not processed for user `f5afa360fc4c40288b65c437519c2a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:23.553588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:23.670490 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4165/8300] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:22:24.388561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:24.413366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e922492-e9d3-464f-9905-24d6162f02b0 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:24.421791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:24.422894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:24.423481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:24.423968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:24.428532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:24.453774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:24.456966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:24.472846 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4136/8301] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:22:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:22:33.661228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=25 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:34.465384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:50.150709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:50.153071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:50.386337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b6211-8b2e-4c94-815c-d589f47a3105 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:50.393673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c44f5-1a2a-4931-8365-1af0595976f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:50.394447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c44f5-1a2a-4931-8365-1af0595976f3 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:50.394790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c44f5-1a2a-4931-8365-1af0595976f3 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:50.395125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c44f5-1a2a-4931-8365-1af0595976f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:50.397733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3d3c44f5-1a2a-4931-8365-1af0595976f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:50.416400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c44f5-1a2a-4931-8365-1af0595976f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:50.418605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c44f5-1a2a-4931-8365-1af0595976f3 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:50.446355 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4166/8302] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:22:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:22:58.107909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:58.192356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74f160-b1b4-498d-a489-82a123bcd7c0 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:58.197761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:58.198556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:58.198702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:58.198983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] PATH_INFO: `/v3/users/c08a4bf7c3184c55b29d90f370476fcd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:58.200427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08a4bf7c3184c55b29d90f370476fcd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:58.202541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:58.262571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08a4bf7c3184c55b29d90f370476fcd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:58.271889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08a4bf7c3184c55b29d90f370476fcd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:58.272611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08a4bf7c3184c55b29d90f370476fcd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:58.283226 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08a4bf7c3184c55b29d90f370476fcd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:58.284067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08a4bf7c3184c55b29d90f370476fcd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:58.320665 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4137/8303] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:22:58 2022] DELETE /identity/v3/users/c08a4bf7c3184c55b29d90f370476fcd => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:22:58.326276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:58.326768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:58.326877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:58.379561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:58.381174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:58.461449 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4167/8304] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:22:58.471199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:58.473330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:58.557774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c32d17-f353-48ab-aa20-2bfdb529090d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:58.562051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47e3f4-ae02-43da-8524-2cd7e5879865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:58.562775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47e3f4-ae02-43da-8524-2cd7e5879865 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:58.563125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47e3f4-ae02-43da-8524-2cd7e5879865 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:58.563470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47e3f4-ae02-43da-8524-2cd7e5879865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:58.573524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47e3f4-ae02-43da-8524-2cd7e5879865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:58.575723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47e3f4-ae02-43da-8524-2cd7e5879865 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:58.580202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:58.588201 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4138/8305] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:22:59.227021 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:59.228840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:59.244893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f13a7b-2a8f-4025-aa6c-35dffa56df5f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:59.251437 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:59.252282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.252624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.252908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] PATH_INFO: `/v3/projects/dc118ce311a943d084a0761eb3583d10` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.254169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc118ce311a943d084a0761eb3583d10)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:59.256250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:59.280334 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:22:59.315314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc118ce311a943d084a0761eb3583d10', 'request_id': 'req-0f9b0aaa-489f-469e-816c-8eb1115d6afc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:59.320994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc118ce311a943d084a0761eb3583d10', 'request_id': 'req-0f9b0aaa-489f-469e-816c-8eb1115d6afc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:59.327590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc118ce311a943d084a0761eb3583d10', 'request_id': 'req-0f9b0aaa-489f-469e-816c-8eb1115d6afc'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:59.340678 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] The token cache is being invalidate because project dc118ce311a943d084a0761eb3583d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:22:59.341170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:59.342011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f9b0aaa-489f-469e-816c-8eb1115d6afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:22:59.343681 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4168/8306] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:22:59 2022] DELETE /identity/v3/projects/dc118ce311a943d084a0761eb3583d10 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:22:59.348578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8dfe-187a-4f02-be7a-2164500da446 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.349060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8dfe-187a-4f02-be7a-2164500da446 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.349159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8dfe-187a-4f02-be7a-2164500da446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.381151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c22f8dfe-187a-4f02-be7a-2164500da446 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:59.382400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c22f8dfe-187a-4f02-be7a-2164500da446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:59.445198 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4139/8307] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:22:59.454826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98c50-3cdb-4b0c-9a06-e4b9ba6e7dec None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.455208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98c50-3cdb-4b0c-9a06-e4b9ba6e7dec None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.455557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98c50-3cdb-4b0c-9a06-e4b9ba6e7dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.476632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fce98c50-3cdb-4b0c-9a06-e4b9ba6e7dec None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:59.477914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fce98c50-3cdb-4b0c-9a06-e4b9ba6e7dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:59.487287 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4169/8308] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:22:59.491593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8dfe-187a-4f02-be7a-2164500da446 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:59.501979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8dfe-187a-4f02-be7a-2164500da446 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:59.507385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:59.508066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.508435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.508804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.509525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:22:59.509934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:59.511628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:59.527277 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4140/8309] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:22:59.529007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce98c50-3cdb-4b0c-9a06-e4b9ba6e7dec None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:59.541688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce98c50-3cdb-4b0c-9a06-e4b9ba6e7dec None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:59.547663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:59.548295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.548584 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.548864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.549508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:59.551531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:59.589654 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4170/8310] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:22:59 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:22:59.594671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:59.607334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268cb9c-c86e-4ede-8a7e-d563aff8babd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:59.695317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:59.696106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.696440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.696779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.697487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:59.699534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:59.725514 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4141/8311] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:22:59 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:22:59.730075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:59.743918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993632fc-f1de-4eeb-91aa-cb42e1531c17 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:59.760092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:59.760922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.761210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.761507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.762993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:59.765040 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:59.776182 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4171/8312] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:22:59.781315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:59.791820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578d0bc-50a9-4323-827f-81fbc8cb7584 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:59.796867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33c2d-6ceb-48d1-aef4-90906b6b6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:59.797474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33c2d-6ceb-48d1-aef4-90906b6b6096 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.797759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33c2d-6ceb-48d1-aef4-90906b6b6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.798118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33c2d-6ceb-48d1-aef4-90906b6b6096 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.798761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33c2d-6ceb-48d1-aef4-90906b6b6096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:59.800455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33c2d-6ceb-48d1-aef4-90906b6b6096 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:59.812589 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4142/8313] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:22:59.817066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:22:59.830848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25567cf-a57f-403b-90df-e14281e6a15a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:22:59.837324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:22:59.838192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.838563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.838910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] PATH_INFO: `/v3/projects/675281d0efa249ad9c4ccf87e67b8e2d/users/eb238d6a62be4772ae6535fce36f9239/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.855984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=675281d0efa249ad9c4ccf87e67b8e2d, user_id=eb238d6a62be4772ae6535fce36f9239, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:22:59.859717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:22:59.884754 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4172/8314] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:22:59 2022] PUT /identity/v3/projects/675281d0efa249ad9c4ccf87e67b8e2d/users/eb238d6a62be4772ae6535fce36f9239/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:22:59.886543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:22:59.886831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:22:59.887106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:22:59.920495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] MFA Rules not processed for user `eb238d6a62be4772ae6535fce36f9239`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:22:59.921991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:22:59.982774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:00.017183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:00.020421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:00.021881 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4143/8315] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:22:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:00.033927 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:00.035482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:00.102302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc08f2-639e-43ad-84e7-37969d3f91ab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:00.105979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df0d74-3fab-428c-b5ae-c1c546363d83 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:00.107629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:00.108364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:00.109031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:00.109438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] PATH_INFO: `/v3/users/f5afa360fc4c40288b65c437519c2a7f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:00.110774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5afa360fc4c40288b65c437519c2a7f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:00.111450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ecf29-634f-4357-87e8-bb1728219804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:00.112069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecf29-634f-4357-87e8-bb1728219804 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:00.112564 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecf29-634f-4357-87e8-bb1728219804 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:00.112850 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecf29-634f-4357-87e8-bb1728219804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:00.113439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:00.122985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ecf29-634f-4357-87e8-bb1728219804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:00.124466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ecf29-634f-4357-87e8-bb1728219804 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:00.133934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4144/8316] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:00.165721 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:00.167498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5afa360fc4c40288b65c437519c2a7f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.181420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5afa360fc4c40288b65c437519c2a7f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.189369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5afa360fc4c40288b65c437519c2a7f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.198146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5afa360fc4c40288b65c437519c2a7f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.198827 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5afa360fc4c40288b65c437519c2a7f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.214410 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4173/8317] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:23:00 2022] DELETE /identity/v3/users/f5afa360fc4c40288b65c437519c2a7f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:23:00.219472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:00.219888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:00.220288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:00.241649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:00.243076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:00.312767 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4145/8318] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:00.324944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:00.327078 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:00.431568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349fbaf-dcb2-4019-880c-3d8c61260acf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:00.436127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:00.436818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:00.437350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:00.437800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:00.448098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:00.449755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:00.462449 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4174/8319] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:00.655855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:00.657537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:00.675020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97d1f7-03fd-4e38-9e6a-88b4170d22f1 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:00.688408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:00.689873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:00.690295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:00.690752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] PATH_INFO: `/v3/projects/371635a1edd041828749d00c5526197c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:00.692236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=371635a1edd041828749d00c5526197c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:00.694455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:00.761474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371635a1edd041828749d00c5526197c', 'request_id': 'req-069eee94-1180-492e-8320-04d458a95ddb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.769195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371635a1edd041828749d00c5526197c', 'request_id': 'req-069eee94-1180-492e-8320-04d458a95ddb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.781520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371635a1edd041828749d00c5526197c', 'request_id': 'req-069eee94-1180-492e-8320-04d458a95ddb'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.796057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] The token cache is being invalidate because project 371635a1edd041828749d00c5526197c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:23:00.796491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.797223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.799188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4146/8320] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:23:00 2022] DELETE /identity/v3/projects/371635a1edd041828749d00c5526197c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:00.803163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:00.814564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d43c2-93f9-41b6-8645-c656080c560f service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:00.886623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:00.887265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:00.888180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:00.888465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] PATH_INFO: `/v3/users/9fdd78391cf6435eb54ea4974ecd0e39` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:00.889615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fdd78391cf6435eb54ea4974ecd0e39)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:00.891583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:00.943856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fdd78391cf6435eb54ea4974ecd0e39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.958025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fdd78391cf6435eb54ea4974ecd0e39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.960353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:00.971593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fdd78391cf6435eb54ea4974ecd0e39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.980214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fdd78391cf6435eb54ea4974ecd0e39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.980859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fdd78391cf6435eb54ea4974ecd0e39'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:00.987787 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4175/8321] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:23:00 2022] DELETE /identity/v3/users/9fdd78391cf6435eb54ea4974ecd0e39 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:01.300120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:01.302228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:01.318094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069eee94-1180-492e-8320-04d458a95ddb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:01.328611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:01.386202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:01.387163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:01.387163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:01.388105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] PATH_INFO: `/v3/projects/52b6406d45f34d02a69acca219c02c4c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:01.389878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b6406d45f34d02a69acca219c02c4c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:01.391885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:01.450163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b6406d45f34d02a69acca219c02c4c', 'request_id': 'req-73a48da4-cd06-4a56-b2c3-c5cae04282d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.458428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b6406d45f34d02a69acca219c02c4c', 'request_id': 'req-73a48da4-cd06-4a56-b2c3-c5cae04282d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.467865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b6406d45f34d02a69acca219c02c4c', 'request_id': 'req-73a48da4-cd06-4a56-b2c3-c5cae04282d2'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.483462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] The token cache is being invalidate because project 52b6406d45f34d02a69acca219c02c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:23:01.484134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.484910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.489453 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4147/8322] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:23:01 2022] DELETE /identity/v3/projects/52b6406d45f34d02a69acca219c02c4c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:01.490760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:01.492326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:01.507364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7784a8-dc5b-4ac4-b50b-d8b88cb91323 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:01.567845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:01.568485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:01.568985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:01.569273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] PATH_INFO: `/v3/users/839c2e5c2fc9402e8a8ceaa32afabc41` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:01.570436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] RBAC: Authorizing `identity:delete_user(user_id=839c2e5c2fc9402e8a8ceaa32afabc41)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:01.572390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:01.627204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839c2e5c2fc9402e8a8ceaa32afabc41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.641549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839c2e5c2fc9402e8a8ceaa32afabc41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.649600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839c2e5c2fc9402e8a8ceaa32afabc41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.658717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839c2e5c2fc9402e8a8ceaa32afabc41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.659707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-87bdb5c0-a0b8-4a33-9dac-4b0fdfbac43d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839c2e5c2fc9402e8a8ceaa32afabc41'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:01.667628 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4176/8323] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:23:01 2022] DELETE /identity/v3/users/839c2e5c2fc9402e8a8ceaa32afabc41 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:02.156256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:02.157897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:02.172984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a48da4-cd06-4a56-b2c3-c5cae04282d2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:02.252094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:02.252744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.253030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.253307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] PATH_INFO: `/v3/projects/0d3b288cfec049f0ae3e8bbf69c6fe65` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.254482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3b288cfec049f0ae3e8bbf69c6fe65)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:02.256636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:02.310353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:02.319200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3b288cfec049f0ae3e8bbf69c6fe65', 'request_id': 'req-3c2962d7-8bb6-4352-9caf-5c10fc511121'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:02.334803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3b288cfec049f0ae3e8bbf69c6fe65', 'request_id': 'req-3c2962d7-8bb6-4352-9caf-5c10fc511121'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:02.347177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3b288cfec049f0ae3e8bbf69c6fe65', 'request_id': 'req-3c2962d7-8bb6-4352-9caf-5c10fc511121'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:02.366526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] The token cache is being invalidate because project 0d3b288cfec049f0ae3e8bbf69c6fe65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:23:02.366820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:02.367495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-3c2962d7-8bb6-4352-9caf-5c10fc511121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:02.369688 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4148/8324] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:23:02 2022] DELETE /identity/v3/projects/0d3b288cfec049f0ae3e8bbf69c6fe65 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:02.375149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9c83-bbcc-45f0-8dd2-0083c4946162 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.375562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9c83-bbcc-45f0-8dd2-0083c4946162 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.375946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de9c83-bbcc-45f0-8dd2-0083c4946162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.405317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-86de9c83-bbcc-45f0-8dd2-0083c4946162 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:02.406814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-86de9c83-bbcc-45f0-8dd2-0083c4946162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:02.495598 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4177/8325] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:02.503915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a383ec0-b8b3-4028-850c-c88ec46ab0d4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.504224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a383ec0-b8b3-4028-850c-c88ec46ab0d4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.504768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a383ec0-b8b3-4028-850c-c88ec46ab0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.530855 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5a383ec0-b8b3-4028-850c-c88ec46ab0d4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:02.532238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5a383ec0-b8b3-4028-850c-c88ec46ab0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:02.575373 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4149/8326] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:02.579467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de9c83-bbcc-45f0-8dd2-0083c4946162 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:02.586478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:02.592215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de9c83-bbcc-45f0-8dd2-0083c4946162 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:02.602773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:02.603677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.604407 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.604975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.605977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:23:02.606381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:02.608027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:02.629302 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4178/8327] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:23:02.633726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a383ec0-b8b3-4028-850c-c88ec46ab0d4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:02.651031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a383ec0-b8b3-4028-850c-c88ec46ab0d4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:02.656760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:02.657454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.657792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.658160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.658998 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:02.660939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:02.710383 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4150/8328] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:23:02 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:02.714234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:02.726513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ff2f0-5e91-4507-8544-9485b08d3bde admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:02.812836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:02.813619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.813724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.814060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.814679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:02.816782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:02.844855 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4179/8329] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:23:02 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:02.846849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:02.859464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851a537-ff71-45f7-b078-5278433b9866 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:02.864640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:02.865387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.865765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.866167 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.867154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:02.868822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:02.880623 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4151/8330] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:23:02.884869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:02.900141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a1e7d-7767-46e4-b796-c7680deea06b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:02.905890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe904b3-74a1-4702-a0ca-b599f003a94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:02.906860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe904b3-74a1-4702-a0ca-b599f003a94d admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.906860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe904b3-74a1-4702-a0ca-b599f003a94d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.907077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe904b3-74a1-4702-a0ca-b599f003a94d admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:02.907701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe904b3-74a1-4702-a0ca-b599f003a94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:02.909683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe904b3-74a1-4702-a0ca-b599f003a94d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:02.936159 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4180/8331] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:23:02.942591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:02.967877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb78cd8-6c00-46cc-9273-9d7247f197e9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:02.985955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:02.986744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:02.987029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:02.987302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] PATH_INFO: `/v3/projects/6f238ca0ce6643da8fcbb2119e9f9424/users/4236471bb9824793a903f3a872976d42/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:03.014163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f238ca0ce6643da8fcbb2119e9f9424, user_id=4236471bb9824793a903f3a872976d42, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:03.023603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:03.040650 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4152/8332] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:23:02 2022] PUT /identity/v3/projects/6f238ca0ce6643da8fcbb2119e9f9424/users/4236471bb9824793a903f3a872976d42/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:03.046589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c70f-85c4-4385-b452-82d6b80d1088 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:03.047128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c70f-85c4-4385-b452-82d6b80d1088 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:03.047484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c70f-85c4-4385-b452-82d6b80d1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:03.070327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1113c70f-85c4-4385-b452-82d6b80d1088 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:03.071585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1113c70f-85c4-4385-b452-82d6b80d1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:03.152462 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4181/8333] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:03.179406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:03.187228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:03.275775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4212ec-550f-4f01-9b2f-cbacecc1b545 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:03.280430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:03.281205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:03.281582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:03.281984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:03.302453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:03.303766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:03.318561 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4153/8334] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:04.515826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35414-8601-4e3a-bb9f-aff859d2f37a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:04.516344 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35414-8601-4e3a-bb9f-aff859d2f37a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:04.516418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df35414-8601-4e3a-bb9f-aff859d2f37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:04.551586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5df35414-8601-4e3a-bb9f-aff859d2f37a None None] MFA Rules not processed for user `4236471bb9824793a903f3a872976d42`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:04.552997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5df35414-8601-4e3a-bb9f-aff859d2f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:04.632530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:04.642819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:04.644247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4182/8335] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:04.660372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e73ddf-f7ec-4ad8-9ecb-ad50e7343e32 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:04.665692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:04.666417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:04.666822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:04.667170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:04.667867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:04.670075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:04.721916 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4154/8336] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:23:04 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:04.723909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35414-8601-4e3a-bb9f-aff859d2f37a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:04.737327 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df35414-8601-4e3a-bb9f-aff859d2f37a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:04.824952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:04.826795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:04.827292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:04.827763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:04.829257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:04.832462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:04.866045 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4183/8337] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:23:04 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:04.869530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:04.894832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c160a-5ace-474c-a333-7ed8679cc2e1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:04.908848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:04.909524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:04.909804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:04.910168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:04.910963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:04.912882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:04.932838 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4155/8338] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:23:04.937322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:04.966588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397a29c-d150-4885-b748-ce95411e1435 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:04.973354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:04.974306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:04.974589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:04.974902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] PATH_INFO: `/v3/projects/27db86a15a9e467c8ee36f3d543cf17b/users/72440ec6e5ed40e5b9cb21c550594ab7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:05.020352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27db86a15a9e467c8ee36f3d543cf17b, user_id=72440ec6e5ed40e5b9cb21c550594ab7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:05.023138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:05.051555 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4184/8339] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:23:04 2022] PUT /identity/v3/projects/27db86a15a9e467c8ee36f3d543cf17b/users/72440ec6e5ed40e5b9cb21c550594ab7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:05.055759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:05.068174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced5bb4-2113-4097-aa2c-343112cf8e26 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:05.147864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:05.185687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a136759-a485-43f9-8add-8a21f6a4f014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:05.186573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a136759-a485-43f9-8add-8a21f6a4f014 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:05.187545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a136759-a485-43f9-8add-8a21f6a4f014 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:05.187841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a136759-a485-43f9-8add-8a21f6a4f014 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:05.188784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a136759-a485-43f9-8add-8a21f6a4f014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:05.190940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a136759-a485-43f9-8add-8a21f6a4f014 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:05.208571 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4156/8340] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:23:05.210144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:05.226965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca76264-dca6-4ea6-8482-5c20f9247938 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:05.232809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4008fa-c84e-4bf7-8e6b-3609ef34779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:05.233484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4008fa-c84e-4bf7-8e6b-3609ef34779b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:05.234051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4008fa-c84e-4bf7-8e6b-3609ef34779b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:05.234333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4008fa-c84e-4bf7-8e6b-3609ef34779b admin admin] PATH_INFO: `/v3/projects/27db86a15a9e467c8ee36f3d543cf17b/users/72440ec6e5ed40e5b9cb21c550594ab7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:05.236279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4008fa-c84e-4bf7-8e6b-3609ef34779b admin admin] RBAC: Authorizing `identity:create_grant(project_id=27db86a15a9e467c8ee36f3d543cf17b, user_id=72440ec6e5ed40e5b9cb21c550594ab7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:05.239395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4008fa-c84e-4bf7-8e6b-3609ef34779b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:05.263851 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4185/8341] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:23:05 2022] PUT /identity/v3/projects/27db86a15a9e467c8ee36f3d543cf17b/users/72440ec6e5ed40e5b9cb21c550594ab7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:06.706045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:06.706535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:06.706635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:06.763435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] MFA Rules not processed for user `72440ec6e5ed40e5b9cb21c550594ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:06.765572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:06.863517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4157/8342] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:06.874709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:06.875292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:06.875292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:06.897266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] MFA Rules not processed for user `4236471bb9824793a903f3a872976d42`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:06.898550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:06.966727 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4186/8343] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:06.977460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:06.982077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:07.057682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b8b88-2edd-4215-9829-6b13ac0f58fd None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:07.063536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:07.064160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:07.064444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:07.064722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:07.072958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:07.074780 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:07.084360 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4158/8344] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:11.992110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:11.993262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:12.157983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38e7e3-5e6d-488e-ac7c-bff78c6e46db None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:12.163279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:12.164853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:12.165380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:12.165863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:12.167902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:12.202386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:12.204341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:12.211490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:12.220019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4187/8345] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:13.522177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:13.547581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4fd3-5849-4b3e-924f-7505ff563306 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:13.567303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:13.567983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:13.568275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:13.568550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:13.570940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:13.593701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:13.595636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:13.612550 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4159/8346] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:16.912577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:16.914580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:16.937495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e70c86-e51c-4c10-b6b4-b13201462b3b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:16.943557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a856ef-5f23-4537-987d-b74abafbf6f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:16.944174 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a856ef-5f23-4537-987d-b74abafbf6f6 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:16.944453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a856ef-5f23-4537-987d-b74abafbf6f6 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:16.944730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a856ef-5f23-4537-987d-b74abafbf6f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:16.946687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-29a856ef-5f23-4537-987d-b74abafbf6f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:17.020549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a856ef-5f23-4537-987d-b74abafbf6f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:17.022222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a856ef-5f23-4537-987d-b74abafbf6f6 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:17.044733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4188/8347] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:16 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:17.336628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:17.353856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfc6ae-79c1-4b39-b003-90fafef10ad5 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:17.359079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:17.359713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:17.359983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:17.360254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:17.362783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:17.409697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:17.450081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:17.451826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:17.461820 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4160/8348] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:27.041241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=34 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:27.457426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=14 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:36.790385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51611543-b013-48d0-92fd-b278fbab5409 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:36.790856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51611543-b013-48d0-92fd-b278fbab5409 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:36.791230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51611543-b013-48d0-92fd-b278fbab5409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:36.844111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-51611543-b013-48d0-92fd-b278fbab5409 None None] MFA Rules not processed for user `b4791d13ab1c45bf8fabec70191426da`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:36.846978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-51611543-b013-48d0-92fd-b278fbab5409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:36.869969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4189/8349] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:37.569084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:37.660389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879d2b0-8a0d-4997-9ed1-7e212bc0f1fb service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:37.665609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:37.666437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:37.666723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:37.667001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] PATH_INFO: `/v3/users/4236471bb9824793a903f3a872976d42` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:37.668388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4236471bb9824793a903f3a872976d42)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:37.670518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:37.757214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4236471bb9824793a903f3a872976d42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:37.765663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4236471bb9824793a903f3a872976d42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:37.766436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4236471bb9824793a903f3a872976d42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:37.776421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4236471bb9824793a903f3a872976d42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:37.777429 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4236471bb9824793a903f3a872976d42'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:37.808173 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4161/8350] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:23:37 2022] DELETE /identity/v3/users/4236471bb9824793a903f3a872976d42 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:38.336733 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51611543-b013-48d0-92fd-b278fbab5409 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:38.359913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51611543-b013-48d0-92fd-b278fbab5409 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:38.519363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:38.520751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:38.522320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:38.523053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] PATH_INFO: `/v3/projects/6f238ca0ce6643da8fcbb2119e9f9424` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:38.524918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f238ca0ce6643da8fcbb2119e9f9424)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:38.528727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:38.632558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f238ca0ce6643da8fcbb2119e9f9424', 'request_id': 'req-26267837-d7df-45fa-a324-98c9f153690c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:38.639833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f238ca0ce6643da8fcbb2119e9f9424', 'request_id': 'req-26267837-d7df-45fa-a324-98c9f153690c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:38.647346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f238ca0ce6643da8fcbb2119e9f9424', 'request_id': 'req-26267837-d7df-45fa-a324-98c9f153690c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:38.674485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] The token cache is being invalidate because project 6f238ca0ce6643da8fcbb2119e9f9424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:23:38.675177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:38.676110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:38.677921 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4190/8351] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:23:38 2022] DELETE /identity/v3/projects/6f238ca0ce6643da8fcbb2119e9f9424 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:38.682696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:38.685335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:38.710632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0d210-4d94-4baf-9550-ea582bba999a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:38.850249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:38.850900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:38.851190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:38.851464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] PATH_INFO: `/v3/users/72440ec6e5ed40e5b9cb21c550594ab7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:38.854570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72440ec6e5ed40e5b9cb21c550594ab7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:38.858098 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:38.912013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:38.960266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72440ec6e5ed40e5b9cb21c550594ab7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:38.982433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72440ec6e5ed40e5b9cb21c550594ab7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:38.983299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72440ec6e5ed40e5b9cb21c550594ab7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.000749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72440ec6e5ed40e5b9cb21c550594ab7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.002749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4c6bcdd-fe54-4e2d-a70e-3ce32cbe3927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72440ec6e5ed40e5b9cb21c550594ab7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.034751 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4162/8352] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:23:38 2022] DELETE /identity/v3/users/72440ec6e5ed40e5b9cb21c550594ab7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:39.309178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:39.311416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:39.326487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26267837-d7df-45fa-a324-98c9f153690c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:39.409446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:39.410194 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:39.410448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:39.411117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] PATH_INFO: `/v3/projects/27db86a15a9e467c8ee36f3d543cf17b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:39.414105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27db86a15a9e467c8ee36f3d543cf17b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:39.417382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:39.483786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:39.514525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27db86a15a9e467c8ee36f3d543cf17b', 'request_id': 'req-f778226e-f268-448b-9cf1-da61da5c1b35'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.523920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27db86a15a9e467c8ee36f3d543cf17b', 'request_id': 'req-f778226e-f268-448b-9cf1-da61da5c1b35'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.532901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27db86a15a9e467c8ee36f3d543cf17b', 'request_id': 'req-f778226e-f268-448b-9cf1-da61da5c1b35'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.561585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] The token cache is being invalidate because project 27db86a15a9e467c8ee36f3d543cf17b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:23:39.562397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.562746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f778226e-f268-448b-9cf1-da61da5c1b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:23:39.565073 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4191/8353] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:23:39 2022] DELETE /identity/v3/projects/27db86a15a9e467c8ee36f3d543cf17b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:39.569814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cb6a3-e9f8-4aab-9907-7923309b13cb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:39.570128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cb6a3-e9f8-4aab-9907-7923309b13cb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:39.570409 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211cb6a3-e9f8-4aab-9907-7923309b13cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:39.618254 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-211cb6a3-e9f8-4aab-9907-7923309b13cb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:39.620173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-211cb6a3-e9f8-4aab-9907-7923309b13cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:39.720934 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4163/8354] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:39.722876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781540-0553-483f-a4fb-f3bffd8bb406 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:39.723572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781540-0553-483f-a4fb-f3bffd8bb406 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:39.723978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef781540-0553-483f-a4fb-f3bffd8bb406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:39.748005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ef781540-0553-483f-a4fb-f3bffd8bb406 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:39.749673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ef781540-0553-483f-a4fb-f3bffd8bb406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:39.765448 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4192/8355] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:39.767104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cb6a3-e9f8-4aab-9907-7923309b13cb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:39.780217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211cb6a3-e9f8-4aab-9907-7923309b13cb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:39.785537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:39.786321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:39.786605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:39.786881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:39.787587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:23:39.787898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:39.790342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:39.810896 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4164/8356] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:23:39.815368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef781540-0553-483f-a4fb-f3bffd8bb406 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:39.839824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef781540-0553-483f-a4fb-f3bffd8bb406 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:39.858780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:39.859705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:39.862865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:39.863304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:39.863995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:39.868441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:39.920165 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4193/8357] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:23:39 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:39.924896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:39.944630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc65d7e-5ac7-4fe3-98e3-4a7e661ef01a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:40.072216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:40.073434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:40.074452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:40.074940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:40.075694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:40.078170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:40.111653 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4165/8358] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:23:39 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:40.115233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:40.131258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf77ea5-c1cf-410a-8a91-41a57894e8e8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:40.148390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:40.149458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:40.150124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:40.150607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:40.151532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:40.154304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:40.172632 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4194/8359] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:23:40.176761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:40.189235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf381d8-5ed1-4eec-98d7-512ed3187dea admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:40.195295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4991f-c676-4da4-b1a6-b85f9c485fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:40.196288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4991f-c676-4da4-b1a6-b85f9c485fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:40.196654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4991f-c676-4da4-b1a6-b85f9c485fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:40.197067 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4991f-c676-4da4-b1a6-b85f9c485fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:40.197916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4991f-c676-4da4-b1a6-b85f9c485fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:40.200230 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4991f-c676-4da4-b1a6-b85f9c485fc4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:40.213548 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4166/8360] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:23:40.217764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:40.237142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b62644-3088-4750-a27c-5d90be0c9d80 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:40.245485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:40.246594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:40.247480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:40.248051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] PATH_INFO: `/v3/projects/9a75b7945f9c4494bfbc75922e8620d3/users/24695f4aa38e411a9ac4cc4eb0727839/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:40.270751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a75b7945f9c4494bfbc75922e8620d3, user_id=24695f4aa38e411a9ac4cc4eb0727839, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:40.274786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:40.302294 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4195/8361] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:23:40 2022] PUT /identity/v3/projects/9a75b7945f9c4494bfbc75922e8620d3/users/24695f4aa38e411a9ac4cc4eb0727839/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:40.313712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ad4b-3f34-47e5-8802-b2d09152480c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:40.313712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ad4b-3f34-47e5-8802-b2d09152480c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:40.313935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6ad4b-3f34-47e5-8802-b2d09152480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:40.336310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:40.370966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c0d6ad4b-3f34-47e5-8802-b2d09152480c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:40.372519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c0d6ad4b-3f34-47e5-8802-b2d09152480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:40.532527 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4167/8362] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:40.545402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:40.547643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:40.722284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544460-cf4f-42a8-bdc5-4d96b750fc74 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:40.732066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:40.732709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:40.732995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:40.733273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:40.752611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:40.755095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:40.769055 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4196/8363] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:23:49.015955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9f576-10af-4761-b7c3-54736c0b7ceb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:49.016547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9f576-10af-4761-b7c3-54736c0b7ceb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:49.016547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9f576-10af-4761-b7c3-54736c0b7ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:49.048577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9fb9f576-10af-4761-b7c3-54736c0b7ceb None None] MFA Rules not processed for user `24695f4aa38e411a9ac4cc4eb0727839`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:23:49.050065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9fb9f576-10af-4761-b7c3-54736c0b7ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:23:49.130210 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4168/8364] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:23:49.134469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:49.154952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8b0a4-f947-485b-b754-f346c4e84ecd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:49.164046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:49.166387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:49.166918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:49.168262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:49.169964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:49.174650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:49.220253 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4197/8365] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:23:49 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:49.224555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9f576-10af-4761-b7c3-54736c0b7ceb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:49.239841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9f576-10af-4761-b7c3-54736c0b7ceb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:49.317392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:49.318205 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:49.318605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:49.318986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:49.320188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:49.322333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:49.353275 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4169/8366] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:23:49 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:23:49.354675 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:49.375001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e1ca7-f238-42f2-8d04-78ed4cdd838a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:49.380212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:49.380856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:49.381181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:49.381465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:49.382165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:49.383831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:49.399771 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4198/8367] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:23:49.404444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:49.423444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ca4c-6a3e-4a3e-b67b-69ace62ccd82 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:49.437211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:49.438558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:49.439028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:49.439267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] PATH_INFO: `/v3/projects/f576db116ebd4a08b154b7a0855cbaf4/users/37f290a75734494288016f45830ec12b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:49.471614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f576db116ebd4a08b154b7a0855cbaf4, user_id=37f290a75734494288016f45830ec12b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:49.476059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:49.499929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:49.502795 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4170/8368] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:23:49 2022] PUT /identity/v3/projects/f576db116ebd4a08b154b7a0855cbaf4/users/37f290a75734494288016f45830ec12b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:49.506751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:49.534967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385789-c7f1-4d58-a045-7236424ecc86 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:49.560348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:49.620247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46394b-f957-426b-9bf9-b5dff3858cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:49.621227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46394b-f957-426b-9bf9-b5dff3858cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:49.621534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46394b-f957-426b-9bf9-b5dff3858cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:49.621810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46394b-f957-426b-9bf9-b5dff3858cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:49.622503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46394b-f957-426b-9bf9-b5dff3858cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:49.624234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46394b-f957-426b-9bf9-b5dff3858cb0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:49.636733 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4199/8369] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:23:49.640825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:23:49.655731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a434e-8572-4bc1-9df8-c5eb75ee5cb8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:23:49.660804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:23:49.661495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:23:49.661787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:23:49.662096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] PATH_INFO: `/v3/projects/f576db116ebd4a08b154b7a0855cbaf4/users/37f290a75734494288016f45830ec12b/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:23:49.664140 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f576db116ebd4a08b154b7a0855cbaf4, user_id=37f290a75734494288016f45830ec12b, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:23:49.666875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:23:49.683667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4171/8370] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:23:49 2022] PUT /identity/v3/projects/f576db116ebd4a08b154b7a0855cbaf4/users/37f290a75734494288016f45830ec12b/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:23:59.633239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=20 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:23:59.675552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=5,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:00.022254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:00.022491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:00.022773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:00.095313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] MFA Rules not processed for user `37f290a75734494288016f45830ec12b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:24:00.096757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:00.242588 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4200/8371] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:24:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:24:00.287532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:00.291004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:00.410689 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c4252-3986-4d63-9803-dcf9574f7043 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:00.415860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:00.417344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:00.418710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:00.418951 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:00.672406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:00.673987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:00.685273 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4172/8372] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:10.237111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=34 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:10.681211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=48 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:16.380803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:16.383025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:16.486855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843074c-a508-4c94-8d25-12c84b740a9d None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:16.491507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:16.491868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:16.492322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:16.492749 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:16.495918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:16.619200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:16.622704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:16.659115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4201/8373] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:16.701351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:16.798988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e53d85-f20c-44a5-8d80-d44b7768ddde service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:16.803863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:16.804580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:16.804934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:16.805438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] PATH_INFO: `/v3/users/24695f4aa38e411a9ac4cc4eb0727839` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:16.806761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24695f4aa38e411a9ac4cc4eb0727839)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:16.809159 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:16.869011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24695f4aa38e411a9ac4cc4eb0727839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:16.878667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24695f4aa38e411a9ac4cc4eb0727839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:16.879441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24695f4aa38e411a9ac4cc4eb0727839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:16.890040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24695f4aa38e411a9ac4cc4eb0727839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:16.890913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24695f4aa38e411a9ac4cc4eb0727839'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:16.955511 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4173/8374] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:24:16 2022] DELETE /identity/v3/users/24695f4aa38e411a9ac4cc4eb0727839 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:24:17.271833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:17.283960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692acb2c-e177-4e27-99fe-a4b10e0471b4 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:17.355652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:17.356417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:17.356774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:17.357122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] PATH_INFO: `/v3/projects/9a75b7945f9c4494bfbc75922e8620d3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:17.358403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a75b7945f9c4494bfbc75922e8620d3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:17.360664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:17.411043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a75b7945f9c4494bfbc75922e8620d3', 'request_id': 'req-6ffc91eb-e9ce-457d-8325-3d683c193381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.415865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:17.417457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a75b7945f9c4494bfbc75922e8620d3', 'request_id': 'req-6ffc91eb-e9ce-457d-8325-3d683c193381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.425230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a75b7945f9c4494bfbc75922e8620d3', 'request_id': 'req-6ffc91eb-e9ce-457d-8325-3d683c193381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.439092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] The token cache is being invalidate because project 9a75b7945f9c4494bfbc75922e8620d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:24:17.439390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.440169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.441933 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4202/8375] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:24:17 2022] DELETE /identity/v3/projects/9a75b7945f9c4494bfbc75922e8620d3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:24:17.445746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:17.447247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:17.461603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731db02b-2138-46b4-89a5-3787ca1c9720 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:17.522583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:17.523234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:17.523518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:17.523794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] PATH_INFO: `/v3/users/37f290a75734494288016f45830ec12b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:17.524983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f290a75734494288016f45830ec12b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:17.527524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:17.550001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:17.578395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f290a75734494288016f45830ec12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.586111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f290a75734494288016f45830ec12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.586792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f290a75734494288016f45830ec12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.595482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f290a75734494288016f45830ec12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.596299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f290a75734494288016f45830ec12b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:17.615214 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4174/8376] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:24:17 2022] DELETE /identity/v3/users/37f290a75734494288016f45830ec12b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:24:17.837774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:17.840341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:17.855210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc91eb-e9ce-457d-8325-3d683c193381 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:17.940316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:17.942318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:17.942692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:17.942982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] PATH_INFO: `/v3/projects/f576db116ebd4a08b154b7a0855cbaf4` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:17.944237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f576db116ebd4a08b154b7a0855cbaf4)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:17.946432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:18.016850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f576db116ebd4a08b154b7a0855cbaf4', 'request_id': 'req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:18.023541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f576db116ebd4a08b154b7a0855cbaf4', 'request_id': 'req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:18.032967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f576db116ebd4a08b154b7a0855cbaf4', 'request_id': 'req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:18.050195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] The token cache is being invalidate because project f576db116ebd4a08b154b7a0855cbaf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:24:18.050617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:18.051425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4cdc3c0e-a0bc-4fb0-9aab-99958f0defa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:18.053352 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4203/8377] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:24:17 2022] DELETE /identity/v3/projects/f576db116ebd4a08b154b7a0855cbaf4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:24:20.543392 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:20.544858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:20.561942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6da96-df56-443d-a488-58105cb7bb83 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:20.620781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:20.621339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:20.621625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:20.621932 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] PATH_INFO: `/v3/users/eb238d6a62be4772ae6535fce36f9239` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:20.623049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb238d6a62be4772ae6535fce36f9239)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:20.624974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:20.673279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb238d6a62be4772ae6535fce36f9239'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:20.680682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb238d6a62be4772ae6535fce36f9239'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:20.681359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb238d6a62be4772ae6535fce36f9239'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:20.688978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb238d6a62be4772ae6535fce36f9239'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:20.689580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb238d6a62be4772ae6535fce36f9239'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:20.708351 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4175/8378] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:24:20 2022] DELETE /identity/v3/users/eb238d6a62be4772ae6535fce36f9239 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:24:20.711382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:20.711861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:20.711861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:20.733876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:24:20.735402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:20.796951 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4204/8379] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:24:20.807162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:20.808889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:20.829364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:20.871805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cd5de-8502-40fd-a76e-c3c5d115cace admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:20.875426 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b133e-0e83-47a5-9871-5833c121b53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:20.876025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b133e-0e83-47a5-9871-5833c121b53a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:20.876302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b133e-0e83-47a5-9871-5833c121b53a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:20.876575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b133e-0e83-47a5-9871-5833c121b53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:20.886443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b133e-0e83-47a5-9871-5833c121b53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:20.887872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b133e-0e83-47a5-9871-5833c121b53a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:20.897369 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4176/8380] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:21.070907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.072367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:21.086416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f9cf5-007b-47f2-ae8a-a9acc333e415 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.091668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.092307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.092594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.092866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] PATH_INFO: `/v3/projects/675281d0efa249ad9c4ccf87e67b8e2d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.094035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=675281d0efa249ad9c4ccf87e67b8e2d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.096714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.108598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:21.147459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '675281d0efa249ad9c4ccf87e67b8e2d', 'request_id': 'req-8daa1588-93ed-4a30-84f5-d45e8a746ebe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:21.153560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '675281d0efa249ad9c4ccf87e67b8e2d', 'request_id': 'req-8daa1588-93ed-4a30-84f5-d45e8a746ebe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:21.159886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '675281d0efa249ad9c4ccf87e67b8e2d', 'request_id': 'req-8daa1588-93ed-4a30-84f5-d45e8a746ebe'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:21.172628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] The token cache is being invalidate because project 675281d0efa249ad9c4ccf87e67b8e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:24:21.172918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:21.173547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8daa1588-93ed-4a30-84f5-d45e8a746ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:24:21.175058 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4205/8381] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:24:21 2022] DELETE /identity/v3/projects/675281d0efa249ad9c4ccf87e67b8e2d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:24:21.180118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4da9-cee1-4185-959d-ce91dce3b466 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.180404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4da9-cee1-4185-959d-ce91dce3b466 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.180678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4da9-cee1-4185-959d-ce91dce3b466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.200145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-440a4da9-cee1-4185-959d-ce91dce3b466 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:24:21.201345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-440a4da9-cee1-4185-959d-ce91dce3b466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:21.263461 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4177/8382] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:24:21.271014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332cc2-1e7f-4a4b-8d51-c3f9a28df254 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.271298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332cc2-1e7f-4a4b-8d51-c3f9a28df254 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.271571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332cc2-1e7f-4a4b-8d51-c3f9a28df254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.291378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7a332cc2-1e7f-4a4b-8d51-c3f9a28df254 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:24:21.292613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7a332cc2-1e7f-4a4b-8d51-c3f9a28df254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:21.300402 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4206/8383] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:24:21.304471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a4da9-cee1-4185-959d-ce91dce3b466 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.314715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a4da9-cee1-4185-959d-ce91dce3b466 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.319236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.319828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.320108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.320381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.321018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:24:21.321318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.323623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.335035 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4178/8384] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:24:21.339296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332cc2-1e7f-4a4b-8d51-c3f9a28df254 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.348496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332cc2-1e7f-4a4b-8d51-c3f9a28df254 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.353295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.353964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.354249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.354537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.355173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.357084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.384642 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4207/8385] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:24:21 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:24:21.389229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.399319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cea37-99ad-4657-b77b-aa3a35040234 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.456353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.456956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.457240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.457517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.458199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.460120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.480539 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4179/8386] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:24:21 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:24:21.484953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.496061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bbcc8-562a-49b3-8046-e59e0b1a7813 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.500709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.501308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.501592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.501906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.502546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.504131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.514856 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4208/8387] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:24:21.518752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.529778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeb16c-ff7d-4b1f-a03c-df4d87a4a186 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.534363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6790d-6fbb-4cd2-85a4-c6235a7009eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.534952 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6790d-6fbb-4cd2-85a4-c6235a7009eb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.535233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6790d-6fbb-4cd2-85a4-c6235a7009eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.535506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6790d-6fbb-4cd2-85a4-c6235a7009eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.536134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6790d-6fbb-4cd2-85a4-c6235a7009eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.537706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6790d-6fbb-4cd2-85a4-c6235a7009eb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.548647 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4180/8388] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:24:21.552770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.563606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a208fad-af1d-4bf4-aa24-2bf48f14e889 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.568300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.568962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.569240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.569509 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] PATH_INFO: `/v3/projects/ea3948a3308440058b1cc46e2f4a6c0d/users/702a7991b9b846b480e2769bad5d9cef/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.583983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3948a3308440058b1cc46e2f4a6c0d, user_id=702a7991b9b846b480e2769bad5d9cef, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.586638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.600215 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4209/8389] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:24:21 2022] PUT /identity/v3/projects/ea3948a3308440058b1cc46e2f4a6c0d/users/702a7991b9b846b480e2769bad5d9cef/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:24:21.612620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.612898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.613171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.641434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] MFA Rules not processed for user `702a7991b9b846b480e2769bad5d9cef`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:24:21.642686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:21.655919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:21.709183 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4181/8390] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:24:21.719738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:21.721488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:21.781741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60191484-a36e-4510-a173-82267b529b46 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:21.785387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:21.786143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:21.786423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:21.786692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:21.795736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:21.797157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:21.806021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4210/8391] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:23.630803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:23.632483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:23.648976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e9b1b-b9f8-4002-b4c6-890472209be9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:23.653394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:23.654265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:23.654568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:23.654948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:23.657158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:23.673738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:23.675347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:23.687849 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4182/8392] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:30.820753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:30.822316 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:30.866089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:30.886866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574538f-f92d-4161-b989-678ab9a8d0ab service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:30.890849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:30.891460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:30.891742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:30.892017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:30.893750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:30.957227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:30.962103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:30.973668 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4211/8393] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:33.680478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=26 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:33.959654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:33.982314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3b192-3fde-429a-8203-3b84209f02aa service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:33.987318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63ea50-83f1-4565-a60d-b1fec99c24c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:33.988206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63ea50-83f1-4565-a60d-b1fec99c24c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:33.988648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63ea50-83f1-4565-a60d-b1fec99c24c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:33.988931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63ea50-83f1-4565-a60d-b1fec99c24c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:33.991622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8f63ea50-83f1-4565-a60d-b1fec99c24c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:34.006907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63ea50-83f1-4565-a60d-b1fec99c24c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:34.008643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63ea50-83f1-4565-a60d-b1fec99c24c4 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:34.021429 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4183/8394] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:35.178058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:24:35.179909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:35.371527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d60027-b99d-4339-936f-c5d7e079f1ea service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:24:35.389988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:24:35.392025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:24:35.393391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:24:35.393860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:24:35.399340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:24:35.426142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:24:35.428016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:24:35.484132 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4212/8395] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:24:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:24:44.014492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:24:45.440973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=65 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:04.172193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19520b-f759-4c1d-a2b4-ce00be6a006f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:04.172762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19520b-f759-4c1d-a2b4-ce00be6a006f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:04.172878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19520b-f759-4c1d-a2b4-ce00be6a006f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:04.201270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7c19520b-f759-4c1d-a2b4-ce00be6a006f None None] MFA Rules not processed for user `b4791d13ab1c45bf8fabec70191426da`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:04.202789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7c19520b-f759-4c1d-a2b4-ce00be6a006f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:04.279092 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4184/8396] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:05.383060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:05.386177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:05.606672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d91b92-b8eb-48fd-9643-5a4e5126d819 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:05.619807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b1793-8587-47f3-b099-767fc4e73305 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:05.620633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b1793-8587-47f3-b099-767fc4e73305 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:05.621057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b1793-8587-47f3-b099-767fc4e73305 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:05.621469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b1793-8587-47f3-b099-767fc4e73305 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:05.623573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6e4b1793-8587-47f3-b099-767fc4e73305 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:05.642962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b1793-8587-47f3-b099-767fc4e73305 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:05.645039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b1793-8587-47f3-b099-767fc4e73305 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:05.672471 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4213/8397] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:14.267487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=25 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:15.653253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=32 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:31.351184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19520b-f759-4c1d-a2b4-ce00be6a006f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:31.416225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19520b-f759-4c1d-a2b4-ce00be6a006f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:31.420789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:31.421420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:31.421706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:31.422022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] PATH_INFO: `/v3/users/702a7991b9b846b480e2769bad5d9cef` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:31.423143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702a7991b9b846b480e2769bad5d9cef)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:31.425073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:31.483460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702a7991b9b846b480e2769bad5d9cef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:31.491274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702a7991b9b846b480e2769bad5d9cef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:31.491974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702a7991b9b846b480e2769bad5d9cef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:31.500769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702a7991b9b846b480e2769bad5d9cef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:31.501523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702a7991b9b846b480e2769bad5d9cef'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:31.519745 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4185/8398] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:31 2022] DELETE /identity/v3/users/702a7991b9b846b480e2769bad5d9cef => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:31.524854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:31.525331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:31.525595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:31.545471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:31.546766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:31.616618 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4214/8399] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:31.618338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:31.619888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:31.627345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:31.629086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:31.690992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56271894-1000-4d75-8879-81f0928d1d70 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:31.698587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34095a88-93c8-43bc-98f9-c1e423a2de2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:31.699301 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34095a88-93c8-43bc-98f9-c1e423a2de2f service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:31.699579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d76a78-6d58-4ca9-8465-99f26daf06de None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:31.699739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34095a88-93c8-43bc-98f9-c1e423a2de2f service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:31.699904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34095a88-93c8-43bc-98f9-c1e423a2de2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:31.701747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-34095a88-93c8-43bc-98f9-c1e423a2de2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:31.703471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:31.704091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:31.704379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:31.704658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:31.717117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:31.718842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:31.730327 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4215/8400] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:31.743830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:31.788421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34095a88-93c8-43bc-98f9-c1e423a2de2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:31.790945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34095a88-93c8-43bc-98f9-c1e423a2de2f service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:31.803347 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4186/8401] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:31.982488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:31.983890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:32.004590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312308b8-9769-449e-9cf3-efc5a16d38a5 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.010397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.015037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.015335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.015613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] PATH_INFO: `/v3/projects/ea3948a3308440058b1cc46e2f4a6c0d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.016916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea3948a3308440058b1cc46e2f4a6c0d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.018979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.072281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea3948a3308440058b1cc46e2f4a6c0d', 'request_id': 'req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:32.081499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea3948a3308440058b1cc46e2f4a6c0d', 'request_id': 'req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:32.089977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea3948a3308440058b1cc46e2f4a6c0d', 'request_id': 'req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:32.106734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] The token cache is being invalidate because project ea3948a3308440058b1cc46e2f4a6c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:32.107033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:32.107756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-59fe2184-e6d1-4ba5-a1b2-95f28d353fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:32.109259 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4216/8402] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:31 2022] DELETE /identity/v3/projects/ea3948a3308440058b1cc46e2f4a6c0d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:32.116361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b61a-a38d-4c94-af8b-a8e23f8ee6ab None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.116713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b61a-a38d-4c94-af8b-a8e23f8ee6ab None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.117084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b61a-a38d-4c94-af8b-a8e23f8ee6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.139224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8590b61a-a38d-4c94-af8b-a8e23f8ee6ab None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:32.140476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8590b61a-a38d-4c94-af8b-a8e23f8ee6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:32.204471 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4187/8403] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:32.212228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303e114-4f3b-436b-b2cc-36cb64c693e3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.212617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303e114-4f3b-436b-b2cc-36cb64c693e3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.212961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303e114-4f3b-436b-b2cc-36cb64c693e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.233504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7303e114-4f3b-436b-b2cc-36cb64c693e3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:32.234843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7303e114-4f3b-436b-b2cc-36cb64c693e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:32.244863 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4217/8404] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:32.248874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590b61a-a38d-4c94-af8b-a8e23f8ee6ab None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.259635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590b61a-a38d-4c94-af8b-a8e23f8ee6ab None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.264694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.265312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.265593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.265908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.266565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:32.266926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.268575 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.285098 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4188/8405] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:25:32.286871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303e114-4f3b-436b-b2cc-36cb64c693e3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.297667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303e114-4f3b-436b-b2cc-36cb64c693e3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.302583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.303192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.303497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.303883 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.304635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.307013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.328838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:32.337731 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4218/8406] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:32 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:32.341571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.352407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961a5c-8f4a-484b-95ea-8b7f49a7d4ee admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.412506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.413116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.413400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.413671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.414428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.417170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.439106 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4189/8407] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:32 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:32.442926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.454238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdea94-7ac3-4c9e-96d2-3e8d262c8dd9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.458885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.459487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.459771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.460053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.460691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.462454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.472289 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4219/8408] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:32.476108 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.486595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d0a12-ac9c-4ccb-a71a-0f936a7b8930 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.491244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3533b-6592-41f8-9419-0acce8f94595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.491841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3533b-6592-41f8-9419-0acce8f94595 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.492122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3533b-6592-41f8-9419-0acce8f94595 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.492398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3533b-6592-41f8-9419-0acce8f94595 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.493570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3533b-6592-41f8-9419-0acce8f94595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.495202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3533b-6592-41f8-9419-0acce8f94595 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.505723 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4190/8409] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:32.509843 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.519858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c99ba-199f-410b-a0d8-66579bab7807 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.525034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.525677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.526002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.526289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] PATH_INFO: `/v3/projects/623c3d73e9a647f1a0f8e3754bf467cc/users/fc93901e34814da8b6a1179e78b8b67f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.540732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=623c3d73e9a647f1a0f8e3754bf467cc, user_id=fc93901e34814da8b6a1179e78b8b67f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.543426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.556871 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4220/8410] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:32 2022] PUT /identity/v3/projects/623c3d73e9a647f1a0f8e3754bf467cc/users/fc93901e34814da8b6a1179e78b8b67f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:32.568144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c667f-1cb3-4be1-b434-ce7223e48379 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.568432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c667f-1cb3-4be1-b434-ce7223e48379 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.568709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c667f-1cb3-4be1-b434-ce7223e48379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.596680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-565c667f-1cb3-4be1-b434-ce7223e48379 None None] MFA Rules not processed for user `fc93901e34814da8b6a1179e78b8b67f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:32.597923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-565c667f-1cb3-4be1-b434-ce7223e48379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:32.622468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:32.665670 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4191/8411] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:32.670287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.732281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4db37-a779-48ad-bce5-40766f97cb5a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.737296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.737933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.738250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.738526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.739164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.741059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.767740 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4221/8412] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:32 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:32.771657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c667f-1cb3-4be1-b434-ce7223e48379 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.783053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c667f-1cb3-4be1-b434-ce7223e48379 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.839522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.840052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.840334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.840607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.841223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.843247 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.864542 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4192/8413] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:32 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:32.868478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.878791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0a451-ff15-43d7-800e-581db8da9213 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.884108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.884698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.884979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.885325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.886017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.887686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.898110 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4222/8414] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:32.902832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.912918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aed485-b0ac-4952-ac94-4836499e62a9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:32.917847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:32.918526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:32.918809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:32.919079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] PATH_INFO: `/v3/projects/e62d0d30b82e4aae9bc6e38778517444/users/8bb81d37da7d41b78f79618f0ab50fba/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:32.935376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62d0d30b82e4aae9bc6e38778517444, user_id=8bb81d37da7d41b78f79618f0ab50fba, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:32.938058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:32.951648 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4193/8415] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:32 2022] PUT /identity/v3/projects/e62d0d30b82e4aae9bc6e38778517444/users/8bb81d37da7d41b78f79618f0ab50fba/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:32.955746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:32.971731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b78b5-b6fe-4b51-955c-c370f248381f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:33.035502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69973b22-3f21-4a36-8680-bf0f2147a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:33.036116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973b22-3f21-4a36-8680-bf0f2147a604 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:33.036403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973b22-3f21-4a36-8680-bf0f2147a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:33.036759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973b22-3f21-4a36-8680-bf0f2147a604 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:33.037307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69973b22-3f21-4a36-8680-bf0f2147a604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:33.038931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69973b22-3f21-4a36-8680-bf0f2147a604 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:33.049593 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4223/8416] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:33.053730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:33.068381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d001a-5d49-4e1c-8549-2530dc0e0d8b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:33.073948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:33.074610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:33.074891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:33.075171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] PATH_INFO: `/v3/projects/e62d0d30b82e4aae9bc6e38778517444/users/8bb81d37da7d41b78f79618f0ab50fba/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:33.077166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62d0d30b82e4aae9bc6e38778517444, user_id=8bb81d37da7d41b78f79618f0ab50fba, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:33.079770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:33.091365 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4194/8417] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:33 2022] PUT /identity/v3/projects/e62d0d30b82e4aae9bc6e38778517444/users/8bb81d37da7d41b78f79618f0ab50fba/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:33.101716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d187-8307-40b7-b5c2-41b303583233 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:33.102272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d187-8307-40b7-b5c2-41b303583233 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:33.102607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45d187-8307-40b7-b5c2-41b303583233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:33.133974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5a45d187-8307-40b7-b5c2-41b303583233 None None] MFA Rules not processed for user `8bb81d37da7d41b78f79618f0ab50fba`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:33.135195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5a45d187-8307-40b7-b5c2-41b303583233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:33.157104 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:33.210185 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4224/8418] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:33.222384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:33.224245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:33.287636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3ff7-d272-4f73-ba8c-7cdea7413aaf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:33.291825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774f2da-afb0-412a-87a4-10a80c236344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:33.292433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774f2da-afb0-412a-87a4-10a80c236344 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:33.292731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774f2da-afb0-412a-87a4-10a80c236344 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:33.293120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774f2da-afb0-412a-87a4-10a80c236344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:33.303348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774f2da-afb0-412a-87a4-10a80c236344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:33.305459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774f2da-afb0-412a-87a4-10a80c236344 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:33.315810 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4195/8419] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:33.746444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d187-8307-40b7-b5c2-41b303583233 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:33.809573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45d187-8307-40b7-b5c2-41b303583233 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:33.814216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:33.814834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:33.815114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:33.815387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] PATH_INFO: `/v3/users/fc93901e34814da8b6a1179e78b8b67f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:33.816483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc93901e34814da8b6a1179e78b8b67f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:33.818434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:33.868146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc93901e34814da8b6a1179e78b8b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:33.880241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc93901e34814da8b6a1179e78b8b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:33.887817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc93901e34814da8b6a1179e78b8b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:33.894637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc93901e34814da8b6a1179e78b8b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:33.895511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc93901e34814da8b6a1179e78b8b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:33.902223 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4225/8420] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:33 2022] DELETE /identity/v3/users/fc93901e34814da8b6a1179e78b8b67f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:33.906743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:33.907202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:33.907316 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:33.926866 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:33.928064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:33.941281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:33.990999 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4196/8421] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:34.000974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:34.002446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:34.045925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:34.065294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c570a58-0ddc-4b91-aceb-94bec9136b02 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:34.069004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:34.069636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:34.069954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:34.070257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:34.079637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:34.081068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:34.091172 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4226/8422] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:34.364898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:34.366388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:34.388706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6295b9f-4f53-4378-b622-9ca23ceeb9ef None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:34.393965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:34.394637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:34.394931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:34.395199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] PATH_INFO: `/v3/projects/623c3d73e9a647f1a0f8e3754bf467cc` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:34.396321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=623c3d73e9a647f1a0f8e3754bf467cc)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:34.398427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:34.447923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '623c3d73e9a647f1a0f8e3754bf467cc', 'request_id': 'req-e8070b80-9343-4435-844a-7cff0019de57'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.458596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '623c3d73e9a647f1a0f8e3754bf467cc', 'request_id': 'req-e8070b80-9343-4435-844a-7cff0019de57'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.463682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '623c3d73e9a647f1a0f8e3754bf467cc', 'request_id': 'req-e8070b80-9343-4435-844a-7cff0019de57'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.477601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] The token cache is being invalidate because project 623c3d73e9a647f1a0f8e3754bf467cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:34.477924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.478648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.480392 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4197/8423] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:34 2022] DELETE /identity/v3/projects/623c3d73e9a647f1a0f8e3754bf467cc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:34.484182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:34.496596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaee5a9-e9f6-41d4-a5dd-b419b3251ffe service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:34.557567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:34.558251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:34.558533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:34.558805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] PATH_INFO: `/v3/users/8bb81d37da7d41b78f79618f0ab50fba` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:34.559911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb81d37da7d41b78f79618f0ab50fba)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:34.561967 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:34.616036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb81d37da7d41b78f79618f0ab50fba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.634941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb81d37da7d41b78f79618f0ab50fba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.642146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb81d37da7d41b78f79618f0ab50fba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.650752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb81d37da7d41b78f79618f0ab50fba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.655748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb81d37da7d41b78f79618f0ab50fba'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.658494 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4227/8424] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:34 2022] DELETE /identity/v3/users/8bb81d37da7d41b78f79618f0ab50fba => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:34.804450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:34.806118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:34.832653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070b80-9343-4435-844a-7cff0019de57 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:34.892867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:34.893514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:34.893844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:34.894123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] PATH_INFO: `/v3/users/8d2bcdaa6eda42b0bd561d657c1072ac` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:34.895274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2bcdaa6eda42b0bd561d657c1072ac)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:34.897201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:34.945099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:34.946648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:34.950188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:34.952524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2bcdaa6eda42b0bd561d657c1072ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.964485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de3462-39ba-4671-ad52-d8bc1df2179b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:34.965623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2bcdaa6eda42b0bd561d657c1072ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.967170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2bcdaa6eda42b0bd561d657c1072ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.972880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:34.973621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:34.973931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:34.974208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] PATH_INFO: `/v3/projects/e62d0d30b82e4aae9bc6e38778517444` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:34.975441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62d0d30b82e4aae9bc6e38778517444)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:34.978393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:34.981088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2bcdaa6eda42b0bd561d657c1072ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:34.981897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-25365161-0116-4e22-a5aa-a4bbcfc9a72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2bcdaa6eda42b0bd561d657c1072ac'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.001602 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4198/8425] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:34 2022] DELETE /identity/v3/users/8d2bcdaa6eda42b0bd561d657c1072ac => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:35.006017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519311f9-be07-4872-b730-12e9a8981396 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.006306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519311f9-be07-4872-b730-12e9a8981396 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.006577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519311f9-be07-4872-b730-12e9a8981396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.026626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-519311f9-be07-4872-b730-12e9a8981396 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:35.027829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-519311f9-be07-4872-b730-12e9a8981396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.031824 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62d0d30b82e4aae9bc6e38778517444', 'request_id': 'req-523ab29f-5537-4de8-a240-bb9880f265ec'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.036263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:35.038759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62d0d30b82e4aae9bc6e38778517444', 'request_id': 'req-523ab29f-5537-4de8-a240-bb9880f265ec'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.047124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62d0d30b82e4aae9bc6e38778517444', 'request_id': 'req-523ab29f-5537-4de8-a240-bb9880f265ec'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.062179 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] The token cache is being invalidate because project e62d0d30b82e4aae9bc6e38778517444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:35.062555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.063094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-523ab29f-5537-4de8-a240-bb9880f265ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.064838 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4228/8426] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:34 2022] DELETE /identity/v3/projects/e62d0d30b82e4aae9bc6e38778517444 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:35.069738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5819f09-e112-4bcd-ae73-38e950309402 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.070069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5819f09-e112-4bcd-ae73-38e950309402 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.071081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5819f09-e112-4bcd-ae73-38e950309402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.090306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4199/8427] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:35.094339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a5819f09-e112-4bcd-ae73-38e950309402 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:35.095280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a5819f09-e112-4bcd-ae73-38e950309402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.100372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519311f9-be07-4872-b730-12e9a8981396 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.102599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-519311f9-be07-4872-b730-12e9a8981396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.106469 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4229/8428] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:35.113956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e159-67d0-439c-8a22-12e9df10e205 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.114246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e159-67d0-439c-8a22-12e9df10e205 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.114522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e159-67d0-439c-8a22-12e9df10e205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.135387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d9e2e159-67d0-439c-8a22-12e9df10e205 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:35.136669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d9e2e159-67d0-439c-8a22-12e9df10e205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.145238 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4230/8429] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:35.150141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e159-67d0-439c-8a22-12e9df10e205 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.159197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e159-67d0-439c-8a22-12e9df10e205 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.163615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519311f9-be07-4872-b730-12e9a8981396 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.163840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.165080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.165368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.165818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.166483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:35.166788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.168046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.168374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.168842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.169153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.169427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.180967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.181705 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4231/8430] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:25:35.182513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.185758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.192379 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4200/8431] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:35.195719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93907d43-92da-4541-81dd-101c14d36128 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.200491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.201105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.201376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.201653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.202353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.204284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.235987 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4232/8432] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:35.237071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.247561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31caf2d-01fc-4117-ad7d-a652e56a0a12 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.309614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.310300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.310591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.310872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.311518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.313485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.325799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:35.338343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4201/8433] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/users => generated 281 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:35.342047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.354214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd2eb9-c35c-4b5e-beae-eb542e2ccd5a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.359156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.359813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.360364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.360619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.361268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.362959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.376870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4233/8434] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:35.378130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.389163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac742388-41d3-459c-991c-1873c832c69a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.394366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257257-aed1-41b4-9160-ecc76dc691cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.395000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257257-aed1-41b4-9160-ecc76dc691cb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.395349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257257-aed1-41b4-9160-ecc76dc691cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.395554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257257-aed1-41b4-9160-ecc76dc691cb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.396199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257257-aed1-41b4-9160-ecc76dc691cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.397817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257257-aed1-41b4-9160-ecc76dc691cb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.409188 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4202/8435] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:35.413426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.424390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d107a690-7ee8-4af7-be44-08c998ea8332 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.429175 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.431032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.431311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.431585 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] PATH_INFO: `/v3/projects/fd75d638d6b1478f9fd7ba29199ab192/users/691d3a624ebc4a70ad48e90e44a4e326/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.447396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd75d638d6b1478f9fd7ba29199ab192, user_id=691d3a624ebc4a70ad48e90e44a4e326, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.450092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.468438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4234/8436] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:35 2022] PUT /identity/v3/projects/fd75d638d6b1478f9fd7ba29199ab192/users/691d3a624ebc4a70ad48e90e44a4e326/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:35.480811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e881d-fc3a-4fea-8781-0dbdb41b4c3b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.481102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e881d-fc3a-4fea-8781-0dbdb41b4c3b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.481377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e881d-fc3a-4fea-8781-0dbdb41b4c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.495239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.495924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.525742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6d2e881d-fc3a-4fea-8781-0dbdb41b4c3b None None] MFA Rules not processed for user `691d3a624ebc4a70ad48e90e44a4e326`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:35.527459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6d2e881d-fc3a-4fea-8781-0dbdb41b4c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.567535 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb08a8e-356c-42f2-97ec-62a8b1a85407 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.572335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.572964 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.573243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.573522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] PATH_INFO: `/v3/projects/6d1050bedc7e43f29cfc5aab839e75cb` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.574684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1050bedc7e43f29cfc5aab839e75cb)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.576598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.593999 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4203/8437] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:35.598293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e881d-fc3a-4fea-8781-0dbdb41b4c3b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.609479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e881d-fc3a-4fea-8781-0dbdb41b4c3b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.615289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.615893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.616177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.616450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.617074 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.619031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.628786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1050bedc7e43f29cfc5aab839e75cb', 'request_id': 'req-2fd9c71b-5590-44a8-9e59-a16397620abb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.635450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1050bedc7e43f29cfc5aab839e75cb', 'request_id': 'req-2fd9c71b-5590-44a8-9e59-a16397620abb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.638028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:35.643563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1050bedc7e43f29cfc5aab839e75cb', 'request_id': 'req-2fd9c71b-5590-44a8-9e59-a16397620abb'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.648523 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4204/8438] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:35.652478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.660123 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] The token cache is being invalidate because project 6d1050bedc7e43f29cfc5aab839e75cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:35.660414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.661074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2fd9c71b-5590-44a8-9e59-a16397620abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:35.662747 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4235/8439] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:35 2022] DELETE /identity/v3/projects/6d1050bedc7e43f29cfc5aab839e75cb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:35.668332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c2bc-a983-4689-8f59-c3f6d0c4a358 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.668616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c2bc-a983-4689-8f59-c3f6d0c4a358 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.668890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c2bc-a983-4689-8f59-c3f6d0c4a358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.688912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-65c1c2bc-a983-4689-8f59-c3f6d0c4a358 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:35.690272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65c1c2bc-a983-4689-8f59-c3f6d0c4a358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.721194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204569cd-3aa3-420b-9aa2-118ddb3ddf41 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.726396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.727198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.727680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.727961 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.728593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.730545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.752682 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4205/8440] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:35.757534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.759204 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.759520 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4236/8441] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:35.767103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a25c8-aa1d-4518-8767-8f74d0df3695 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.767395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a25c8-aa1d-4518-8767-8f74d0df3695 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.767676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a25c8-aa1d-4518-8767-8f74d0df3695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.771765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cde37-a44b-403e-b39a-c26fcec9617f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.778365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.778971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.779255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.779528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.780200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.781845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.787273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-232a25c8-aa1d-4518-8767-8f74d0df3695 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:35.788469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-232a25c8-aa1d-4518-8767-8f74d0df3695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:35.792551 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4206/8442] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:35.796599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.799706 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4237/8443] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:35.803650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a25c8-aa1d-4518-8767-8f74d0df3695 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.809930 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abf1e9-7db9-41f4-9cad-46ec0f3dd66d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.812973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a25c8-aa1d-4518-8767-8f74d0df3695 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.814552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.815703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.815703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.815703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] PATH_INFO: `/v3/projects/444328fa879648fd816de65bfcf9b5c2/users/f54c7b70e0b14e4ba09b8feef575ae87/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.817920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.818490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.818779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.819059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.819705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:35.820014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.822181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.831417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444328fa879648fd816de65bfcf9b5c2, user_id=f54c7b70e0b14e4ba09b8feef575ae87, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.834000 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4238/8444] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:25:35.837966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.839165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.849716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2012384-73c8-4288-8ba0-354473025423 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.855353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.855960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.856268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.856561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.857171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.858414 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4207/8445] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:35 2022] PUT /identity/v3/projects/444328fa879648fd816de65bfcf9b5c2/users/f54c7b70e0b14e4ba09b8feef575ae87/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:35.859130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.862532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.871701 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:35.890879 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4239/8446] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:35.895012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.932085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd1748-22e5-498d-adf2-3ddc8bbc5817 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.958489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f0cc9-d0ad-4d9f-9022-f819bc1dd8ff admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:35.963608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.964348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.964639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:35.964921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:35.965581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:35.967567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:35.997554 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4240/8447] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:35 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:35.998573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:35.998892 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:35.999549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:35.999945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.000235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.000877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.002587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.009100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d0bad-4058-4e5a-82d8-ea328fd610dd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:36.016090 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4208/8448] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:36.016224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:36.016224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.016224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.016224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.016593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.017114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:36.017731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.027217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084e41c-76b6-430f-85f0-3efbd9013fdb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:36.029272 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4241/8449] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:36.032073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c91904-e161-4de5-8a05-992742ac2b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:36.032980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c91904-e161-4de5-8a05-992742ac2b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.033149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:36.033274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c91904-e161-4de5-8a05-992742ac2b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.033560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c91904-e161-4de5-8a05-992742ac2b25 admin admin] PATH_INFO: `/v3/projects/444328fa879648fd816de65bfcf9b5c2/users/f54c7b70e0b14e4ba09b8feef575ae87/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.035520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c91904-e161-4de5-8a05-992742ac2b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444328fa879648fd816de65bfcf9b5c2, user_id=f54c7b70e0b14e4ba09b8feef575ae87, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.038232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c91904-e161-4de5-8a05-992742ac2b25 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.043688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29973f9f-66ff-42ff-ac6e-f487a2d29b32 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:36.049293 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4209/8450] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:36 2022] PUT /identity/v3/projects/444328fa879648fd816de65bfcf9b5c2/users/f54c7b70e0b14e4ba09b8feef575ae87/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:36.059801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.060093 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.060371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.073676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:36.090075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] MFA Rules not processed for user `f54c7b70e0b14e4ba09b8feef575ae87`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:36.091381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:36.112833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:36.113438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.113714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.114012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.114663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.116233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.127421 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4242/8451] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:36.131411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:36.145289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e48074-6db7-4ed1-b2fe-2185007b376c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:36.151395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9ac41-3598-4ec9-9e13-c7187036b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:36.152046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ac41-3598-4ec9-9e13-c7187036b242 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.152326 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ac41-3598-4ec9-9e13-c7187036b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.152601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ac41-3598-4ec9-9e13-c7187036b242 admin admin] PATH_INFO: `/v3/projects/1d4a1a5fdb2d409a8d3c1a9f0566e3ab/users/304301e69ec447a2a040c30f63633b68/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.165308 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4210/8452] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:36.168428 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9ac41-3598-4ec9-9e13-c7187036b242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4a1a5fdb2d409a8d3c1a9f0566e3ab, user_id=304301e69ec447a2a040c30f63633b68, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.171599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9ac41-3598-4ec9-9e13-c7187036b242 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.178170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:36.180153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:36.184978 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4243/8453] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:36 2022] PUT /identity/v3/projects/1d4a1a5fdb2d409a8d3c1a9f0566e3ab/users/304301e69ec447a2a040c30f63633b68/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:36.195353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.195696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.195972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.213326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:36.225653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] MFA Rules not processed for user `304301e69ec447a2a040c30f63633b68`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:36.227639 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:36.248425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f8ec5-36c4-413c-ab2d-87b6d6bf26bb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:36.293974 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4244/8454] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:36.298369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585522bd-56ee-4a3d-98fc-2c8172cbabea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:36.298980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585522bd-56ee-4a3d-98fc-2c8172cbabea service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.299288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585522bd-56ee-4a3d-98fc-2c8172cbabea service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.299515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585522bd-56ee-4a3d-98fc-2c8172cbabea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.304838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:36.306563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:36.320888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbdb87-767d-497e-b771-cbaa84db0fcd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:36.324810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:36.325404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.325688 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.326000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.334523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.335962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.345767 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4245/8455] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:36.362298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585522bd-56ee-4a3d-98fc-2c8172cbabea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.363749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585522bd-56ee-4a3d-98fc-2c8172cbabea service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.374146 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4211/8456] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:36.839983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:36.904095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cff3f7-7a3e-4cbc-95b3-9b83dc6d42a7 service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:36.908640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:36.909265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:36.909554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:36.909847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] PATH_INFO: `/v3/users/691d3a624ebc4a70ad48e90e44a4e326` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:36.910971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=691d3a624ebc4a70ad48e90e44a4e326)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:36.912890 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:36.929245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:36.959531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691d3a624ebc4a70ad48e90e44a4e326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:36.972417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691d3a624ebc4a70ad48e90e44a4e326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:36.981555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691d3a624ebc4a70ad48e90e44a4e326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:36.988999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691d3a624ebc4a70ad48e90e44a4e326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:36.989619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691d3a624ebc4a70ad48e90e44a4e326'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:36.995717 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4246/8457] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:36 2022] DELETE /identity/v3/users/691d3a624ebc4a70ad48e90e44a4e326 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:37.000361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:37.000805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:37.000914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:37.019902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:37.021095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:37.081567 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4212/8458] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:37.092455 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:37.093882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:37.154572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702e35e-0d39-44bb-a80c-db8cb865c819 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:37.158232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:37.158828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:37.159108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:37.159400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:37.169500 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:37.170975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:37.180735 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4247/8459] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:37.455343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:37.456795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:37.470298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc959a21-d442-4b99-9309-0778781f1df6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:37.475552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:37.477111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:37.477402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:37.477675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] PATH_INFO: `/v3/projects/fd75d638d6b1478f9fd7ba29199ab192` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:37.478858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd75d638d6b1478f9fd7ba29199ab192)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:37.480735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:37.506491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:37.532885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd75d638d6b1478f9fd7ba29199ab192', 'request_id': 'req-761fb643-02bd-4fd1-a261-2c35de47a17c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.539638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd75d638d6b1478f9fd7ba29199ab192', 'request_id': 'req-761fb643-02bd-4fd1-a261-2c35de47a17c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.548835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd75d638d6b1478f9fd7ba29199ab192', 'request_id': 'req-761fb643-02bd-4fd1-a261-2c35de47a17c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.560513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] The token cache is being invalidate because project fd75d638d6b1478f9fd7ba29199ab192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:37.560817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.561424 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.562919 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4213/8460] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:37 2022] DELETE /identity/v3/projects/fd75d638d6b1478f9fd7ba29199ab192 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:37.566871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:37.577582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7603-5bc9-4f3e-8fad-a77e73f3d1b0 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:37.635786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:37.636422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:37.636708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:37.636985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] PATH_INFO: `/v3/users/f54c7b70e0b14e4ba09b8feef575ae87` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:37.638164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54c7b70e0b14e4ba09b8feef575ae87)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:37.640077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:37.680419 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:37.688285 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54c7b70e0b14e4ba09b8feef575ae87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.702019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54c7b70e0b14e4ba09b8feef575ae87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.709239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54c7b70e0b14e4ba09b8feef575ae87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.718132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54c7b70e0b14e4ba09b8feef575ae87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.718930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54c7b70e0b14e4ba09b8feef575ae87'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:37.736709 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4248/8461] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:37 2022] DELETE /identity/v3/users/f54c7b70e0b14e4ba09b8feef575ae87 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:37.876687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:37.878155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:37.891571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fb643-02bd-4fd1-a261-2c35de47a17c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:37.954761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:37.955405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:37.955693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:37.955969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] PATH_INFO: `/v3/users/304301e69ec447a2a040c30f63633b68` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:37.957479 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=304301e69ec447a2a040c30f63633b68)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:37.959460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.021724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.023642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '304301e69ec447a2a040c30f63633b68'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.024606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.031801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '304301e69ec447a2a040c30f63633b68'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.032600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '304301e69ec447a2a040c30f63633b68'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.041718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc498a1-ec25-4c54-a805-360b088cf65d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.042840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '304301e69ec447a2a040c30f63633b68'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.043714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7044d53-8aa6-41ad-be88-12e68c5ae8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '304301e69ec447a2a040c30f63633b68'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.047157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.047833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.048121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.048399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] PATH_INFO: `/v3/projects/444328fa879648fd816de65bfcf9b5c2` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.049541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444328fa879648fd816de65bfcf9b5c2)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.051507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.066242 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4214/8462] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:37 2022] DELETE /identity/v3/users/304301e69ec447a2a040c30f63633b68 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:38.070825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.071402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.071896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.092484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:38.093872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.108693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444328fa879648fd816de65bfcf9b5c2', 'request_id': 'req-373e24d7-4ec4-43ba-ab8e-2032752f77e8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.117036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444328fa879648fd816de65bfcf9b5c2', 'request_id': 'req-373e24d7-4ec4-43ba-ab8e-2032752f77e8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.123092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:38.125364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444328fa879648fd816de65bfcf9b5c2', 'request_id': 'req-373e24d7-4ec4-43ba-ab8e-2032752f77e8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.138746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] The token cache is being invalidate because project 444328fa879648fd816de65bfcf9b5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:38.139121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.143099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-373e24d7-4ec4-43ba-ab8e-2032752f77e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.145108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4249/8463] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:38 2022] DELETE /identity/v3/projects/444328fa879648fd816de65bfcf9b5c2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:38.149978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64de5e-b2fd-4c17-8828-5545015d2599 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.150270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64de5e-b2fd-4c17-8828-5545015d2599 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.150547 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64de5e-b2fd-4c17-8828-5545015d2599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.166763 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4215/8464] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:38.171740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2c64de5e-b2fd-4c17-8828-5545015d2599 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:38.172951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2c64de5e-b2fd-4c17-8828-5545015d2599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.176783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.178200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.182153 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4250/8465] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:38.190054 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0add44-c96e-4845-8fa1-fb37125c5255 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.190345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0add44-c96e-4845-8fa1-fb37125c5255 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.190626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0add44-c96e-4845-8fa1-fb37125c5255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.210324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-af0add44-c96e-4845-8fa1-fb37125c5255 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:38.211545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-af0add44-c96e-4845-8fa1-fb37125c5255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.219965 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4251/8466] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:38.224214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0add44-c96e-4845-8fa1-fb37125c5255 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.234218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0add44-c96e-4845-8fa1-fb37125c5255 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.239089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.239676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.239952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.240220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.240848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:38.241147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.243376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.246601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2ab8d-2985-4e5e-b006-2672f81c6f67 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.250339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.250947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.251229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.251506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.258659 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4252/8467] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:25:38.259609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.261326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.262812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.269014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65415e4-d21b-4348-bb29-694e8eec4ab9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.272707 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4216/8468] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:38.273643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.274434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.274718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.274990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.275606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.277659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.306132 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4253/8469] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:38.310102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.320543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881355d-5bb5-4df0-97e8-2a9d49aecadb service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.381673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.382640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.383492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.383879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.384535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.386703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.412730 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4217/8470] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:38.414169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.426289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8fdf2-c376-42fd-8fc3-edbda407b04f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.431767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.432426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.432708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.432987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.433767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.435432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.447127 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4254/8471] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:38.451029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.462165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38a84f-e1ac-4118-8fb7-bccdf2a3392a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.467081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc2c26-25e8-4201-ad00-6f488a23609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.467695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc2c26-25e8-4201-ad00-6f488a23609e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.468054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc2c26-25e8-4201-ad00-6f488a23609e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.468249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc2c26-25e8-4201-ad00-6f488a23609e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.468890 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc2c26-25e8-4201-ad00-6f488a23609e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.470571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc2c26-25e8-4201-ad00-6f488a23609e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.481405 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4218/8472] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:38.485865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.497423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b5a53-3dfd-47bd-89f4-14b3596a8da9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.502443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.503098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.503381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.503686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] PATH_INFO: `/v3/projects/d22315c1330d4b5ea8f8f5ccb35a76fb/users/9f2fee40dd6240aca0ebbf08df1b2009/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.519864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22315c1330d4b5ea8f8f5ccb35a76fb, user_id=9f2fee40dd6240aca0ebbf08df1b2009, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.522674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.536241 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4255/8473] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:38 2022] PUT /identity/v3/projects/d22315c1330d4b5ea8f8f5ccb35a76fb/users/9f2fee40dd6240aca0ebbf08df1b2009/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:38.548450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf443-a80c-486c-882c-f30a86457ce4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.548936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf443-a80c-486c-882c-f30a86457ce4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.549284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf443-a80c-486c-882c-f30a86457ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.570315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.572845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.595676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:38.603817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-9a2cf443-a80c-486c-882c-f30a86457ce4 None None] MFA Rules not processed for user `9f2fee40dd6240aca0ebbf08df1b2009`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:38.605483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9a2cf443-a80c-486c-882c-f30a86457ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.647549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce616c-ca7f-433c-bc9e-f79d88d35021 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.648128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:38.653581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.654476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.654758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.655032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] PATH_INFO: `/v3/projects/1d4a1a5fdb2d409a8d3c1a9f0566e3ab` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.656183 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4a1a5fdb2d409a8d3c1a9f0566e3ab)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.658112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.674204 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4219/8474] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:38.678808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cf443-a80c-486c-882c-f30a86457ce4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.692642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cf443-a80c-486c-882c-f30a86457ce4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.707947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4a1a5fdb2d409a8d3c1a9f0566e3ab', 'request_id': 'req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.714426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4a1a5fdb2d409a8d3c1a9f0566e3ab', 'request_id': 'req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.721062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4a1a5fdb2d409a8d3c1a9f0566e3ab', 'request_id': 'req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.734604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] The token cache is being invalidate because project 1d4a1a5fdb2d409a8d3c1a9f0566e3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:38.734902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.735803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4fef1e55-20c9-4a7f-a6ba-147a205ca8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:38.737406 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4256/8475] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:38 2022] DELETE /identity/v3/projects/1d4a1a5fdb2d409a8d3c1a9f0566e3ab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:38.746226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18303f6-56f7-43f7-a7d9-babf8d982506 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.746533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18303f6-56f7-43f7-a7d9-babf8d982506 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.746811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18303f6-56f7-43f7-a7d9-babf8d982506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.752799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.753414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.753700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.754020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.754651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.756598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.767435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c18303f6-56f7-43f7-a7d9-babf8d982506 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:38.768734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c18303f6-56f7-43f7-a7d9-babf8d982506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.778741 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4257/8476] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:38.786691 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4220/8477] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:38.788089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96a22-8595-42a4-97c7-49e27b1f05a5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.788378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96a22-8595-42a4-97c7-49e27b1f05a5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.788658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96a22-8595-42a4-97c7-49e27b1f05a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.789096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.791319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.809914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-37d96a22-8595-42a4-97c7-49e27b1f05a5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:38.811058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-37d96a22-8595-42a4-97c7-49e27b1f05a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:38.862889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf14ea-1da3-4914-b160-0d26613caff2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.868087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.868839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.869322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.869440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.870070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.871989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.877914 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4258/8478] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:38.883495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d96a22-8595-42a4-97c7-49e27b1f05a5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.893613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d96a22-8595-42a4-97c7-49e27b1f05a5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.894102 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4221/8479] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:38.897694 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.902148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.902768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.903046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.903324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.903968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:38.904275 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.905894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.908395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbcedf-2e5f-41ca-8b49-097119ce19d2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.913047 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.913653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.913968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.914534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.915255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.916864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.921863 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4259/8480] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:25:38.923358 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.931249 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4222/8481] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:38.932235 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.933943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff819f-1363-426f-9d4a-3e1161087637 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.939221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.940368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.940661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.940938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.941591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.942050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c2865-fcb3-4b2a-a546-0c51dc543b10 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:38.943609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.947646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:38.948313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:38.948597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:38.948873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] PATH_INFO: `/v3/projects/b9834101fceb4d53acbdbe2e5149d3ed/users/e30ab08ec88d4e8e991693d668445837/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:38.965552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9834101fceb4d53acbdbe2e5149d3ed, user_id=e30ab08ec88d4e8e991693d668445837, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:38.969182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:38.974111 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4260/8482] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:38.979481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:38.984368 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4223/8483] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:38 2022] PUT /identity/v3/projects/b9834101fceb4d53acbdbe2e5149d3ed/users/e30ab08ec88d4e8e991693d668445837/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:38.985671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:38.988417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.014665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:39.047504 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063fb01-ac43-4c32-abbf-6c26f3d21db4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.052231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.052833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.053119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.053396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.053529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9eed5-fe09-4680-954a-1edba0ef380d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.054079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.056085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.058912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.059520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.059805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.060135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.060726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.062363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.074511 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4224/8484] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:39.078588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.079250 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4261/8485] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:38 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:39.082939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.089377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8b573-fadb-4624-8489-dda34a91d83e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.094540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87bcf0-45a8-4b44-91e2-8515d74da88b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.094690 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283ab58-e122-456c-b168-8cfb1bd4a156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.095270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283ab58-e122-456c-b168-8cfb1bd4a156 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.095556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283ab58-e122-456c-b168-8cfb1bd4a156 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.095836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283ab58-e122-456c-b168-8cfb1bd4a156 admin admin] PATH_INFO: `/v3/projects/b9834101fceb4d53acbdbe2e5149d3ed/users/e30ab08ec88d4e8e991693d668445837/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.097946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283ab58-e122-456c-b168-8cfb1bd4a156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9834101fceb4d53acbdbe2e5149d3ed, user_id=e30ab08ec88d4e8e991693d668445837, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.099605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.100308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.100490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.100745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.101169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283ab58-e122-456c-b168-8cfb1bd4a156 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.101493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.103006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.112388 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4225/8486] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:39 2022] PUT /identity/v3/projects/b9834101fceb4d53acbdbe2e5149d3ed/users/e30ab08ec88d4e8e991693d668445837/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:39.119921 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4262/8487] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:39.123550 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.123840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.123943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.124118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.135650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe9c94-1ad1-4d56-b0ff-c86f81bcd580 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.152337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] MFA Rules not processed for user `e30ab08ec88d4e8e991693d668445837`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:39.153541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:39.195085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.196433 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.196743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.197025 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.197798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.199486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.209805 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4263/8488] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:39.214089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.224050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d380a9-3726-4076-af2f-074dc8803ae9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.224500 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4226/8489] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:39.229795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbcc7f-c8e9-482c-a74f-d5f592c32a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.230190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbcc7f-c8e9-482c-a74f-d5f592c32a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.230469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbcc7f-c8e9-482c-a74f-d5f592c32a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.230767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbcc7f-c8e9-482c-a74f-d5f592c32a79 admin admin] PATH_INFO: `/v3/projects/92f59058d6274eb7b3472a96ab56b55c/users/87cb7cceef924db79e4add131992b8e1/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.235062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.236770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:39.247786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbcc7f-c8e9-482c-a74f-d5f592c32a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f59058d6274eb7b3472a96ab56b55c, user_id=87cb7cceef924db79e4add131992b8e1, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.250513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbcc7f-c8e9-482c-a74f-d5f592c32a79 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.265226 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4264/8490] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:39 2022] PUT /identity/v3/projects/92f59058d6274eb7b3472a96ab56b55c/users/87cb7cceef924db79e4add131992b8e1/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:39.270178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.270469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.270748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.289507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:39.290881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:39.298834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6bd4-c1a4-4778-90a8-69a7aa505e9e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.340233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:39.345289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:39.351732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.352600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.352916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.353194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.356373 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4265/8491] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:39.363341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.364885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:39.415862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.417320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.426575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529009e-e960-4982-8ca5-ea987e18bb8f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.427926 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4227/8492] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:39.430960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.431562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.431841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.432113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.441919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.443536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.454575 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4266/8493] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:39.538158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:39.551352 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8e7cb-d488-4517-bcbd-09cc1b3140cb service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:39.556477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca48c7-bb6a-4321-b887-13605c3b98ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:39.557166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca48c7-bb6a-4321-b887-13605c3b98ca service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:39.557667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca48c7-bb6a-4321-b887-13605c3b98ca service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:39.558039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca48c7-bb6a-4321-b887-13605c3b98ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:39.559978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d5ca48c7-bb6a-4321-b887-13605c3b98ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:39.638832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca48c7-bb6a-4321-b887-13605c3b98ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:39.640326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca48c7-bb6a-4321-b887-13605c3b98ca service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:39.650169 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4228/8494] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:41.246366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:41.257010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09f00e-34cf-45d5-9fa2-6e47b40d6bdd service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:41.262774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:41.263397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:41.263677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:41.263948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] PATH_INFO: `/v3/users/9f2fee40dd6240aca0ebbf08df1b2009` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:41.265065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f2fee40dd6240aca0ebbf08df1b2009)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:41.267007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:41.324047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f2fee40dd6240aca0ebbf08df1b2009'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:41.337349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f2fee40dd6240aca0ebbf08df1b2009'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:41.344424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f2fee40dd6240aca0ebbf08df1b2009'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:41.353038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f2fee40dd6240aca0ebbf08df1b2009'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:41.353845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f2fee40dd6240aca0ebbf08df1b2009'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:41.362521 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4267/8495] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:41 2022] DELETE /identity/v3/users/9f2fee40dd6240aca0ebbf08df1b2009 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:41.367052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:41.368079 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:41.368170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:41.388512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:41.389730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:41.452459 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4229/8496] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:41.468750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:41.470325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:41.541338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d0a06-9a69-46d5-ac5f-04def4eb196d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:41.545859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:41.547956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:41.548098 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:41.548377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:41.560108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:41.561567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:41.574146 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4268/8497] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:42.105367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:42.106917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:42.122467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2c31e-4ddc-4336-882b-447726e6fa21 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:42.127386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:42.128336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:42.128638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:42.128915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] PATH_INFO: `/v3/projects/d22315c1330d4b5ea8f8f5ccb35a76fb` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:42.130134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22315c1330d4b5ea8f8f5ccb35a76fb)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:42.132350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:42.190031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:42.209069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22315c1330d4b5ea8f8f5ccb35a76fb', 'request_id': 'req-084b47d7-a1c5-4964-9896-163c86649f0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.220049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22315c1330d4b5ea8f8f5ccb35a76fb', 'request_id': 'req-084b47d7-a1c5-4964-9896-163c86649f0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.231714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22315c1330d4b5ea8f8f5ccb35a76fb', 'request_id': 'req-084b47d7-a1c5-4964-9896-163c86649f0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.248963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] The token cache is being invalidate because project d22315c1330d4b5ea8f8f5ccb35a76fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:42.249355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.251782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.254367 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4230/8498] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:42 2022] DELETE /identity/v3/projects/d22315c1330d4b5ea8f8f5ccb35a76fb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:42.258552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:42.272270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e4d24-19c4-4a40-b342-296d20c8f06e service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:42.286023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:42.353641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:42.354481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:42.354758 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:42.355036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] PATH_INFO: `/v3/users/e30ab08ec88d4e8e991693d668445837` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:42.356256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30ab08ec88d4e8e991693d668445837)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:42.359094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:42.450644 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30ab08ec88d4e8e991693d668445837'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.495679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30ab08ec88d4e8e991693d668445837'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.517978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30ab08ec88d4e8e991693d668445837'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.535516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30ab08ec88d4e8e991693d668445837'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.536492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-52e2ff13-1710-488e-80a0-ed32ec0ea3dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30ab08ec88d4e8e991693d668445837'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:42.548059 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4269/8499] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:42 2022] DELETE /identity/v3/users/e30ab08ec88d4e8e991693d668445837 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:43.120126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:43.121902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:43.138947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b47d7-a1c5-4964-9896-163c86649f0f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:43.230702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:43.231264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.231538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.231845 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] PATH_INFO: `/v3/projects/b9834101fceb4d53acbdbe2e5149d3ed` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.233015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9834101fceb4d53acbdbe2e5149d3ed)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:43.234978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:43.308675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9834101fceb4d53acbdbe2e5149d3ed', 'request_id': 'req-8fcfbfba-6552-4139-aa65-73a724a3ec0e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:43.319180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9834101fceb4d53acbdbe2e5149d3ed', 'request_id': 'req-8fcfbfba-6552-4139-aa65-73a724a3ec0e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:43.330911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9834101fceb4d53acbdbe2e5149d3ed', 'request_id': 'req-8fcfbfba-6552-4139-aa65-73a724a3ec0e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:43.348170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] The token cache is being invalidate because project b9834101fceb4d53acbdbe2e5149d3ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:43.348556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:43.349346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8fcfbfba-6552-4139-aa65-73a724a3ec0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:43.351288 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4231/8500] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:43 2022] DELETE /identity/v3/projects/b9834101fceb4d53acbdbe2e5149d3ed => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:43.356794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a8ff-6d22-40e3-8d4e-1b173cbcc48a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.357320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a8ff-6d22-40e3-8d4e-1b173cbcc48a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.357425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13a8ff-6d22-40e3-8d4e-1b173cbcc48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.382125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-fd13a8ff-6d22-40e3-8d4e-1b173cbcc48a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:43.384854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fd13a8ff-6d22-40e3-8d4e-1b173cbcc48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:43.495083 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4270/8501] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:43.497112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9c08-1e45-4551-ab0c-7d1a8b9fde07 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.497405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9c08-1e45-4551-ab0c-7d1a8b9fde07 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.497688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9c08-1e45-4551-ab0c-7d1a8b9fde07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.533583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f03a9c08-1e45-4551-ab0c-7d1a8b9fde07 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:43.534473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f03a9c08-1e45-4551-ab0c-7d1a8b9fde07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:43.544221 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4232/8502] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:43.549712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13a8ff-6d22-40e3-8d4e-1b173cbcc48a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:43.560542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13a8ff-6d22-40e3-8d4e-1b173cbcc48a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:43.566647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:43.567797 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.568266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.568722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.570884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:43.571410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:43.573713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:43.593870 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4271/8503] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:25:43.597340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a9c08-1e45-4551-ab0c-7d1a8b9fde07 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:43.607419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a9c08-1e45-4551-ab0c-7d1a8b9fde07 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:43.613253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:43.614162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.614601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.615050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.615935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:43.618782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:43.661402 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4233/8504] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:43 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:43.662478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:43.673133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718208f-a87c-415d-8ba9-60df9214f938 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:43.694672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:43.750707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:43.750892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.751159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.752124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.752968 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:43.754991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:43.794470 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4272/8505] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:43 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:43.800158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:43.812070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3289f29-e480-4e0c-9913-1315642e46a0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:43.817231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:43.818071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.818501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.818894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.819935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:43.822040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:43.839013 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4234/8506] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:43.840336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:43.851391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0030cd2-c36d-483a-84f8-1be09dead497 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:43.857010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be72bec-f8ae-41b0-b939-2a5ada638b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:43.857917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72bec-f8ae-41b0-b939-2a5ada638b62 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.858381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72bec-f8ae-41b0-b939-2a5ada638b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.858658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72bec-f8ae-41b0-b939-2a5ada638b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.859311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be72bec-f8ae-41b0-b939-2a5ada638b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:43.861219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be72bec-f8ae-41b0-b939-2a5ada638b62 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:43.876022 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4273/8507] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:43.877584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:43.889880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725b6dd-bacc-4f6e-9a24-64627ce53b68 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:43.896051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:43.896759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.899130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.899420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] PATH_INFO: `/v3/projects/4726b5f2761c4c5a92fcd73f88ca5bda/users/931dbbf8d4c84dc4a3d41d2182b3aa40/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.903179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:43.920221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4726b5f2761c4c5a92fcd73f88ca5bda, user_id=931dbbf8d4c84dc4a3d41d2182b3aa40, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:43.923185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:43.941752 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4235/8508] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:43 2022] PUT /identity/v3/projects/4726b5f2761c4c5a92fcd73f88ca5bda/users/931dbbf8d4c84dc4a3d41d2182b3aa40/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:43.952567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619709f8-ee9a-4b27-bdd8-f856ac0801cd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:43.952939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619709f8-ee9a-4b27-bdd8-f856ac0801cd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:43.953420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619709f8-ee9a-4b27-bdd8-f856ac0801cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:43.996668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-619709f8-ee9a-4b27-bdd8-f856ac0801cd None None] MFA Rules not processed for user `931dbbf8d4c84dc4a3d41d2182b3aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:44.000016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-619709f8-ee9a-4b27-bdd8-f856ac0801cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:44.111546 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4274/8509] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:44.123536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.125904 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:44.219011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4de2b1-b744-4510-a483-a862d4f90adf admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.223141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.224228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.224626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.225641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.237629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.239708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.257178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4236/8510] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:44.259935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.260232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.260534 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.279668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.301541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] MFA Rules not processed for user `87cb7cceef924db79e4add131992b8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:44.302994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:44.364974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd089b0-fc40-4165-adf2-7462c59104c8 service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.370419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.371250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.371524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.371800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] PATH_INFO: `/v3/users/931dbbf8d4c84dc4a3d41d2182b3aa40` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.372957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=931dbbf8d4c84dc4a3d41d2182b3aa40)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.374956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.384923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4275/8511] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:44.389662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.392038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:44.407370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101e851-d641-4661-8b9a-2ee505ac59dd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.416136 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.416759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.417047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.417323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.418007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.422447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.447856 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '931dbbf8d4c84dc4a3d41d2182b3aa40'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:44.448499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:44.457881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '931dbbf8d4c84dc4a3d41d2182b3aa40'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:44.458583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '931dbbf8d4c84dc4a3d41d2182b3aa40'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:44.475336 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4276/8512] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:44 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:44.477144 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '931dbbf8d4c84dc4a3d41d2182b3aa40'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:44.479586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-2834949f-882c-4e09-b467-86bed5105951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '931dbbf8d4c84dc4a3d41d2182b3aa40'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:44.487037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.490108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:44.502352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4237/8513] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:44 2022] DELETE /identity/v3/users/931dbbf8d4c84dc4a3d41d2182b3aa40 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:44.506854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.507148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.507486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.525075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:44.534458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:44.536084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:44.582335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee3457-adc2-4f2f-a15e-988c78a46c4a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.635246 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4238/8514] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:44.645439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.647816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:44.674514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.675811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.676308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.676790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.677759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.680655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.711639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4277/8515] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:44 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:44.721716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.723244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30979697-d1d3-4dfe-9c74-a4168f01bef6 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.735404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfbf82-c64e-4710-9373-b2a64f928661 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.741227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.742156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.742626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.743068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.743738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.746033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.746886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.747713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.751440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.751801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.766684 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4278/8516] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:44.769607 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.772064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.779665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.798840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3423637-ea95-4125-9172-24a48cdb8d18 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.805955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.806638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.807081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.807372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] PATH_INFO: `/v3/projects/1830b4f217224825a86fab1b8103117b/users/a2b935bfd8644e879de4a1d33b12e2b7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.819026 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4239/8517] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:44.826494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1830b4f217224825a86fab1b8103117b, user_id=a2b935bfd8644e879de4a1d33b12e2b7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.830659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.852109 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4279/8518] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:44 2022] PUT /identity/v3/projects/1830b4f217224825a86fab1b8103117b/users/a2b935bfd8644e879de4a1d33b12e2b7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:25:44.856389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.885635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57374a-5b85-4385-a8a8-63df827c4c6c service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:44.973755 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:44.974559 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:44.975185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:44.976092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:44.977389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:44.979269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:44.998472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:44.998595 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4240/8519] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:45.017468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3400fc2-8d5a-4cd8-8723-d5c825c8e424 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.024207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6eba2-e126-4658-ad9e-c1d6e8c82007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.024896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6eba2-e126-4658-ad9e-c1d6e8c82007 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.025185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6eba2-e126-4658-ad9e-c1d6e8c82007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.025462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6eba2-e126-4658-ad9e-c1d6e8c82007 admin admin] PATH_INFO: `/v3/projects/1830b4f217224825a86fab1b8103117b/users/a2b935bfd8644e879de4a1d33b12e2b7/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.027686 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6eba2-e126-4658-ad9e-c1d6e8c82007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1830b4f217224825a86fab1b8103117b, user_id=a2b935bfd8644e879de4a1d33b12e2b7, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.030434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6eba2-e126-4658-ad9e-c1d6e8c82007 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.042142 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4280/8520] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:44 2022] PUT /identity/v3/projects/1830b4f217224825a86fab1b8103117b/users/a2b935bfd8644e879de4a1d33b12e2b7/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:45.142773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:45.144938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:45.161343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f8c47-a3a1-4002-8037-9b111b9e32a0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.215884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:45.229874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.230741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.231200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.231646 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] PATH_INFO: `/v3/projects/4726b5f2761c4c5a92fcd73f88ca5bda` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.232830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4726b5f2761c4c5a92fcd73f88ca5bda)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.235404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.292220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4726b5f2761c4c5a92fcd73f88ca5bda', 'request_id': 'req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:45.300190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4726b5f2761c4c5a92fcd73f88ca5bda', 'request_id': 'req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:45.310125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4726b5f2761c4c5a92fcd73f88ca5bda', 'request_id': 'req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:45.323435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] The token cache is being invalidate because project 4726b5f2761c4c5a92fcd73f88ca5bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:45.323740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:45.324410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fec79ba1-4c8a-4a8a-b95e-dc9e606c9d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:45.326052 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4241/8521] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:45 2022] DELETE /identity/v3/projects/4726b5f2761c4c5a92fcd73f88ca5bda => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:45.330842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014489a-042c-439c-b5cd-9f0ffc309308 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.331135 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014489a-042c-439c-b5cd-9f0ffc309308 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.331411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014489a-042c-439c-b5cd-9f0ffc309308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.355687 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d014489a-042c-439c-b5cd-9f0ffc309308 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:45.357138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d014489a-042c-439c-b5cd-9f0ffc309308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:45.411631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:45.436293 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4281/8522] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:45.444046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383b05-751a-4bcc-aa2d-0c1c8f51e320 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.444347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383b05-751a-4bcc-aa2d-0c1c8f51e320 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.444627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383b05-751a-4bcc-aa2d-0c1c8f51e320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.467422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-08383b05-751a-4bcc-aa2d-0c1c8f51e320 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:45.468686 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08383b05-751a-4bcc-aa2d-0c1c8f51e320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:45.482125 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4242/8523] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:45.483628 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014489a-042c-439c-b5cd-9f0ffc309308 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:45.497851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014489a-042c-439c-b5cd-9f0ffc309308 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.503287 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.504046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.504363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.504611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.505262 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:45.505566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.507272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.520699 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4282/8524] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:25:45.525073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383b05-751a-4bcc-aa2d-0c1c8f51e320 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:45.535017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383b05-751a-4bcc-aa2d-0c1c8f51e320 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.541598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.542173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.542443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.542759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.543421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.545451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.584015 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4243/8525] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:45 2022] POST /identity/v3/projects => generated 359 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:45.585582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:45.597755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92ebdd-d7ae-423f-847c-3fd484e869f5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.665699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.666876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.667594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.667926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.668910 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.670874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.706665 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4283/8526] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:45 2022] POST /identity/v3/users => generated 280 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:45.710754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:45.723713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed357c-ea53-481a-bfb2-9b5bfc62e2be admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.728827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.729463 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.729752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.730096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.730783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.732551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.746743 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4244/8527] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:25:45.763512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:45.779095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c419b1-d923-4976-a074-c5069961b6f8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.787795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c978b50-1d18-4c9a-98ee-0e81dda52134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.788424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c978b50-1d18-4c9a-98ee-0e81dda52134 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.788705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c978b50-1d18-4c9a-98ee-0e81dda52134 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.789009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c978b50-1d18-4c9a-98ee-0e81dda52134 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.790111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c978b50-1d18-4c9a-98ee-0e81dda52134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.791750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c978b50-1d18-4c9a-98ee-0e81dda52134 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.809037 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4284/8528] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:45.813675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:45.829340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448276-e2ce-4a35-b1db-af10152d33bd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:45.843422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:45.844106 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.844556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.844999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] PATH_INFO: `/v3/projects/e09324cae13e4485ba19cf734ea3fcc8/users/deacf0b782f44a87b2e52c95e4a8998f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.874327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09324cae13e4485ba19cf734ea3fcc8, user_id=deacf0b782f44a87b2e52c95e4a8998f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:45.877000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:45.905529 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4245/8529] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:45 2022] PUT /identity/v3/projects/e09324cae13e4485ba19cf734ea3fcc8/users/deacf0b782f44a87b2e52c95e4a8998f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:45.911192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561bf8-b9d2-4172-940f-2c512b3a07ec None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:45.915005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561bf8-b9d2-4172-940f-2c512b3a07ec None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:45.916013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f561bf8-b9d2-4172-940f-2c512b3a07ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:45.957395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4f561bf8-b9d2-4172-940f-2c512b3a07ec None None] MFA Rules not processed for user `deacf0b782f44a87b2e52c95e4a8998f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:45.959362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4f561bf8-b9d2-4172-940f-2c512b3a07ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:46.070382 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4285/8530] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:46.110509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:46.113290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:46.189207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5283a-f6d2-44fd-86e3-9bd815ecbd04 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:46.194515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05791f4f-ec4d-4f8a-a7f8-065944123e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:46.196519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05791f4f-ec4d-4f8a-a7f8-065944123e24 service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:46.197193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05791f4f-ec4d-4f8a-a7f8-065944123e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:46.197693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05791f4f-ec4d-4f8a-a7f8-065944123e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:46.215280 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05791f4f-ec4d-4f8a-a7f8-065944123e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:46.218645 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05791f4f-ec4d-4f8a-a7f8-065944123e24 service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:46.232224 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4246/8531] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:46.248887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f561bf8-b9d2-4172-940f-2c512b3a07ec None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:46.307999 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:46.355406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f561bf8-b9d2-4172-940f-2c512b3a07ec None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:46.362472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:46.363460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:46.363946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:46.364463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] PATH_INFO: `/v3/users/deacf0b782f44a87b2e52c95e4a8998f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:46.365993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deacf0b782f44a87b2e52c95e4a8998f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:46.372188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:46.465368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deacf0b782f44a87b2e52c95e4a8998f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:46.498517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deacf0b782f44a87b2e52c95e4a8998f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:46.511946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deacf0b782f44a87b2e52c95e4a8998f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:46.528007 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deacf0b782f44a87b2e52c95e4a8998f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:46.529783 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deacf0b782f44a87b2e52c95e4a8998f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:46.544715 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4286/8532] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:46 2022] DELETE /identity/v3/users/deacf0b782f44a87b2e52c95e4a8998f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:46.545820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:46.546153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:46.546436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:46.573242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:46.574942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:46.587277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:46.674281 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4247/8533] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:46.680677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:46.682219 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:46.778030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38093-04b4-4cd9-8836-54291dd0c1a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:46.787023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7ec46-aa09-452c-83c3-fc8feaa61d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:46.787938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7ec46-aa09-452c-83c3-fc8feaa61d20 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:46.788256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7ec46-aa09-452c-83c3-fc8feaa61d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:46.789389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7ec46-aa09-452c-83c3-fc8feaa61d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:46.814227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7ec46-aa09-452c-83c3-fc8feaa61d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:46.818393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7ec46-aa09-452c-83c3-fc8feaa61d20 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:46.832700 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4287/8534] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:25:47.423899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:47.427663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:47.443756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793ff5f-e9c9-4c67-bc0b-40eaed2ba7fc None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:47.452661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:47.453553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:47.453941 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:47.454289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] PATH_INFO: `/v3/projects/e09324cae13e4485ba19cf734ea3fcc8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:47.455520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09324cae13e4485ba19cf734ea3fcc8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:47.458504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:47.514773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09324cae13e4485ba19cf734ea3fcc8', 'request_id': 'req-9353472c-931c-479c-836c-6e34e0dac681'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:47.521859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09324cae13e4485ba19cf734ea3fcc8', 'request_id': 'req-9353472c-931c-479c-836c-6e34e0dac681'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:47.532672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09324cae13e4485ba19cf734ea3fcc8', 'request_id': 'req-9353472c-931c-479c-836c-6e34e0dac681'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:47.550978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] The token cache is being invalidate because project e09324cae13e4485ba19cf734ea3fcc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:47.551595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:47.552505 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-9353472c-931c-479c-836c-6e34e0dac681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:47.554237 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4248/8535] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:47 2022] DELETE /identity/v3/projects/e09324cae13e4485ba19cf734ea3fcc8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:47.559818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce679bf0-22ad-4db8-8df1-449ea48254af None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:47.560536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce679bf0-22ad-4db8-8df1-449ea48254af None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:47.560919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce679bf0-22ad-4db8-8df1-449ea48254af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:47.621395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ce679bf0-22ad-4db8-8df1-449ea48254af None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:47.622708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ce679bf0-22ad-4db8-8df1-449ea48254af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:47.706259 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4288/8536] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:47.714829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08793ff2-9167-4d19-baab-d725925a014c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:47.715270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08793ff2-9167-4d19-baab-d725925a014c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:47.715618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08793ff2-9167-4d19-baab-d725925a014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:47.739238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-08793ff2-9167-4d19-baab-d725925a014c None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:47.740618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08793ff2-9167-4d19-baab-d725925a014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:47.750945 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4249/8537] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:47.754908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce679bf0-22ad-4db8-8df1-449ea48254af None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:47.761000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:47.771554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce679bf0-22ad-4db8-8df1-449ea48254af None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:47.780177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:47.780671 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:47.780958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:47.781234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:47.781925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:25:47.782239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:47.784286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:47.810289 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4289/8538] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:25:47.811453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08793ff2-9167-4d19-baab-d725925a014c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:47.825990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08793ff2-9167-4d19-baab-d725925a014c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:47.831790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:47.832606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:47.832900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:47.833317 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:47.834022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:47.838020 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:47.890877 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4250/8539] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:25:47 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:25:47.894943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:47.909955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121c553-a699-4ed5-9942-b2ae3c9e1461 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:48.039725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:48.040346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:48.040809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:48.041092 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:48.041729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:48.044167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:48.080546 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4290/8540] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:25:47 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:25:48.082112 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:48.096171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322bfa2-9575-4a2f-be21-bd2d863fe1e1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:48.111322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:48.112202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:48.112749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:48.113213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:48.114016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:48.115782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:48.146083 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4251/8541] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:48.151203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:48.167929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8540b-69a1-4473-87be-429a9d822dbb admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:48.175246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc0ca5-bf2b-4c9e-848c-404b436788cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:48.176347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc0ca5-bf2b-4c9e-848c-404b436788cd admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:48.176347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc0ca5-bf2b-4c9e-848c-404b436788cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:48.176568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc0ca5-bf2b-4c9e-848c-404b436788cd admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:48.177613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc0ca5-bf2b-4c9e-848c-404b436788cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:48.179601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc0ca5-bf2b-4c9e-848c-404b436788cd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:48.204376 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4291/8542] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:25:48.209657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:48.221603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf1d28-c4f4-4026-8314-3cc2ae1c9a3e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:48.238765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:48.239474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:48.241511 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:48.241809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] PATH_INFO: `/v3/projects/7693ab8a9ce24ff898409095f48622bd/users/506fc30e70e44452a648171d10ce5b3d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:48.259929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7693ab8a9ce24ff898409095f48622bd, user_id=506fc30e70e44452a648171d10ce5b3d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:48.262737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:48.284822 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4252/8543] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:25:48 2022] PUT /identity/v3/projects/7693ab8a9ce24ff898409095f48622bd/users/506fc30e70e44452a648171d10ce5b3d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:48.290736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e9b2-a003-45c9-a6a3-b44d0344a872 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:48.291038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e9b2-a003-45c9-a6a3-b44d0344a872 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:48.291320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e9b2-a003-45c9-a6a3-b44d0344a872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:48.317699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a163e9b2-a003-45c9-a6a3-b44d0344a872 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:48.319708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a163e9b2-a003-45c9-a6a3-b44d0344a872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:48.394119 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4292/8544] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:25:48.404840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:48.406618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:48.471633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:48.505133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90547e-d795-4641-b9b8-733c32cd529d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:48.517968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:48.518496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:48.518781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:48.519058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:48.532685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:48.534975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:48.550886 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4253/8545] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:49.641632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6799c-9427-4689-b6fd-07a0ffc11906 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:49.642159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6799c-9427-4689-b6fd-07a0ffc11906 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:49.642239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6799c-9427-4689-b6fd-07a0ffc11906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:49.675001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-75e6799c-9427-4689-b6fd-07a0ffc11906 None None] MFA Rules not processed for user `506fc30e70e44452a648171d10ce5b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:49.676337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-75e6799c-9427-4689-b6fd-07a0ffc11906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:49.772893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:49.779458 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4293/8546] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:49.795069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:49.797276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:49.886910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729c86e-8e94-4ea0-b772-6da3ff86a06d service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:49.891678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:49.892310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:49.892594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:49.892871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:49.903276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:49.904984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:49.925970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4254/8547] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:49.948867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb0c3-7d90-4e86-8717-d5881b3c19ea None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:49.949163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb0c3-7d90-4e86-8717-d5881b3c19ea None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:49.949443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb0c3-7d90-4e86-8717-d5881b3c19ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:49.998094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c3abb0c3-7d90-4e86-8717-d5881b3c19ea None None] MFA Rules not processed for user `a2b935bfd8644e879de4a1d33b12e2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:49.999568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c3abb0c3-7d90-4e86-8717-d5881b3c19ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:50.105406 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4294/8548] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:50.126628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:50.128782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:50.231922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70349d79-6691-463c-8ef4-b0999da6da0e service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:50.239423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:50.240179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:50.240528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:50.240864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:50.243150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:50.330057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:50.330057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:50.340355 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4255/8549] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:51.429216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ddc6-9155-462b-96eb-868356154c09 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:51.430434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ddc6-9155-462b-96eb-868356154c09 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:51.430434 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ddc6-9155-462b-96eb-868356154c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:51.451654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-de89ddc6-9155-462b-96eb-868356154c09 None None] MFA Rules not processed for user `87cb7cceef924db79e4add131992b8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:25:51.453029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-de89ddc6-9155-462b-96eb-868356154c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:51.460560 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4295/8550] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:25:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:25:51.890018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:51.895330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:51.926215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c847e-7751-497c-9ec7-8311542f27cd service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:51.930920 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:51.931150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:51.931633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:51.932770 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:51.934757 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:51.948870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:51.958536 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:51.967728 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4256/8551] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:25:52.175239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ddc6-9155-462b-96eb-868356154c09 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:52.198469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ddc6-9155-462b-96eb-868356154c09 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:52.203477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:52.204269 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:52.204638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:52.204985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] PATH_INFO: `/v3/users/506fc30e70e44452a648171d10ce5b3d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:52.208329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=506fc30e70e44452a648171d10ce5b3d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:52.210393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:52.269467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '506fc30e70e44452a648171d10ce5b3d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.284412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '506fc30e70e44452a648171d10ce5b3d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.292822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '506fc30e70e44452a648171d10ce5b3d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.300984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '506fc30e70e44452a648171d10ce5b3d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.302151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '506fc30e70e44452a648171d10ce5b3d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.309550 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4296/8552] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:25:52 2022] DELETE /identity/v3/users/506fc30e70e44452a648171d10ce5b3d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:52.600873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:52.611085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:52.618188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6af1f-0efd-46e4-9f47-71b9c1553d23 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:52.701781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:52.703148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:52.703576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:52.703962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] PATH_INFO: `/v3/projects/7693ab8a9ce24ff898409095f48622bd` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:52.705820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7693ab8a9ce24ff898409095f48622bd)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:52.708045 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:52.779013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7693ab8a9ce24ff898409095f48622bd', 'request_id': 'req-71617cdd-e990-42cc-954a-f226c2866110'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.785868 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7693ab8a9ce24ff898409095f48622bd', 'request_id': 'req-71617cdd-e990-42cc-954a-f226c2866110'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.797455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7693ab8a9ce24ff898409095f48622bd', 'request_id': 'req-71617cdd-e990-42cc-954a-f226c2866110'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.813553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] The token cache is being invalidate because project 7693ab8a9ce24ff898409095f48622bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:25:52.814052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.814867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:25:52.819021 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4257/8553] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:25:52 2022] DELETE /identity/v3/projects/7693ab8a9ce24ff898409095f48622bd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:25:55.504273 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:25:55.505871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:55.572684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:25:55.607526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241707f6-469f-4834-8f3a-615036612c13 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:25:55.676798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:25:55.677416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:25:55.677694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:25:55.678009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:25:55.680277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:25:55.705813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:25:55.707440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:25:55.722289 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4297/8554] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:25:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:26:02.031361 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:26:02.033162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:02.053402 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71617cdd-e990-42cc-954a-f226c2866110 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:26:02.059703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:26:02.060313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:26:02.060599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:26:02.060875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:26:02.062644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:02.077621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:26:02.079259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:26:02.094320 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4258/8555] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:26:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:26:04.618711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:26:04.620526 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:04.684439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a48478-5f38-43e3-89f1-4a216a1a1a8c service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:26:04.688295 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:26:04.688913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:26:04.689196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:26:04.689476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:26:04.691231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:04.756543 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:26:04.757936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:26:04.767702 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4298/8556] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:26:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:26:12.087583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:26:14.763439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:26:18.921211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:26:18.922807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:19.007812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1c7dc-e7b4-43a6-9b58-b437326ec24e service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:26:19.012899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:26:19.013509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:26:19.013807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:26:19.014126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:26:19.015875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:19.030325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:26:19.031900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:26:19.045031 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4259/8557] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:26:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:26:29.037554 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=37 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:26:42.810182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:26:42.811629 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:42.829420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2813d3-8b2a-42c6-a13e-7edf8ea89dd2 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:26:42.834495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:26:42.835111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:26:42.835382 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:26:42.835662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:26:42.837367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:26:42.851539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:26:42.853110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:26:42.865981 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4299/8558] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:26:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:26:52.858972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:27:00.584021 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:27:00.600336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba81466-9615-4625-b3a2-99ef4f7c982f service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:27:00.607527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97a29d-06c4-4e73-aac0-bd4bc823d2ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:27:00.608233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a29d-06c4-4e73-aac0-bd4bc823d2ee service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:00.608587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a29d-06c4-4e73-aac0-bd4bc823d2ee service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:00.608987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a29d-06c4-4e73-aac0-bd4bc823d2ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:00.611103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e97a29d-06c4-4e73-aac0-bd4bc823d2ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:00.627995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a29d-06c4-4e73-aac0-bd4bc823d2ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:27:00.629781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a29d-06c4-4e73-aac0-bd4bc823d2ee service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:27:00.662904 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4260/8559] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:27:00.906708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:27:00.908736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:00.978347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea2c4-79b2-4ea9-b341-6808d92d9c16 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:27:00.983187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:27:00.983818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:00.984101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:00.984379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] PATH_INFO: `/v3/users/b4791d13ab1c45bf8fabec70191426da` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:00.985765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4791d13ab1c45bf8fabec70191426da)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:27:00.987715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:27:01.033873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4791d13ab1c45bf8fabec70191426da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.045615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4791d13ab1c45bf8fabec70191426da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.052485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4791d13ab1c45bf8fabec70191426da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.060366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4791d13ab1c45bf8fabec70191426da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.060984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4791d13ab1c45bf8fabec70191426da'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.066790 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4300/8560] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:27:00 2022] DELETE /identity/v3/users/b4791d13ab1c45bf8fabec70191426da => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:27:01.071374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:01.071657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:01.071928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:01.091643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:27:01.092826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:01.151720 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4261/8561] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:27:01.161786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:27:01.163271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:01.225217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d806bc-dae9-4267-b694-ceedb9bd4e07 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:27:01.229011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:27:01.229624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:01.230457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:01.230865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:01.240420 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:27:01.241867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:27:01.252096 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4301/8562] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:27:01.424003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:27:01.425434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:01.439239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579df7b-e216-44d5-ae18-90710d6fff25 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:27:01.444141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:27:01.444781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:01.445063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:01.445342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] PATH_INFO: `/v3/projects/f4f036f0122f4879aacb1db06eb8b588` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:01.446485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f036f0122f4879aacb1db06eb8b588)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:27:01.448476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:27:01.497326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f036f0122f4879aacb1db06eb8b588', 'request_id': 'req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.503724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f036f0122f4879aacb1db06eb8b588', 'request_id': 'req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.511565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f036f0122f4879aacb1db06eb8b588', 'request_id': 'req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.521973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] The token cache is being invalidate because project f4f036f0122f4879aacb1db06eb8b588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:27:01.522264 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.522960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.524456 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4262/8563] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:27:01 2022] DELETE /identity/v3/projects/f4f036f0122f4879aacb1db06eb8b588 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:27:01.529182 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:27:01.539300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb09745-c11f-47e5-b852-067a61f2e46a service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:27:01.557766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:27:01.597993 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:27:01.598553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:01.598841 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:01.599122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] PATH_INFO: `/v3/users/09038e5611424270a0afa7928b3cf791` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:01.600240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09038e5611424270a0afa7928b3cf791)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:27:01.602201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:27:01.648239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09038e5611424270a0afa7928b3cf791'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.660945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09038e5611424270a0afa7928b3cf791'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.668559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09038e5611424270a0afa7928b3cf791'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.675972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09038e5611424270a0afa7928b3cf791'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.676614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09038e5611424270a0afa7928b3cf791'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:01.682522 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4302/8564] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:27:01 2022] DELETE /identity/v3/users/09038e5611424270a0afa7928b3cf791 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:27:01.859068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:27:01.860853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:01.874847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab72566-ec5f-40c2-8d4f-22ff0c95ab0f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:27:01.941522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:27:01.942196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:01.942482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:01.942759 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] PATH_INFO: `/v3/projects/48270d4f7205461aa0563e2ef69c6c44` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:01.943870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48270d4f7205461aa0563e2ef69c6c44)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:27:01.945783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:27:01.957148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:27:01.994387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48270d4f7205461aa0563e2ef69c6c44', 'request_id': 'req-84953508-a840-40ee-bd0c-be42412ea4c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:02.001009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48270d4f7205461aa0563e2ef69c6c44', 'request_id': 'req-84953508-a840-40ee-bd0c-be42412ea4c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:02.008535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48270d4f7205461aa0563e2ef69c6c44', 'request_id': 'req-84953508-a840-40ee-bd0c-be42412ea4c0'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:02.019698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] The token cache is being invalidate because project 48270d4f7205461aa0563e2ef69c6c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:27:02.019986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:02.020603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:27:02.022104 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4263/8565] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:27:01 2022] DELETE /identity/v3/projects/48270d4f7205461aa0563e2ef69c6c44 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:27:09.986157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:27:09.987655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:10.050576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875074c-0e2b-4efc-a740-df78449d5954 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:27:10.100520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:27:10.101041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:27:10.101319 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:27:10.101596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:27:10.103322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:27:10.116743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:27:10.118730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:27:10.125665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:27:10.141210 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4303/8566] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:27:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:27:12.016871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:29:17.045119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:29:17.046544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:29:17.115297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84953508-a840-40ee-bd0c-be42412ea4c0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:29:17.118945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:29:17.119539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:29:17.119819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:29:17.120090 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:29:17.121772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:29:17.201608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:29:17.203202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:29:17.222565 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4264/8567] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:29:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:29:27.219472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=65 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:30:33.484977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:30:33.486618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:30:33.512132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb0ffa-824a-42b9-ab72-64da4c3f5104 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:30:33.515954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:30:33.516533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:30:33.516812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:30:33.517081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:30:33.518792 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:30:33.530739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:30:33.532152 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:30:33.543290 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4304/8568] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:30:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:30:43.537199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=10 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:14.622716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:14.625290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:14.738754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419129f0-c5c4-4d69-9880-58c38251d5fa service cinder] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:14.744435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:14.745040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:14.745324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:14.745600 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:14.747342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:14.806936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:14.808539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:14.817399 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4265/8569] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:15.016946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:15.018583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:15.031483 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec85cbc-52e9-42c7-8e9b-476c4879f085 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:15.035457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:15.036051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:15.036329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:15.036612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:15.038684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:15.055770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:15.057418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:15.072115 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4305/8570] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:16.291841 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:16.304413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8440e-a3c8-47ab-8492-d5839cc4c843 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:16.308036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:16.308638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:16.308919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:16.309193 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:16.310978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:16.327779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:16.329213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:16.343322 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4266/8571] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:16.545351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:16.548542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:16.577071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5091110-f9bd-4d12-ab02-d980177a87fa service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:16.603701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:16.604887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:16.605396 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:16.605923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:16.608119 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:16.631872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:16.633727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:16.651035 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4306/8572] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:18.187088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:18.198578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9aa4e1-924e-4adf-8bb7-0e7d4fbae9d2 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:18.202541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:18.203176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:18.203456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:18.203728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:18.205464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:18.260797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:18.274028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:18.275480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:18.294970 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4267/8573] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:21.891488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:21.893016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:21.993641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a685-022d-4858-a540-a8358d3546cc service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:21.998762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:21.999365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:21.999674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:21.999945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:22.001640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:22.016980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:22.018540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:22.031969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4307/8574] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:24.889504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:24.894356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:24.904451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46971289-c5f8-4de7-aaf7-ac80108b0fb2 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:24.909350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:24.910026 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:24.910310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:24.910585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] PATH_INFO: `/v3/users/87cb7cceef924db79e4add131992b8e1` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:24.912509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87cb7cceef924db79e4add131992b8e1)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:24.914472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:24.968766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87cb7cceef924db79e4add131992b8e1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:24.976788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87cb7cceef924db79e4add131992b8e1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:24.977797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87cb7cceef924db79e4add131992b8e1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:24.985185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87cb7cceef924db79e4add131992b8e1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:24.985784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87cb7cceef924db79e4add131992b8e1'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.003660 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4268/8575] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:24 2022] DELETE /identity/v3/users/87cb7cceef924db79e4add131992b8e1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:25.180553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:25.191586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b155d-4f9a-4132-b57d-37ef29c9df86 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:25.250408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:25.250963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:25.251245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:25.251530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] PATH_INFO: `/v3/projects/92f59058d6274eb7b3472a96ab56b55c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:25.252631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f59058d6274eb7b3472a96ab56b55c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:25.254561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:25.300558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f59058d6274eb7b3472a96ab56b55c', 'request_id': 'req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.305352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:25.307093 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f59058d6274eb7b3472a96ab56b55c', 'request_id': 'req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.315058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f59058d6274eb7b3472a96ab56b55c', 'request_id': 'req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.327756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] The token cache is being invalidate because project 92f59058d6274eb7b3472a96ab56b55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:25.328050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.328673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.330485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4308/8576] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:25 2022] DELETE /identity/v3/projects/92f59058d6274eb7b3472a96ab56b55c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:25.335179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:25.336586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:25.349667 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf20b1-ba1b-4e83-9dcb-3c209b3640f1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:25.406410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:25.407041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:25.407325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:25.407602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] PATH_INFO: `/v3/users/a2b935bfd8644e879de4a1d33b12e2b7` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:25.408739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b935bfd8644e879de4a1d33b12e2b7)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:25.410655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:25.457748 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b935bfd8644e879de4a1d33b12e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.465212 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b935bfd8644e879de4a1d33b12e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.465929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b935bfd8644e879de4a1d33b12e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.467526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:25.474786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b935bfd8644e879de4a1d33b12e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.475531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-057c24c9-c760-44c8-a545-fa980b357f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b935bfd8644e879de4a1d33b12e2b7'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.494711 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4269/8577] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:25 2022] DELETE /identity/v3/users/a2b935bfd8644e879de4a1d33b12e2b7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:25.668394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:25.669896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:25.686313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1624b1-d31a-4dd1-bb07-aece0ba08eac admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:25.747310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:25.747940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:25.748223 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:25.748499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] PATH_INFO: `/v3/projects/1830b4f217224825a86fab1b8103117b` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:25.749599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1830b4f217224825a86fab1b8103117b)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:25.751576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:25.797804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1830b4f217224825a86fab1b8103117b', 'request_id': 'req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.803441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1830b4f217224825a86fab1b8103117b', 'request_id': 'req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.809605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1830b4f217224825a86fab1b8103117b', 'request_id': 'req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.822997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] The token cache is being invalidate because project 1830b4f217224825a86fab1b8103117b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:25.823288 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.823940 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-2cdae5ac-a9f2-4a69-9bfa-199c5eb27703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:25.825422 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4309/8578] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:25 2022] DELETE /identity/v3/projects/1830b4f217224825a86fab1b8103117b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:31:25.830377 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba842e4-a580-4ef6-af69-cb7ab568914e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:25.830669 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba842e4-a580-4ef6-af69-cb7ab568914e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:25.830949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba842e4-a580-4ef6-af69-cb7ab568914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:25.870698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-aba842e4-a580-4ef6-af69-cb7ab568914e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:25.871903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-aba842e4-a580-4ef6-af69-cb7ab568914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:25.938889 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4270/8579] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:25.946877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac28ca-83aa-40c1-ab97-e1d67632b86f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:25.947170 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac28ca-83aa-40c1-ab97-e1d67632b86f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:25.947451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac28ca-83aa-40c1-ab97-e1d67632b86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:25.967294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-11ac28ca-83aa-40c1-ab97-e1d67632b86f None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:25.968498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-11ac28ca-83aa-40c1-ab97-e1d67632b86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:25.977215 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4310/8580] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:25.981415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba842e4-a580-4ef6-af69-cb7ab568914e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:25.992642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba842e4-a580-4ef6-af69-cb7ab568914e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:25.997188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:25.997781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:25.998094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:25.998367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:25.999003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:31:25.999302 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:26.000882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:26.012512 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4271/8581] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:26.016787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac28ca-83aa-40c1-ab97-e1d67632b86f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:26.026864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac28ca-83aa-40c1-ab97-e1d67632b86f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:26.031665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:26.032264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:26.032549 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:26.032826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:26.033486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:26.035436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:26.073223 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4311/8582] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:26 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:26.077198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:26.088064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a1995-281b-4ea0-bcba-a4c6d0a7082c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:26.144133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:26.144800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:26.145092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:26.145370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:26.146547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:26.148494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:26.169800 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4272/8583] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:26 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:26.173875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:26.184374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6438e5-a5be-4a9d-9e3c-94c1225286a3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:26.189080 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:26.189670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:26.189990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:26.190261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:26.190891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:26.192561 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:26.202366 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4312/8584] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:26.206349 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:26.217036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fa22f-6066-4bb8-a7cf-7c4b70eeda16 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:26.221758 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb31107-320c-4c77-8007-8b35f95cfbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:26.222390 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31107-320c-4c77-8007-8b35f95cfbac admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:26.222673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31107-320c-4c77-8007-8b35f95cfbac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:26.222948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31107-320c-4c77-8007-8b35f95cfbac admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:26.224154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31107-320c-4c77-8007-8b35f95cfbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:26.225916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31107-320c-4c77-8007-8b35f95cfbac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:26.236637 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4273/8585] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:26.240997 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:26.251039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f307740-26cc-4dc3-a935-1b27c579a89e admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:26.255795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:26.256435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:26.256720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:26.257000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] PATH_INFO: `/v3/projects/a6f99e230d364f0889d2fd272fbaf1a5/users/d2c40ac4d22d4d2d9a07e293e921f046/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:26.273774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f99e230d364f0889d2fd272fbaf1a5, user_id=d2c40ac4d22d4d2d9a07e293e921f046, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:26.276451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:26.291309 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4313/8586] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:26 2022] PUT /identity/v3/projects/a6f99e230d364f0889d2fd272fbaf1a5/users/d2c40ac4d22d4d2d9a07e293e921f046/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:31:26.296368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092794a-d10f-43ee-bdca-723b6fec6683 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:26.296656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092794a-d10f-43ee-bdca-723b6fec6683 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:26.296935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092794a-d10f-43ee-bdca-723b6fec6683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:26.315455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8092794a-d10f-43ee-bdca-723b6fec6683 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:26.316651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8092794a-d10f-43ee-bdca-723b6fec6683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:26.335615 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:26.377923 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4274/8587] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:26.389291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:26.391015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:26.396909 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:26.454186 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c546f-f515-4f2a-95e4-108998db2dd1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:26.457798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:26.458781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:26.459069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:26.459345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:26.468546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:26.470065 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:26.480290 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4314/8588] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:30.329494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36267e57-b231-4d5c-895b-0dcacf8d19f9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:30.330060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36267e57-b231-4d5c-895b-0dcacf8d19f9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:30.330636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36267e57-b231-4d5c-895b-0dcacf8d19f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:30.363971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-36267e57-b231-4d5c-895b-0dcacf8d19f9 None None] MFA Rules not processed for user `d2c40ac4d22d4d2d9a07e293e921f046`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:30.366852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36267e57-b231-4d5c-895b-0dcacf8d19f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:30.441968 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4275/8589] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:30.447514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:30.457641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93eef1-380b-4123-99c9-246ef14fd77a service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:30.463831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:30.464525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:30.464902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:30.465246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:30.466024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:30.468023 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:30.490545 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4315/8590] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:30 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:30.494682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36267e57-b231-4d5c-895b-0dcacf8d19f9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:30.506861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36267e57-b231-4d5c-895b-0dcacf8d19f9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:30.511683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864750ce-2cd2-45ea-80fa-1f0f5a75739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:30.512306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864750ce-2cd2-45ea-80fa-1f0f5a75739d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:30.512584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864750ce-2cd2-45ea-80fa-1f0f5a75739d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:30.512858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864750ce-2cd2-45ea-80fa-1f0f5a75739d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:30.513504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864750ce-2cd2-45ea-80fa-1f0f5a75739d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:30.515174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864750ce-2cd2-45ea-80fa-1f0f5a75739d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:30.526676 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4276/8591] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:31:30.530921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:30.542551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de91ebb-f3eb-48ef-bc0f-d80e19d9100a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:30.547952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:30.548559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:30.548850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:30.549128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] PATH_INFO: `/v3/system/users/9e5c861da3e2484789fbae95d17782c8/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:30.566333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9e5c861da3e2484789fbae95d17782c8, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:30.567907 ncdv-master-1 devstack@keystone.service[93527]: WARNING py.warnings [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 21:31:30.567907 ncdv-master-1 devstack@keystone.service[93527]: warnings.warn(msg) Sep 10 21:31:30.567907 ncdv-master-1 devstack@keystone.service[93527]: Sep 10 21:31:30.568867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:30.576614 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4316/8592] 10.222.0.33 () {60 vars in 1378 bytes} [Sat Sep 10 21:31:30 2022] PUT /identity/v3/system/users/9e5c861da3e2484789fbae95d17782c8/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:31:30.587681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35435192-e69b-4ccd-8115-c1f9277c7efe None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:30.588052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35435192-e69b-4ccd-8115-c1f9277c7efe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:30.588380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35435192-e69b-4ccd-8115-c1f9277c7efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:30.625793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-35435192-e69b-4ccd-8115-c1f9277c7efe None None] MFA Rules not processed for user `9e5c861da3e2484789fbae95d17782c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:30.627156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-35435192-e69b-4ccd-8115-c1f9277c7efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:30.720479 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4277/8593] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:30 2022] POST /identity/v3/auth/tokens => generated 2415 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 21:31:30.724930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:30.850538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f483135-a575-42eb-ac9a-3d3ffd93a802 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:30.870390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:30.893938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e5c861da3e2484789fbae95d17782c8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:30.894659 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:30.895000 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:30.895405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] PATH_INFO: `/v3/services` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:30.895978 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:30.898755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:30.907029 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4317/8594] 10.222.0.33 () {58 vars in 1110 bytes} [Sat Sep 10 21:31:30 2022] GET /identity/v3/services => generated 2191 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:30.911508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35435192-e69b-4ccd-8115-c1f9277c7efe None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:30.940102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:31.052727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35435192-e69b-4ccd-8115-c1f9277c7efe None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:31.091172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e5c861da3e2484789fbae95d17782c8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:31.091786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:31.092069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:31.092345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] PATH_INFO: `/v3/limits` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:31.093005 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:31.094342 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Sep 10 21:31:31.140152 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application raise exception.NoLimitReference Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: ERROR keystone.server.flask.application Sep 10 21:31:31.141310 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4278/8595] 10.222.0.33 () {60 vars in 1126 bytes} [Sat Sep 10 21:31:30 2022] POST /identity/v3/limits => generated 124 bytes in 230 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 10 21:31:31.146519 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:31.251513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f68961-2638-4297-9fe4-9230d6866b7d None tempest-ImageQuotaTest-1383182335-system-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:31.284277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e5c861da3e2484789fbae95d17782c8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:31.284876 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:31.285157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:31.285450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:31.286583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:31.287638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:31.293875 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4318/8596] 10.222.0.33 () {58 vars in 1137 bytes} [Sat Sep 10 21:31:31 2022] GET /identity/v3/registered_limits => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:31.299897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:31.375618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:31.410531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a67644-377b-4cc4-a940-81790bc16fca None tempest-ImageQuotaTest-1383182335-system-admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:31.447608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a509a-6705-485a-9240-2cbff678a409 None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e5c861da3e2484789fbae95d17782c8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:31.448231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a509a-6705-485a-9240-2cbff678a409 None tempest-ImageQuotaTest-1383182335-system-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:31.448518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a509a-6705-485a-9240-2cbff678a409 None tempest-ImageQuotaTest-1383182335-system-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:31.448794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a509a-6705-485a-9240-2cbff678a409 None tempest-ImageQuotaTest-1383182335-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:31.449407 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a509a-6705-485a-9240-2cbff678a409 None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:31.450524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a509a-6705-485a-9240-2cbff678a409 None tempest-ImageQuotaTest-1383182335-system-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:31.456020 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4279/8597] 10.222.0.33 () {58 vars in 1137 bytes} [Sat Sep 10 21:31:31 2022] GET /identity/v3/registered_limits => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:32.145086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:32.147717 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:32.162915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3dd3e-d05d-4cff-ba72-99f076c76b5a None tempest-ImageQuotaTest-1383182335-system-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:32.167527 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:32.168191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:32.168478 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:32.168759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] PATH_INFO: `/v3/users/2cacc5411a7f4771b8f4ef7d5eb652e3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:32.170034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cacc5411a7f4771b8f4ef7d5eb652e3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:32.172031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:32.227615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:32.232912 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cacc5411a7f4771b8f4ef7d5eb652e3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.251602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cacc5411a7f4771b8f4ef7d5eb652e3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.259889 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cacc5411a7f4771b8f4ef7d5eb652e3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.269108 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cacc5411a7f4771b8f4ef7d5eb652e3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.269846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cacc5411a7f4771b8f4ef7d5eb652e3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.280456 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4319/8598] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:32 2022] DELETE /identity/v3/users/2cacc5411a7f4771b8f4ef7d5eb652e3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:32.282245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:32.282993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:32.282993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:32.307157 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:32.309118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:32.398643 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4280/8599] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:31:32.408469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:32.410066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:32.481712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3becf-6b0f-48a5-ac53-ede7abf4df7b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:32.486090 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:32.486729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:32.487014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:32.487293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:32.498129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:32.499661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:32.512021 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4320/8600] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:32.823189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:32.824876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:32.839495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25fdb9-9ba2-46c6-b221-b9bbd9173c67 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:32.847740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:32.851236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:32.856176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:32.856469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] PATH_INFO: `/v3/projects/a12c8f1460bd4b93990ce09297d0ea53` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:32.857797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12c8f1460bd4b93990ce09297d0ea53)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:32.860048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:32.920653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12c8f1460bd4b93990ce09297d0ea53', 'request_id': 'req-f4187e2c-d73f-47df-85e5-dc4bc69544ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.928343 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12c8f1460bd4b93990ce09297d0ea53', 'request_id': 'req-f4187e2c-d73f-47df-85e5-dc4bc69544ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.943910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12c8f1460bd4b93990ce09297d0ea53', 'request_id': 'req-f4187e2c-d73f-47df-85e5-dc4bc69544ee'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.959030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] The token cache is being invalidate because project a12c8f1460bd4b93990ce09297d0ea53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:32.959637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.960893 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:32.968369 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4281/8601] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:32 2022] DELETE /identity/v3/projects/a12c8f1460bd4b93990ce09297d0ea53 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:32.973224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:32.986363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fe934-eb65-4217-b0d2-527395189bd6 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:33.054577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:33.055266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:33.055593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:33.055901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] PATH_INFO: `/v3/users/4001d51354dc4117ab7f740d7aecf9b3` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:33.073810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4001d51354dc4117ab7f740d7aecf9b3)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:33.076372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:33.135810 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4001d51354dc4117ab7f740d7aecf9b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.142181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:33.153044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4001d51354dc4117ab7f740d7aecf9b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.164029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4001d51354dc4117ab7f740d7aecf9b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.172864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4001d51354dc4117ab7f740d7aecf9b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.173422 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-abea2496-e818-4848-b23d-9cfa9cc301e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4001d51354dc4117ab7f740d7aecf9b3'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.184069 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4321/8602] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:32 2022] DELETE /identity/v3/users/4001d51354dc4117ab7f740d7aecf9b3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:33.659800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:33.661357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:33.682315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4187e2c-d73f-47df-85e5-dc4bc69544ee admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:33.700338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:33.758943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:33.759596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:33.759909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:33.760182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] PATH_INFO: `/v3/projects/55665d6a7f5f4a8faea85c4f8c1c547d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:33.772937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=55665d6a7f5f4a8faea85c4f8c1c547d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:33.775175 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:33.827692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55665d6a7f5f4a8faea85c4f8c1c547d', 'request_id': 'req-8d6eeaea-d37b-4487-bc68-c9f9111697cd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.842222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55665d6a7f5f4a8faea85c4f8c1c547d', 'request_id': 'req-8d6eeaea-d37b-4487-bc68-c9f9111697cd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.859431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55665d6a7f5f4a8faea85c4f8c1c547d', 'request_id': 'req-8d6eeaea-d37b-4487-bc68-c9f9111697cd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.872768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] The token cache is being invalidate because project 55665d6a7f5f4a8faea85c4f8c1c547d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:33.873075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.873768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8d6eeaea-d37b-4487-bc68-c9f9111697cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:33.881617 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4282/8603] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:33 2022] DELETE /identity/v3/projects/55665d6a7f5f4a8faea85c4f8c1c547d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:33.883176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5240065-1dc2-464c-b8ac-6ea1e1c78bf9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:33.883493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5240065-1dc2-464c-b8ac-6ea1e1c78bf9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:33.883777 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5240065-1dc2-464c-b8ac-6ea1e1c78bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:33.907015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a5240065-1dc2-464c-b8ac-6ea1e1c78bf9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:33.908409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a5240065-1dc2-464c-b8ac-6ea1e1c78bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:33.998848 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4322/8604] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:34.007695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c84be-353c-4c79-b86a-e7c2ff88d525 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.007999 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c84be-353c-4c79-b86a-e7c2ff88d525 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.008272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c84be-353c-4c79-b86a-e7c2ff88d525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.032050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a28c84be-353c-4c79-b86a-e7c2ff88d525 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:34.032935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a28c84be-353c-4c79-b86a-e7c2ff88d525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:34.043156 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4283/8605] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:34.047133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5240065-1dc2-464c-b8ac-6ea1e1c78bf9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.057794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5240065-1dc2-464c-b8ac-6ea1e1c78bf9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.062598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.063315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.063672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.064011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.064731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:31:34.065107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.067728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.080768 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4323/8606] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:34.085334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c84be-353c-4c79-b86a-e7c2ff88d525 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.095118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c84be-353c-4c79-b86a-e7c2ff88d525 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.101331 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.101978 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.102266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.102544 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.103189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.105103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.140648 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4284/8607] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:34 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:34.146397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.164451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59265895-e1fb-4667-8032-7496a6031a84 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.246229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.246806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.247094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.247374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.248005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.250061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.274244 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4324/8608] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:34 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:31:34.278339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.290232 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a25842-0bd2-47b6-a804-0d5da510e6c5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.296018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.296650 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.297375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.297375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.297980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.299732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.312438 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4285/8609] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:34.316338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.329307 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ee84c-3003-4036-8422-344d824af287 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.335667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253433-4699-4128-8cea-c6fb7c0061f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.336394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253433-4699-4128-8cea-c6fb7c0061f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.336747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253433-4699-4128-8cea-c6fb7c0061f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.337087 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253433-4699-4128-8cea-c6fb7c0061f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.337853 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253433-4699-4128-8cea-c6fb7c0061f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.339536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253433-4699-4128-8cea-c6fb7c0061f0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.361867 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4325/8610] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:34.365885 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.376220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80217660-15e8-45d7-8c33-f9da22de7801 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.380806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.381462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.381761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.382056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] PATH_INFO: `/v3/projects/322f19482545448a9fa320541c3db8f3/users/6597fcc0995c475cb135d5b9ddd9b457/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.397809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] RBAC: Authorizing `identity:create_grant(project_id=322f19482545448a9fa320541c3db8f3, user_id=6597fcc0995c475cb135d5b9ddd9b457, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.400489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.415102 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4286/8611] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:34 2022] PUT /identity/v3/projects/322f19482545448a9fa320541c3db8f3/users/6597fcc0995c475cb135d5b9ddd9b457/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:34.427103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04eb02-2905-422f-8877-1e0a8379832a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.427390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04eb02-2905-422f-8877-1e0a8379832a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.427693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04eb02-2905-422f-8877-1e0a8379832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.456696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9b04eb02-2905-422f-8877-1e0a8379832a None None] MFA Rules not processed for user `6597fcc0995c475cb135d5b9ddd9b457`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:34.458894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9b04eb02-2905-422f-8877-1e0a8379832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:34.487168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:34.537118 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4326/8612] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:34.541779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.593188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:34.650942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80660d31-28a4-48ea-beb0-28db3bb3042e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.664965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.666294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.666760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.667200 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.668063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.671056 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.731090 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4287/8613] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:34 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:31:34.735726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04eb02-2905-422f-8877-1e0a8379832a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.755980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04eb02-2905-422f-8877-1e0a8379832a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.827096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.827728 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.828435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.828724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.829366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.831444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.873515 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4327/8614] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:34 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:34.874949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.892914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2ff0d-fc61-4e4f-acdd-21b64ac4aeeb admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.904281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.905184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.905670 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.907800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:34.908837 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:34.912503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:34.942043 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4288/8615] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:34.945880 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:34.971376 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43254db-b46b-4617-90b3-0c3231cdb352 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:34.980656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:34.981942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:34.982472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:34.982767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] PATH_INFO: `/v3/projects/5ad1064e92684e31a9dfc6d929683430/users/ad38791b54204b449ddd0c6c92021e1c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.014395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad1064e92684e31a9dfc6d929683430, user_id=ad38791b54204b449ddd0c6c92021e1c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:35.018512 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:35.040513 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4328/8616] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:34 2022] PUT /identity/v3/projects/5ad1064e92684e31a9dfc6d929683430/users/ad38791b54204b449ddd0c6c92021e1c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:31:35.041798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:35.058035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ea7b3-2b80-4c5a-8fbd-45a241d24bd6 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:35.156454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:35.157417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:35.157764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:35.158299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.159210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:35.167458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:35.182621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:35.183225 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4289/8617] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:31:35.187241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:35.187784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:35.199192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0543ac-4679-4085-b851-a7c97d5111ae admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:35.211322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298ed73-fe59-4c02-a9f3-3b1008c34253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:35.213063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ed73-fe59-4c02-a9f3-3b1008c34253 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:35.213567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ed73-fe59-4c02-a9f3-3b1008c34253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:35.215088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298ed73-fe59-4c02-a9f3-3b1008c34253 admin admin] PATH_INFO: `/v3/projects/5ad1064e92684e31a9dfc6d929683430/users/ad38791b54204b449ddd0c6c92021e1c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.228626 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298ed73-fe59-4c02-a9f3-3b1008c34253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad1064e92684e31a9dfc6d929683430, user_id=ad38791b54204b449ddd0c6c92021e1c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:35.232182 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298ed73-fe59-4c02-a9f3-3b1008c34253 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:35.249857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4290/8618] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:35 2022] PUT /identity/v3/projects/5ad1064e92684e31a9dfc6d929683430/users/ad38791b54204b449ddd0c6c92021e1c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:35.264404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:35.264695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:35.264970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.291976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fb600-5cef-4512-8628-bb48c1660275 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:35.303380 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] MFA Rules not processed for user `ad38791b54204b449ddd0c6c92021e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:35.305556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:35.338297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:35.370863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:35.371480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:35.371766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:35.373169 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.374473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:35.390066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:35.391725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:35.400350 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4291/8619] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:35.400966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:35.402923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:35.413634 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4329/8620] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:35.421468 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:35.423082 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:35.425942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:35.502865 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c85dd-821c-4ddb-800e-123cafe8463c None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:35.508572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fa8ea-b7b2-4ed8-9156-b0d8e72e1482 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:35.509397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:35.511468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:35.511753 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:35.512181 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] PATH_INFO: `/v3/users/d2c40ac4d22d4d2d9a07e293e921f046` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.513103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:35.513714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:35.514037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:35.514322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.514928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c40ac4d22d4d2d9a07e293e921f046)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:35.516879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:35.575492 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c40ac4d22d4d2d9a07e293e921f046'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:35.583796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c40ac4d22d4d2d9a07e293e921f046'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:35.584491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c40ac4d22d4d2d9a07e293e921f046'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:35.591118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:35.592665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c40ac4d22d4d2d9a07e293e921f046'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:35.592977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:35.593627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c40ac4d22d4d2d9a07e293e921f046'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:35.604236 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4330/8621] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:35.612314 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4292/8622] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:35 2022] DELETE /identity/v3/users/d2c40ac4d22d4d2d9a07e293e921f046 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:35.804640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:35.828107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0852f-efa1-49b5-b3c5-0cf4b46f26aa service cinder] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:35.918944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:35.919592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:35.919874 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:35.920151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] PATH_INFO: `/v3/projects/a6f99e230d364f0889d2fd272fbaf1a5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:35.921305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f99e230d364f0889d2fd272fbaf1a5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:35.923254 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:35.938873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=100 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:35.983976 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f99e230d364f0889d2fd272fbaf1a5', 'request_id': 'req-d65f6576-0b76-4cf9-b6ba-935471ac3aae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:35.990502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f99e230d364f0889d2fd272fbaf1a5', 'request_id': 'req-d65f6576-0b76-4cf9-b6ba-935471ac3aae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:35.996951 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f99e230d364f0889d2fd272fbaf1a5', 'request_id': 'req-d65f6576-0b76-4cf9-b6ba-935471ac3aae'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.009991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] The token cache is being invalidate because project a6f99e230d364f0889d2fd272fbaf1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:36.010292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.010932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.012455 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4331/8623] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:35 2022] DELETE /identity/v3/projects/a6f99e230d364f0889d2fd272fbaf1a5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:36.016456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:36.017901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:36.031527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158eb930-2233-41ba-91d4-e64b11b4ea96 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:36.089524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:36.090196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:36.090484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:36.090762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] PATH_INFO: `/v3/users/9e5c861da3e2484789fbae95d17782c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:36.091912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5c861da3e2484789fbae95d17782c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:36.093872 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:36.127526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:36.156406 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5c861da3e2484789fbae95d17782c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.163576 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5c861da3e2484789fbae95d17782c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.164244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5c861da3e2484789fbae95d17782c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.171741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5c861da3e2484789fbae95d17782c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.172500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4dad86f8-ba14-4a9b-86a0-684f0d6f05ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5c861da3e2484789fbae95d17782c8'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:36.190416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4293/8624] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:36 2022] DELETE /identity/v3/users/9e5c861da3e2484789fbae95d17782c8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:38.581716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:38.583818 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:38.596272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f6576-0b76-4cf9-b6ba-935471ac3aae admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:38.652097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:38.652745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:38.653026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:38.653300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] PATH_INFO: `/v3/users/6597fcc0995c475cb135d5b9ddd9b457` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:38.654425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6597fcc0995c475cb135d5b9ddd9b457)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:38.656336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:38.701838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6597fcc0995c475cb135d5b9ddd9b457'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:38.716282 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6597fcc0995c475cb135d5b9ddd9b457'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:38.723134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6597fcc0995c475cb135d5b9ddd9b457'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:38.730988 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6597fcc0995c475cb135d5b9ddd9b457'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:38.731756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6597fcc0995c475cb135d5b9ddd9b457'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:38.738032 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4332/8625] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:38 2022] DELETE /identity/v3/users/6597fcc0995c475cb135d5b9ddd9b457 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:38.742691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:38.743158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:38.743282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:38.763062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:38.764265 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:38.822857 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4294/8626] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:38.833133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:38.834592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:38.854546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:38.896961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e913c-0324-426f-91ff-28a62341fe7b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:38.900645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:38.901246 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:38.901529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:38.901805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:38.911210 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:38.912727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:38.922453 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4333/8627] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:39.199243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:39.200735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:39.219217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680b66-40b9-433a-a907-bd3b4b53bcd7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:39.224990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:39.225627 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:39.225944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:39.226216 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] PATH_INFO: `/v3/projects/322f19482545448a9fa320541c3db8f3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:39.227328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=322f19482545448a9fa320541c3db8f3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:39.229238 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:39.276068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322f19482545448a9fa320541c3db8f3', 'request_id': 'req-eddc648b-abca-4716-b06a-fc8534c144f5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.282281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322f19482545448a9fa320541c3db8f3', 'request_id': 'req-eddc648b-abca-4716-b06a-fc8534c144f5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.289858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322f19482545448a9fa320541c3db8f3', 'request_id': 'req-eddc648b-abca-4716-b06a-fc8534c144f5'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.300350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] The token cache is being invalidate because project 322f19482545448a9fa320541c3db8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:39.300637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.301991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.303655 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4295/8628] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:39 2022] DELETE /identity/v3/projects/322f19482545448a9fa320541c3db8f3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:39.307842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:39.317925 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648de6-1eb6-41b5-875a-96148edd553e service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:39.379138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:39.380051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:39.380424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:39.380698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] PATH_INFO: `/v3/users/ad38791b54204b449ddd0c6c92021e1c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:39.381815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad38791b54204b449ddd0c6c92021e1c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:39.383742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:39.433595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad38791b54204b449ddd0c6c92021e1c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.445426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad38791b54204b449ddd0c6c92021e1c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.452276 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad38791b54204b449ddd0c6c92021e1c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.459975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad38791b54204b449ddd0c6c92021e1c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.460627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-a92145da-71b7-4414-b90f-34b61b09e3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad38791b54204b449ddd0c6c92021e1c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.466533 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4334/8629] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:39 2022] DELETE /identity/v3/users/ad38791b54204b449ddd0c6c92021e1c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:39.746286 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:39.747773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:39.761456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc648b-abca-4716-b06a-fc8534c144f5 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:39.788664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:39.828996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:39.829631 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:39.829948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:39.830241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] PATH_INFO: `/v3/projects/5ad1064e92684e31a9dfc6d929683430` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:39.831336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad1064e92684e31a9dfc6d929683430)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:39.833808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:39.879640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad1064e92684e31a9dfc6d929683430', 'request_id': 'req-ec6b14e8-4356-47c1-8055-1a4171f5467a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.885808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad1064e92684e31a9dfc6d929683430', 'request_id': 'req-ec6b14e8-4356-47c1-8055-1a4171f5467a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.893677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad1064e92684e31a9dfc6d929683430', 'request_id': 'req-ec6b14e8-4356-47c1-8055-1a4171f5467a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.904844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] The token cache is being invalidate because project 5ad1064e92684e31a9dfc6d929683430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:39.905141 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.905801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-ec6b14e8-4356-47c1-8055-1a4171f5467a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:39.907311 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4296/8630] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:39 2022] DELETE /identity/v3/projects/5ad1064e92684e31a9dfc6d929683430 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:39.913226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c39f-de19-4155-b376-9877b1747bf3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:39.913720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c39f-de19-4155-b376-9877b1747bf3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:39.913858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38c39f-de19-4155-b376-9877b1747bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:39.933413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ba38c39f-de19-4155-b376-9877b1747bf3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:39.934580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ba38c39f-de19-4155-b376-9877b1747bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:39.968588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:39.997194 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4335/8631] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:40.005698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a94832-4b3b-4c9d-892f-ff6a1f9877ad None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.006041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a94832-4b3b-4c9d-892f-ff6a1f9877ad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.006320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a94832-4b3b-4c9d-892f-ff6a1f9877ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.026445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-30a94832-4b3b-4c9d-892f-ff6a1f9877ad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:40.027678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-30a94832-4b3b-4c9d-892f-ff6a1f9877ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:40.036153 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4297/8632] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:40.040575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38c39f-de19-4155-b376-9877b1747bf3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:40.050341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38c39f-de19-4155-b376-9877b1747bf3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:40.055597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:40.056190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.056474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.056772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.057411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:31:40.057716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:40.059343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:40.071103 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4336/8633] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:40.075387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a94832-4b3b-4c9d-892f-ff6a1f9877ad None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:40.085731 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a94832-4b3b-4c9d-892f-ff6a1f9877ad None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:40.090500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:40.091097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.091384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.091663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.092287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:40.094252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:40.122874 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4298/8634] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:40 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:40.127009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:40.137452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb4934-8b9f-4710-814a-5f7ccef10dd6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:40.196009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:40.196635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.196920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.197228 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.197954 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:40.199879 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:40.223752 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4337/8635] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:40 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:40.224982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:40.235967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5121c2-cb3f-43d7-a52b-a4eeb02cfb7d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:40.242015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:40.242614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.242899 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.243179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.243820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:40.245428 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:40.255290 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4299/8636] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:40.259352 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:40.269416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e26a4-b93e-4f44-9e1d-ae550d2e8c1a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:40.274212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a2dfa-bc65-4de3-8af7-f79e4a3910dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:40.274807 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a2dfa-bc65-4de3-8af7-f79e4a3910dc admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.275091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a2dfa-bc65-4de3-8af7-f79e4a3910dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.275369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a2dfa-bc65-4de3-8af7-f79e4a3910dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.276019 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a2dfa-bc65-4de3-8af7-f79e4a3910dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:40.277589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a2dfa-bc65-4de3-8af7-f79e4a3910dc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:40.288222 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4338/8637] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:40.292451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:40.303038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa11ecd-1a6e-4ac0-a13d-9bee5b0487f3 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:40.308114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:40.308756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.309043 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.309321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] PATH_INFO: `/v3/projects/f7de2245ab2c4e53910398a5b7f6981c/users/465d8a49f92b4beaa99b43ec7fe09234/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.323833 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7de2245ab2c4e53910398a5b7f6981c, user_id=465d8a49f92b4beaa99b43ec7fe09234, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:40.326543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:40.340446 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4300/8638] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:40 2022] PUT /identity/v3/projects/f7de2245ab2c4e53910398a5b7f6981c/users/465d8a49f92b4beaa99b43ec7fe09234/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:40.352463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69124ed1-8881-4f9c-898c-fb5b868dc344 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.352754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69124ed1-8881-4f9c-898c-fb5b868dc344 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.353037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69124ed1-8881-4f9c-898c-fb5b868dc344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.382399 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-69124ed1-8881-4f9c-898c-fb5b868dc344 None None] MFA Rules not processed for user `465d8a49f92b4beaa99b43ec7fe09234`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:40.383615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-69124ed1-8881-4f9c-898c-fb5b868dc344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:40.450397 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4339/8639] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:40.463038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:40.465036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:40.524179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:40.554982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23b77a-edfa-4773-8fd4-075a43762bc1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:40.558673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ad141-8e9d-4a72-8d57-55a8016dd48c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:40.559266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ad141-8e9d-4a72-8d57-55a8016dd48c service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:40.559545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ad141-8e9d-4a72-8d57-55a8016dd48c service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:40.559818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ad141-8e9d-4a72-8d57-55a8016dd48c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:40.569066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ad141-8e9d-4a72-8d57-55a8016dd48c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:40.570521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ad141-8e9d-4a72-8d57-55a8016dd48c service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:40.579528 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4301/8640] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:42.233289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69124ed1-8881-4f9c-898c-fb5b868dc344 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:42.273541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:42.298047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69124ed1-8881-4f9c-898c-fb5b868dc344 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:42.302753 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:42.303373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:42.303657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:42.303932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] PATH_INFO: `/v3/users/465d8a49f92b4beaa99b43ec7fe09234` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:42.305035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] RBAC: Authorizing `identity:delete_user(user_id=465d8a49f92b4beaa99b43ec7fe09234)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:42.306992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:42.352669 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465d8a49f92b4beaa99b43ec7fe09234'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.365032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465d8a49f92b4beaa99b43ec7fe09234'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.371969 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465d8a49f92b4beaa99b43ec7fe09234'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.379138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465d8a49f92b4beaa99b43ec7fe09234'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.380503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465d8a49f92b4beaa99b43ec7fe09234'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.386384 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4340/8641] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:42 2022] DELETE /identity/v3/users/465d8a49f92b4beaa99b43ec7fe09234 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:42.390931 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:42.391394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:42.391503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:42.411881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:42.413095 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:42.470330 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4302/8642] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:42.481258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:42.482856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:42.542881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6e0f1-6eda-4dde-9a32-d6c8b615bfab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:42.546537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766a6cb-10e1-4215-8e28-29e0f2ef5dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:42.547163 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766a6cb-10e1-4215-8e28-29e0f2ef5dea service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:42.547448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766a6cb-10e1-4215-8e28-29e0f2ef5dea service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:42.547724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766a6cb-10e1-4215-8e28-29e0f2ef5dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:42.557213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766a6cb-10e1-4215-8e28-29e0f2ef5dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:42.558667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766a6cb-10e1-4215-8e28-29e0f2ef5dea service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:42.569266 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4341/8643] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:31:42.834567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:42.836116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:42.850016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb153c9-2f22-4e5b-828a-17ad39dc11f7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:42.855128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:42.855367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:42.855797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:42.855968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] PATH_INFO: `/v3/projects/f7de2245ab2c4e53910398a5b7f6981c` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:42.857344 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7de2245ab2c4e53910398a5b7f6981c)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:42.859497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:42.912819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7de2245ab2c4e53910398a5b7f6981c', 'request_id': 'req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.919101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7de2245ab2c4e53910398a5b7f6981c', 'request_id': 'req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.927132 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7de2245ab2c4e53910398a5b7f6981c', 'request_id': 'req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.937588 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] The token cache is being invalidate because project f7de2245ab2c4e53910398a5b7f6981c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:42.937911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.938567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b7eed9dd-4f7f-4d46-955a-41fe0cf2e38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:42.940167 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4303/8644] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:42 2022] DELETE /identity/v3/projects/f7de2245ab2c4e53910398a5b7f6981c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:31:42.947621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c183-44e5-4494-b1fd-a69cbcbd6356 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:42.947911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c183-44e5-4494-b1fd-a69cbcbd6356 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:42.948195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1c183-44e5-4494-b1fd-a69cbcbd6356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:42.968340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b1a1c183-44e5-4494-b1fd-a69cbcbd6356 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:42.969557 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b1a1c183-44e5-4494-b1fd-a69cbcbd6356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:43.023462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:43.034188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4342/8645] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:31:43.042545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3792506-0156-4f43-a281-4b3590065496 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.042882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3792506-0156-4f43-a281-4b3590065496 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.043109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3792506-0156-4f43-a281-4b3590065496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.064436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a3792506-0156-4f43-a281-4b3590065496 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:43.065657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a3792506-0156-4f43-a281-4b3590065496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:43.073759 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4304/8646] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:43.078031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1c183-44e5-4494-b1fd-a69cbcbd6356 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:43.088250 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1c183-44e5-4494-b1fd-a69cbcbd6356 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:43.092833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:43.093430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.093711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.094022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.094679 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:31:43.094972 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:43.096593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:43.108541 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4343/8647] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:43.113057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3792506-0156-4f43-a281-4b3590065496 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:43.122973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3792506-0156-4f43-a281-4b3590065496 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:43.127618 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:43.128209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.128491 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.128764 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.129586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:43.131500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:43.158085 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4305/8648] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:43 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:43.163068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:43.173694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f5088-9962-4b33-abe8-af3ea44d20f1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:43.231095 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:43.231710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.231992 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.232266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.232906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:43.235129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:43.256817 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4344/8649] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:43 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:43.261101 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:43.272354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb293-4bc5-418e-a188-49ea14ffa56a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:43.277038 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:43.277568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.277878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.278154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.278787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:43.280360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:43.290538 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4306/8650] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:31:43.294309 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:43.304257 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42775771-d64d-477d-bcbb-3eeb2e741ada admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:43.308924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade11b73-fdcf-4637-b312-7e1658dabf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:43.309516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade11b73-fdcf-4637-b312-7e1658dabf36 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.309795 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade11b73-fdcf-4637-b312-7e1658dabf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.310120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade11b73-fdcf-4637-b312-7e1658dabf36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.310757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade11b73-fdcf-4637-b312-7e1658dabf36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:43.312330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade11b73-fdcf-4637-b312-7e1658dabf36 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:43.323701 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4345/8651] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:43.327796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:43.338288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e546fc8-82f9-4e63-8ba0-17995ade8f16 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:43.342874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:43.343509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.343790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.344063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] PATH_INFO: `/v3/projects/a2e031b0eebf43ab98786a243e9dc7e5/users/dfcf19bb7b0940e4bda32093a325b381/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.356035 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:43.359795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e031b0eebf43ab98786a243e9dc7e5, user_id=dfcf19bb7b0940e4bda32093a325b381, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:43.362648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:43.381884 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4307/8652] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:43 2022] PUT /identity/v3/projects/a2e031b0eebf43ab98786a243e9dc7e5/users/dfcf19bb7b0940e4bda32093a325b381/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:43.392220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60fdd-3e54-46d2-9576-47e43884cfe5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.392514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60fdd-3e54-46d2-9576-47e43884cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.392794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60fdd-3e54-46d2-9576-47e43884cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.420682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-85c60fdd-3e54-46d2-9576-47e43884cfe5 None None] MFA Rules not processed for user `dfcf19bb7b0940e4bda32093a325b381`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:43.421950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-85c60fdd-3e54-46d2-9576-47e43884cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:43.487590 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4346/8653] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:43.500099 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:43.501888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:43.561249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb01a54-2102-47b8-ab12-0a6ccac1557e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:43.564983 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581213d8-7321-4b8a-8688-f37e1cc9d6dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fe5d3512475454080c8528db2208241', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:43.565587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581213d8-7321-4b8a-8688-f37e1cc9d6dd service cinder] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:43.565910 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581213d8-7321-4b8a-8688-f37e1cc9d6dd service cinder] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:43.566190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581213d8-7321-4b8a-8688-f37e1cc9d6dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:43.575239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581213d8-7321-4b8a-8688-f37e1cc9d6dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:43.576678 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581213d8-7321-4b8a-8688-f37e1cc9d6dd service cinder] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:43.585765 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4308/8654] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:45.082949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60fdd-3e54-46d2-9576-47e43884cfe5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:45.135840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:45.150397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60fdd-3e54-46d2-9576-47e43884cfe5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:45.154913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:45.155544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.155828 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.156103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] PATH_INFO: `/v3/users/dfcf19bb7b0940e4bda32093a325b381` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.157224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfcf19bb7b0940e4bda32093a325b381)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:45.159199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:45.207770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfcf19bb7b0940e4bda32093a325b381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.220001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfcf19bb7b0940e4bda32093a325b381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.227685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfcf19bb7b0940e4bda32093a325b381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.236038 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfcf19bb7b0940e4bda32093a325b381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.236732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfcf19bb7b0940e4bda32093a325b381'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.243032 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4347/8655] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:45 2022] DELETE /identity/v3/users/dfcf19bb7b0940e4bda32093a325b381 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:45.247524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.247985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.248094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.267881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:45.269073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:45.327654 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4309/8656] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:45.337487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:45.338930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:45.401553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580afa-1452-41a0-af99-fc4196714b9b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:45.405185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c347710-f091-4c8f-bf51-26cbc6c15d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:45.405790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c347710-f091-4c8f-bf51-26cbc6c15d11 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.406105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c347710-f091-4c8f-bf51-26cbc6c15d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.406381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c347710-f091-4c8f-bf51-26cbc6c15d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.416133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c347710-f091-4c8f-bf51-26cbc6c15d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:45.417554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c347710-f091-4c8f-bf51-26cbc6c15d11 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:45.427798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4348/8657] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:45.702374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:45.703876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:45.720882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7d610-9c35-4332-ad53-9f16ab03eaa9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:45.725617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:45.726298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.726654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.726818 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] PATH_INFO: `/v3/projects/a2e031b0eebf43ab98786a243e9dc7e5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.727922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e031b0eebf43ab98786a243e9dc7e5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:45.729844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:45.777368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e031b0eebf43ab98786a243e9dc7e5', 'request_id': 'req-8c0131a2-1446-46c7-a636-073e75a495ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.783502 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e031b0eebf43ab98786a243e9dc7e5', 'request_id': 'req-8c0131a2-1446-46c7-a636-073e75a495ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.791121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e031b0eebf43ab98786a243e9dc7e5', 'request_id': 'req-8c0131a2-1446-46c7-a636-073e75a495ea'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.801847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] The token cache is being invalidate because project a2e031b0eebf43ab98786a243e9dc7e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:45.802277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.803066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-8c0131a2-1446-46c7-a636-073e75a495ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:45.804539 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4310/8658] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:45 2022] DELETE /identity/v3/projects/a2e031b0eebf43ab98786a243e9dc7e5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:45.809449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29844-5b54-42db-9ab0-b0d117a0ed71 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.809780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29844-5b54-42db-9ab0-b0d117a0ed71 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.810099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf29844-5b54-42db-9ab0-b0d117a0ed71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.830022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7cf29844-5b54-42db-9ab0-b0d117a0ed71 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:45.831213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7cf29844-5b54-42db-9ab0-b0d117a0ed71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:45.891435 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4349/8659] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:45.900516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a429a8-1402-4b61-a705-386a2dcd9f62 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.900806 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a429a8-1402-4b61-a705-386a2dcd9f62 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.901086 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a429a8-1402-4b61-a705-386a2dcd9f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.920579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-58a429a8-1402-4b61-a705-386a2dcd9f62 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:45.921767 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-58a429a8-1402-4b61-a705-386a2dcd9f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:45.930370 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4311/8660] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:45.934493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf29844-5b54-42db-9ab0-b0d117a0ed71 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:45.945264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf29844-5b54-42db-9ab0-b0d117a0ed71 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:45.949780 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:45.950413 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.950699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.950980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.951623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:31:45.951931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:45.953523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:45.965857 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4350/8661] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:45.970078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a429a8-1402-4b61-a705-386a2dcd9f62 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:45.980085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a429a8-1402-4b61-a705-386a2dcd9f62 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:45.985006 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:45.985592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:45.985918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:45.986198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:45.986822 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:45.989290 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:45.999209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:46.018294 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4312/8662] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:45 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:46.022641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:46.027881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:46.034395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5a302-c32d-4dcf-92fd-38a2e269dee4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:46.092560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:46.093195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:46.093421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:46.093695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:46.094351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:46.096267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:46.117667 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4351/8663] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:46 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:46.121782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:46.133477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc1bb-a2fa-47b2-a15c-c0277a337f44 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:46.138454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:46.139060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:46.139348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:46.139609 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:46.140372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:46.141975 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:46.152340 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4313/8664] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:46.156271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:46.166977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08038-c932-465e-b902-95c04efb1352 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:46.171946 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d603-1cd5-4c0e-9408-67c1e25e227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:46.172545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d603-1cd5-4c0e-9408-67c1e25e227f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:46.172832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d603-1cd5-4c0e-9408-67c1e25e227f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:46.173110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d603-1cd5-4c0e-9408-67c1e25e227f admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:46.173759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d603-1cd5-4c0e-9408-67c1e25e227f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:46.175431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d603-1cd5-4c0e-9408-67c1e25e227f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:46.186928 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4352/8665] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:46.191012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:46.200969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c61a34-8dd7-404c-8323-56d3044b93ed admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:46.205449 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:46.206120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:46.206408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:46.206685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] PATH_INFO: `/v3/projects/88e2b2b083df40f687353ec90c0a752d/users/77a636c562fa48a0b14b998008975960/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:46.221350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e2b2b083df40f687353ec90c0a752d, user_id=77a636c562fa48a0b14b998008975960, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:46.224541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:46.237306 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4314/8666] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:46 2022] PUT /identity/v3/projects/88e2b2b083df40f687353ec90c0a752d/users/77a636c562fa48a0b14b998008975960/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:46.248022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:46.248311 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:46.248582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:46.276324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] MFA Rules not processed for user `77a636c562fa48a0b14b998008975960`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:46.277524 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:46.341138 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4353/8667] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:46.353285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:46.355123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:46.415071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3f63d-f55d-4b26-9cb4-d73de8af7c43 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:46.418797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:46.419417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:46.419699 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:46.419976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:46.428839 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:46.430333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:46.439002 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4315/8668] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:48.746667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:48.748052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:48.822141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5839b-b143-4247-aab3-38643e2aeabf None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:48.827147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad06b72-5b47-4567-9736-17143fa74e36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:48.827751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad06b72-5b47-4567-9736-17143fa74e36 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:48.828035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad06b72-5b47-4567-9736-17143fa74e36 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:48.828312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad06b72-5b47-4567-9736-17143fa74e36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:48.830052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fad06b72-5b47-4567-9736-17143fa74e36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:48.844514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad06b72-5b47-4567-9736-17143fa74e36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:48.846068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad06b72-5b47-4567-9736-17143fa74e36 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:48.852447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:48.861289 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4354/8669] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:49.542070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:49.605622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9624b47-c7e0-45d5-8a49-90585c31f5d4 service glance] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:49.610400 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:49.611013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:49.611292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:49.611565 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] PATH_INFO: `/v3/users/77a636c562fa48a0b14b998008975960` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:49.612658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a636c562fa48a0b14b998008975960)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:49.615560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:49.660719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a636c562fa48a0b14b998008975960'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:49.667786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a636c562fa48a0b14b998008975960'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:49.668451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a636c562fa48a0b14b998008975960'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:49.670888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:49.676724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a636c562fa48a0b14b998008975960'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:49.677473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a636c562fa48a0b14b998008975960'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:49.695928 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4316/8670] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:49 2022] DELETE /identity/v3/users/77a636c562fa48a0b14b998008975960 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:49.700572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:49.700884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:49.701188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:49.720349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:49.721622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:49.781297 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4355/8671] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:49.791679 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:49.793121 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:49.855762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb433-6e33-46d1-a44a-c39be6fde630 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:49.859754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5922a5-eb1c-4381-b625-2eff2649d575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:49.860370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5922a5-eb1c-4381-b625-2eff2649d575 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:49.860656 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5922a5-eb1c-4381-b625-2eff2649d575 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:49.860934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5922a5-eb1c-4381-b625-2eff2649d575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:49.876734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5922a5-eb1c-4381-b625-2eff2649d575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:49.878433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5922a5-eb1c-4381-b625-2eff2649d575 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:49.888437 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4317/8672] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:50.157301 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.158813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:50.173101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10328c-2acf-44c1-8752-3c79a5fd4057 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.178211 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.178847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.179132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.179409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] PATH_INFO: `/v3/projects/88e2b2b083df40f687353ec90c0a752d` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.180530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e2b2b083df40f687353ec90c0a752d)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.182481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.231767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e2b2b083df40f687353ec90c0a752d', 'request_id': 'req-f42cf796-5a44-4e4f-9335-0f5def98147d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:50.237514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e2b2b083df40f687353ec90c0a752d', 'request_id': 'req-f42cf796-5a44-4e4f-9335-0f5def98147d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:50.243813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e2b2b083df40f687353ec90c0a752d', 'request_id': 'req-f42cf796-5a44-4e4f-9335-0f5def98147d'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:50.256283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] The token cache is being invalidate because project 88e2b2b083df40f687353ec90c0a752d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:50.256597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:50.257232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-f42cf796-5a44-4e4f-9335-0f5def98147d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:50.258775 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4356/8673] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:50 2022] DELETE /identity/v3/projects/88e2b2b083df40f687353ec90c0a752d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:50.263994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23cec1-5456-4453-b9c7-5009a079bbe5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.264282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23cec1-5456-4453-b9c7-5009a079bbe5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.264560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23cec1-5456-4453-b9c7-5009a079bbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.284996 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5b23cec1-5456-4453-b9c7-5009a079bbe5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:50.286976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5b23cec1-5456-4453-b9c7-5009a079bbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:50.346853 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4318/8674] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:50.355536 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b6042-9a1c-42fe-98f0-e14cba4f1b43 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.355826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b6042-9a1c-42fe-98f0-e14cba4f1b43 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.356105 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b6042-9a1c-42fe-98f0-e14cba4f1b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.377378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6d2b6042-9a1c-42fe-98f0-e14cba4f1b43 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:50.378622 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6d2b6042-9a1c-42fe-98f0-e14cba4f1b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:50.387083 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4357/8675] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:50.391455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23cec1-5456-4453-b9c7-5009a079bbe5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.402209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23cec1-5456-4453-b9c7-5009a079bbe5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.406778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.407364 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.407643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.407913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.408543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:31:50.408842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.411441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.422737 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4319/8676] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:31:50.427491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b6042-9a1c-42fe-98f0-e14cba4f1b43 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.436875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b6042-9a1c-42fe-98f0-e14cba4f1b43 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.442292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.443028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.443314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.443587 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.444212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.446148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.473778 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4358/8677] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:50 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:50.478170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.489250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a006b92-c430-4c93-a881-247dc2378e03 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.544730 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.545371 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.545604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.545913 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.546546 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.548470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.560169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:50.571670 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4320/8678] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:50 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:50.575918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.586732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5bd46-d49e-43b8-87e8-f7074dbe14d1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.591718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.592353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.592682 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.592958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.593593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.595243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.606485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4359/8679] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:50.610827 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.622370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802a12b-2c6a-43bd-9ed7-1e425343c1e0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.627125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee870d-6578-4f91-b728-8cee1e366e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.627756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee870d-6578-4f91-b728-8cee1e366e41 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.628077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee870d-6578-4f91-b728-8cee1e366e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.628391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee870d-6578-4f91-b728-8cee1e366e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.629114 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee870d-6578-4f91-b728-8cee1e366e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.630775 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee870d-6578-4f91-b728-8cee1e366e41 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.641393 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4321/8680] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:50.645768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.655693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48090a12-d2df-4e75-b5d3-bf869e7de989 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.661059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.661712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.662017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.662294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] PATH_INFO: `/v3/projects/0521d39f8e5d4e41af7cb681608c5c6b/users/bfa159d87b8c4351865845cc8778b33c/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.677442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0521d39f8e5d4e41af7cb681608c5c6b, user_id=bfa159d87b8c4351865845cc8778b33c, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.680106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.694798 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4360/8681] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:50 2022] PUT /identity/v3/projects/0521d39f8e5d4e41af7cb681608c5c6b/users/bfa159d87b8c4351865845cc8778b33c/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:50.706169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057a94-249d-43e6-9ba8-a641ffc5ff97 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.706462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057a94-249d-43e6-9ba8-a641ffc5ff97 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.706743 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057a94-249d-43e6-9ba8-a641ffc5ff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.736438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e5057a94-249d-43e6-9ba8-a641ffc5ff97 None None] MFA Rules not processed for user `bfa159d87b8c4351865845cc8778b33c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:50.737639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e5057a94-249d-43e6-9ba8-a641ffc5ff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:50.803143 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4322/8682] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:50.807838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.830056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:50.873732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238e2c9-d6f6-40e9-9142-9e2f0dc14d4c admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.878575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.879785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.880069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.880345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.880965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.882929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:50.912238 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4361/8683] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:31:50 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:31:50.914119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057a94-249d-43e6-9ba8-a641ffc5ff97 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:50.924990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057a94-249d-43e6-9ba8-a641ffc5ff97 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:50.982530 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:50.983148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:50.983430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:50.983997 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:50.984639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:50.986647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:51.009316 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4323/8684] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:31:50 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:31:51.013206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:51.024513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d683c6-1e6f-43a7-8b94-b684feebb4d9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:51.031157 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d44230-f790-440c-97df-c5e7bccfa964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:51.031794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d44230-f790-440c-97df-c5e7bccfa964 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:51.032073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d44230-f790-440c-97df-c5e7bccfa964 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:51.032348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d44230-f790-440c-97df-c5e7bccfa964 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:51.032985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d44230-f790-440c-97df-c5e7bccfa964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:51.034620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d44230-f790-440c-97df-c5e7bccfa964 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:51.045760 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4362/8685] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:31:51.050078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:51.060936 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44c5a5-da7a-4b5c-aa7c-d10ea49f8573 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:51.065453 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:51.066173 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:51.067032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:51.067397 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] PATH_INFO: `/v3/projects/2ec5661e557a41ddb2006df4f56c651f/users/5b7dbab772f64ee09387f5350055e513/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:51.082517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec5661e557a41ddb2006df4f56c651f, user_id=5b7dbab772f64ee09387f5350055e513, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:51.085191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:51.098279 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4324/8686] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:31:51 2022] PUT /identity/v3/projects/2ec5661e557a41ddb2006df4f56c651f/users/5b7dbab772f64ee09387f5350055e513/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:51.108351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800580-9cdc-47b1-a44c-d0b3a91d311f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:51.108640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800580-9cdc-47b1-a44c-d0b3a91d311f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:51.109203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800580-9cdc-47b1-a44c-d0b3a91d311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:51.138664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-09800580-9cdc-47b1-a44c-d0b3a91d311f None None] MFA Rules not processed for user `5b7dbab772f64ee09387f5350055e513`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:51.139877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-09800580-9cdc-47b1-a44c-d0b3a91d311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:51.204375 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4363/8687] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:51 2022] POST /identity/v3/auth/tokens => generated 3267 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:51.234210 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:51.235995 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:51.279595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:51.330586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5409d-563e-4da9-9846-cd02dff911f2 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:51.335746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aaa11-2486-4166-8c6c-651489f3a81a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:51.336393 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aaa11-2486-4166-8c6c-651489f3a81a service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:51.336677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aaa11-2486-4166-8c6c-651489f3a81a service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:51.336957 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aaa11-2486-4166-8c6c-651489f3a81a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:51.345198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aaa11-2486-4166-8c6c-651489f3a81a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:51.346861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aaa11-2486-4166-8c6c-651489f3a81a service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:51.355437 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4325/8688] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 676 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:31:51.905370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09800580-9cdc-47b1-a44c-d0b3a91d311f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:51.968132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09800580-9cdc-47b1-a44c-d0b3a91d311f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:51.974821 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:51.975458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:51.975745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:51.976020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] PATH_INFO: `/v3/users/bfa159d87b8c4351865845cc8778b33c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:51.977141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa159d87b8c4351865845cc8778b33c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:51.979227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:52.008258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:52.027729 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa159d87b8c4351865845cc8778b33c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.040875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa159d87b8c4351865845cc8778b33c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.049049 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa159d87b8c4351865845cc8778b33c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.057339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa159d87b8c4351865845cc8778b33c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.058341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa159d87b8c4351865845cc8778b33c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.064939 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4364/8689] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:51 2022] DELETE /identity/v3/users/bfa159d87b8c4351865845cc8778b33c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:52.070477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:52.070912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:52.071029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:52.091799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:31:52.092987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:52.152494 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4326/8690] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:31:52.162263 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:52.163664 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:52.224835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294d25d-6e44-4db7-b1ca-32e9041f8fbf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:52.228553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:52.229156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:52.229440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:52.229715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:52.239312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:52.240738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:52.250407 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4365/8691] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:31:52.524528 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:52.526169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:52.540639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cb4a9-c9e2-45cf-aae6-469880adaf33 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:52.546081 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:52.546834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:52.546982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:52.547253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] PATH_INFO: `/v3/projects/0521d39f8e5d4e41af7cb681608c5c6b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:52.548718 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0521d39f8e5d4e41af7cb681608c5c6b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:52.550838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:52.603587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0521d39f8e5d4e41af7cb681608c5c6b', 'request_id': 'req-0846ba92-ceb1-40d2-b520-80455a45b8bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.610104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0521d39f8e5d4e41af7cb681608c5c6b', 'request_id': 'req-0846ba92-ceb1-40d2-b520-80455a45b8bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.618639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0521d39f8e5d4e41af7cb681608c5c6b', 'request_id': 'req-0846ba92-ceb1-40d2-b520-80455a45b8bd'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.630245 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] The token cache is being invalidate because project 0521d39f8e5d4e41af7cb681608c5c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:52.630539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.631166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.632633 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4327/8692] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:52 2022] DELETE /identity/v3/projects/0521d39f8e5d4e41af7cb681608c5c6b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:52.636633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:52.646642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c9c7-5545-43a6-b6f7-48622874a2c1 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:52.703800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:52.704347 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:52.704625 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:52.704896 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] PATH_INFO: `/v3/users/5b7dbab772f64ee09387f5350055e513` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:52.706041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7dbab772f64ee09387f5350055e513)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:52.709159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:52.754805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7dbab772f64ee09387f5350055e513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.755929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:52.768569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7dbab772f64ee09387f5350055e513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.775936 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7dbab772f64ee09387f5350055e513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.782489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7dbab772f64ee09387f5350055e513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.783240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-927d27ae-689f-453e-908a-b6c041dce366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7dbab772f64ee09387f5350055e513'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:52.789493 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4366/8693] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:31:52 2022] DELETE /identity/v3/users/5b7dbab772f64ee09387f5350055e513 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:31:53.073452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:31:53.074938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:31:53.088496 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846ba92-ceb1-40d2-b520-80455a45b8bd admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:31:53.120022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:31:53.149039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:31:53.149672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:31:53.149992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:31:53.150268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] PATH_INFO: `/v3/projects/2ec5661e557a41ddb2006df4f56c651f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:31:53.151374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec5661e557a41ddb2006df4f56c651f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:31:53.153271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:31:53.198906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec5661e557a41ddb2006df4f56c651f', 'request_id': 'req-61116d5a-ad86-4c7e-8f33-806f0a464b39'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:53.204965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec5661e557a41ddb2006df4f56c651f', 'request_id': 'req-61116d5a-ad86-4c7e-8f33-806f0a464b39'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:53.213262 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec5661e557a41ddb2006df4f56c651f', 'request_id': 'req-61116d5a-ad86-4c7e-8f33-806f0a464b39'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:53.226243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] The token cache is being invalidate because project 2ec5661e557a41ddb2006df4f56c651f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:31:53.226592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:53.227450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-61116d5a-ad86-4c7e-8f33-806f0a464b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:31:53.230115 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4328/8694] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:31:53 2022] DELETE /identity/v3/projects/2ec5661e557a41ddb2006df4f56c651f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:02.786225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:02.959813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945baeb1-c3f5-4082-a7a9-1cd9b92de08e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:02.960101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945baeb1-c3f5-4082-a7a9-1cd9b92de08e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:02.960378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945baeb1-c3f5-4082-a7a9-1cd9b92de08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:02.989560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-945baeb1-c3f5-4082-a7a9-1cd9b92de08e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:02.990816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-945baeb1-c3f5-4082-a7a9-1cd9b92de08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:03.050031 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4367/8695] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:03.060442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadbeda-91df-4e2f-a810-c34702adc9a4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.060949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadbeda-91df-4e2f-a810-c34702adc9a4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.061064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadbeda-91df-4e2f-a810-c34702adc9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.083137 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-5dadbeda-91df-4e2f-a810-c34702adc9a4 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:03.084356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5dadbeda-91df-4e2f-a810-c34702adc9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:03.092425 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4329/8696] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:03.097294 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945baeb1-c3f5-4082-a7a9-1cd9b92de08e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.107354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945baeb1-c3f5-4082-a7a9-1cd9b92de08e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.112229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.113395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.114043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.114187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.114812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:32:03.115121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.116761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.128469 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4368/8697] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:32:03.145329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadbeda-91df-4e2f-a810-c34702adc9a4 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.156509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadbeda-91df-4e2f-a810-c34702adc9a4 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.162715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.163322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.163644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.163878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.164495 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.166498 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.196197 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4330/8698] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:32:03 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:32:03.200933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.213402 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa907b2b-1f0d-4591-811d-fa3c96106246 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.270261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.270869 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.271153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.271429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.272051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.274005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.294919 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4369/8699] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:32:03 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:32:03.299054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.309582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf21e4-1400-441d-a6c8-ce05c2d42966 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.314328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.314954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.315240 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.315515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.316708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.318322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.328160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4331/8700] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:32:03.332945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.342787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845f227-4612-4afb-9d69-6e769efa02a4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.347670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65608e5-58ec-45a5-a2bd-4178acd279d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.348393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65608e5-58ec-45a5-a2bd-4178acd279d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.348674 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65608e5-58ec-45a5-a2bd-4178acd279d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.348952 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65608e5-58ec-45a5-a2bd-4178acd279d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.349581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65608e5-58ec-45a5-a2bd-4178acd279d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.351256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65608e5-58ec-45a5-a2bd-4178acd279d4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.362053 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4370/8701] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:32:03.366237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.376068 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9161d-1240-4d37-8830-71a683dc939a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.381263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.381965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.382336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.382623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] PATH_INFO: `/v3/projects/474a67ba997844588d0fb15769b8da21/users/6c2d7fb5ad6842a082c7fb21f0be87a9/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.397953 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] RBAC: Authorizing `identity:create_grant(project_id=474a67ba997844588d0fb15769b8da21, user_id=6c2d7fb5ad6842a082c7fb21f0be87a9, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.400593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.413557 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4332/8702] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:32:03 2022] PUT /identity/v3/projects/474a67ba997844588d0fb15769b8da21/users/6c2d7fb5ad6842a082c7fb21f0be87a9/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:03.426569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186a44-cfcc-42c4-a664-07dc9d03efec None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.426857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186a44-cfcc-42c4-a664-07dc9d03efec None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.427159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186a44-cfcc-42c4-a664-07dc9d03efec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.455408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-64186a44-cfcc-42c4-a664-07dc9d03efec None None] MFA Rules not processed for user `6c2d7fb5ad6842a082c7fb21f0be87a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:03.456537 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-64186a44-cfcc-42c4-a664-07dc9d03efec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:03.514746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:03.524732 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4371/8703] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:03.529310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.590729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a946d-39ea-41e9-92cf-78983367f19b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.595432 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.596031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.596313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.596591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.597787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.599711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.604129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:03.628732 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4333/8704] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:32:03 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:32:03.632779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64186a44-cfcc-42c4-a664-07dc9d03efec None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.643732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64186a44-cfcc-42c4-a664-07dc9d03efec None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.701941 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.702550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.702838 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.703116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.703750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.705732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.727960 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4372/8705] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:32:03 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:32:03.732126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.742613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ef423-4741-4d15-878c-0fd47c86dcd1 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.747243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.747831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.748115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.748388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.749019 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.750634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.760950 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4334/8706] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:32:03.765027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.775711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc7160-2e1c-433c-9559-07b44014eb96 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.780284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.780924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.781206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.781480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] PATH_INFO: `/v3/projects/8401746486cd43369ed2cca6a7ac7603/users/5436bb53ff54427e808fcffd2203619c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.797306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8401746486cd43369ed2cca6a7ac7603, user_id=5436bb53ff54427e808fcffd2203619c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.800008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.812341 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4373/8707] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:32:03 2022] PUT /identity/v3/projects/8401746486cd43369ed2cca6a7ac7603/users/5436bb53ff54427e808fcffd2203619c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:03.816870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.827097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860c291-1d1d-491c-bd9d-a136afac6e34 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.883333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732905-c840-46d5-bc35-df9f9c121c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.884077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732905-c840-46d5-bc35-df9f9c121c33 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.884358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732905-c840-46d5-bc35-df9f9c121c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.884632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732905-c840-46d5-bc35-df9f9c121c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.885278 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732905-c840-46d5-bc35-df9f9c121c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.886918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732905-c840-46d5-bc35-df9f9c121c33 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.897248 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4335/8708] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:32:03.901489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:03.912325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc5ccb-172a-47cc-8133-f51ad3dd45b5 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:03.917039 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705b9a9-5d93-4bb6-b285-b5b0cc157970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:03.917685 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705b9a9-5d93-4bb6-b285-b5b0cc157970 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.918008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705b9a9-5d93-4bb6-b285-b5b0cc157970 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.918286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705b9a9-5d93-4bb6-b285-b5b0cc157970 admin admin] PATH_INFO: `/v3/projects/8401746486cd43369ed2cca6a7ac7603/users/5436bb53ff54427e808fcffd2203619c/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.920224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705b9a9-5d93-4bb6-b285-b5b0cc157970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8401746486cd43369ed2cca6a7ac7603, user_id=5436bb53ff54427e808fcffd2203619c, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:03.922932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705b9a9-5d93-4bb6-b285-b5b0cc157970 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:03.932974 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4374/8709] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:32:03 2022] PUT /identity/v3/projects/8401746486cd43369ed2cca6a7ac7603/users/5436bb53ff54427e808fcffd2203619c/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:03.943117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:03.943435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:03.943712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:03.974307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] MFA Rules not processed for user `5436bb53ff54427e808fcffd2203619c`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:03.974307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:04.045783 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4336/8710] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:04.058720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:04.059009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:04.059286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:04.079677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] MFA Rules not processed for user `6c2d7fb5ad6842a082c7fb21f0be87a9`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:04.080894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:04.137332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4375/8711] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:04.150991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:04.152706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:04.208070 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:04.231614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe7396-493c-46b7-bbb0-baeeb9ca8467 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:04.237251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:04.237879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:04.238379 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:04.238691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:04.246388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:04.247969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:04.257365 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4337/8712] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:32:04.729244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:04.730689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:04.786665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:04.812240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976532b6-e706-4b7b-b777-9c93800813a3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:04.817960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:04.818575 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:04.818857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:04.819133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:04.820825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:04.835303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:04.836947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:04.849496 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4376/8713] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:06.774088 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:06.775647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:06.842153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08866b25-72a7-43e2-90f3-9a1560d84270 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:06.846015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:06.846637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:06.846915 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:06.847487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:06.849350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:06.862237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:06.864214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:06.874986 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4338/8714] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:13.861118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:13.876387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb422e6-d8e2-4b5a-9b03-fa1cad1e090c service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:13.882365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218b0ee-d72f-478e-a523-1c47ead15199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:13.882973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218b0ee-d72f-478e-a523-1c47ead15199 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:13.883252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218b0ee-d72f-478e-a523-1c47ead15199 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:13.883529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218b0ee-d72f-478e-a523-1c47ead15199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:13.885238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-7218b0ee-d72f-478e-a523-1c47ead15199 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:13.900676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218b0ee-d72f-478e-a523-1c47ead15199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:13.902290 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218b0ee-d72f-478e-a523-1c47ead15199 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:13.916026 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4377/8715] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:16.596579 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:16.612924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f8fc3-4383-4a9e-8cce-8bfd4cb45d22 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:16.618977 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:16.619567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:16.619842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:16.620116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:16.628097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:16.630459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:16.642355 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4339/8716] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:16.658654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c567425-4ed5-4462-8ba8-6a4be1344f6e None None] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:16.659117 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c567425-4ed5-4462-8ba8-6a4be1344f6e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:16.659215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c567425-4ed5-4462-8ba8-6a4be1344f6e None None] PATH_INFO: `/` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:16.660543 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4378/8717] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:32:16.665644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] Authenticating service token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 21:32:16.680416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:16.689783 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b11bb-fb53-498a-8cfc-b16fce296b7b service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:16.695084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abec25-0a25-4bb2-85ae-648583a2693a tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5436bb53ff54427e808fcffd2203619c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8401746486cd43369ed2cca6a7ac7603', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:16.695702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abec25-0a25-4bb2-85ae-648583a2693a tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:16.695988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abec25-0a25-4bb2-85ae-648583a2693a tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:16.696266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abec25-0a25-4bb2-85ae-648583a2693a tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] PATH_INFO: `/v3/projects/8401746486cd43369ed2cca6a7ac7603` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:16.697714 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abec25-0a25-4bb2-85ae-648583a2693a tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] RBAC: Authorizing `identity:get_project(project_id=8401746486cd43369ed2cca6a7ac7603)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:16.699543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abec25-0a25-4bb2-85ae-648583a2693a tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:16.701173 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4340/8718] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 21:32:16 2022] GET /identity/v3/projects/8401746486cd43369ed2cca6a7ac7603 => generated 391 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:32:16.859435 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c567425-4ed5-4462-8ba8-6a4be1344f6e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:16.873589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c567425-4ed5-4462-8ba8-6a4be1344f6e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:16.879192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:16.880155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:16.880492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:16.880773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:16.882525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:16.894609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:16.896022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:16.906583 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4379/8719] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:17.067032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98b2cc-8d82-4e76-8835-cd62c97dfd94 None None] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:17.067315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98b2cc-8d82-4e76-8835-cd62c97dfd94 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:17.067591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98b2cc-8d82-4e76-8835-cd62c97dfd94 None None] PATH_INFO: `/` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:17.068905 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4341/8720] 10.222.0.33 () {54 vars in 912 bytes} [Sat Sep 10 21:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 21:32:17.073559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] Authenticating service token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 10 21:32:17.074977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:17.094496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:17.104870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67765629-6240-4e1f-941d-1d342889df74 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:17.110128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5436bb53ff54427e808fcffd2203619c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8401746486cd43369ed2cca6a7ac7603', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:17.110750 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:17.111036 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:17.111323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] PATH_INFO: `/v3/projects/8401746486cd43369ed2cca6a7ac7603` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:17.112448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] RBAC: Authorizing `identity:get_project(project_id=8401746486cd43369ed2cca6a7ac7603)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:17.114418 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:17.116039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4380/8721] 10.222.0.33 () {58 vars in 1443 bytes} [Sat Sep 10 21:32:17 2022] GET /identity/v3/projects/8401746486cd43369ed2cca6a7ac7603 => generated 391 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:32:17.270968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98b2cc-8d82-4e76-8835-cd62c97dfd94 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:17.272660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ed98b2cc-8d82-4e76-8835-cd62c97dfd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:17.288527 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98b2cc-8d82-4e76-8835-cd62c97dfd94 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:17.292811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c177cc1-a159-4fcb-895e-91599f063634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:17.293415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177cc1-a159-4fcb-895e-91599f063634 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:17.293671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177cc1-a159-4fcb-895e-91599f063634 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:17.293993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c177cc1-a159-4fcb-895e-91599f063634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:17.295651 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3c177cc1-a159-4fcb-895e-91599f063634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:17.308762 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c177cc1-a159-4fcb-895e-91599f063634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:17.310221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c177cc1-a159-4fcb-895e-91599f063634 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:17.321642 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4342/8722] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:21.113443 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:21.177132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4be29-93f1-4c31-9b9a-e747022a0b12 tempest-ServersAdminNegativeTestJSON-420190226 tempest-ServersAdminNegativeTestJSON-420190226-project-admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:21.181741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:21.182395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:21.182681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:21.182962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] PATH_INFO: `/v3/users/6c2d7fb5ad6842a082c7fb21f0be87a9` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:21.184081 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2d7fb5ad6842a082c7fb21f0be87a9)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:21.186033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:21.233010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2d7fb5ad6842a082c7fb21f0be87a9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.245852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2d7fb5ad6842a082c7fb21f0be87a9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.252803 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2d7fb5ad6842a082c7fb21f0be87a9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.260341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2d7fb5ad6842a082c7fb21f0be87a9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.261134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2d7fb5ad6842a082c7fb21f0be87a9'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.267039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4381/8723] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:32:21 2022] DELETE /identity/v3/users/6c2d7fb5ad6842a082c7fb21f0be87a9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:21.272584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:21.273097 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:21.273206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:21.292465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:21.293696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:21.353722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4343/8724] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:21.364394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:21.365995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:21.396637 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:21.430451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd388-a62f-4dc9-862d-de6ce9e1b251 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:21.434088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:21.434694 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:21.434979 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:21.435255 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:21.444270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:21.445698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:21.455168 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4382/8725] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:21.629365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:21.630982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:21.643566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cea65-b9ee-45c8-85c6-de0ad8a87646 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:21.648963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:21.649608 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:21.649919 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:21.650198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] PATH_INFO: `/v3/projects/474a67ba997844588d0fb15769b8da21` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:21.651325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474a67ba997844588d0fb15769b8da21)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:21.653250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:21.697348 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:21.701105 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474a67ba997844588d0fb15769b8da21', 'request_id': 'req-41770e4a-85bd-416b-837a-a184eda3e696'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.709236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474a67ba997844588d0fb15769b8da21', 'request_id': 'req-41770e4a-85bd-416b-837a-a184eda3e696'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.716577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474a67ba997844588d0fb15769b8da21', 'request_id': 'req-41770e4a-85bd-416b-837a-a184eda3e696'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.727700 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] The token cache is being invalidate because project 474a67ba997844588d0fb15769b8da21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:32:21.727994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.728619 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.730137 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4344/8726] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:32:21 2022] DELETE /identity/v3/projects/474a67ba997844588d0fb15769b8da21 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:21.734477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:21.744222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d59b-f06d-479c-a63b-a199ef6a2947 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:21.800153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:21.800785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:21.801070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:21.801378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] PATH_INFO: `/v3/users/5436bb53ff54427e808fcffd2203619c` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:21.802517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5436bb53ff54427e808fcffd2203619c)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:21.804429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:21.851767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5436bb53ff54427e808fcffd2203619c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.864568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5436bb53ff54427e808fcffd2203619c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.871496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5436bb53ff54427e808fcffd2203619c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.881742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5436bb53ff54427e808fcffd2203619c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.882417 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0c2f0cbd-d6da-4efc-afac-b391030bd2c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5436bb53ff54427e808fcffd2203619c'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:21.888376 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4383/8727] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:32:21 2022] DELETE /identity/v3/users/5436bb53ff54427e808fcffd2203619c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:32:22.169324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.170810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:22.188176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41770e4a-85bd-416b-837a-a184eda3e696 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.247320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.247960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.248243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.248517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] PATH_INFO: `/v3/projects/8401746486cd43369ed2cca6a7ac7603` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.249614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8401746486cd43369ed2cca6a7ac7603)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.251560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.297682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8401746486cd43369ed2cca6a7ac7603', 'request_id': 'req-1af77c8b-f6ad-4f3a-a61e-476562b8d159'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:22.304480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8401746486cd43369ed2cca6a7ac7603', 'request_id': 'req-1af77c8b-f6ad-4f3a-a61e-476562b8d159'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:22.311455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8401746486cd43369ed2cca6a7ac7603', 'request_id': 'req-1af77c8b-f6ad-4f3a-a61e-476562b8d159'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:22.322879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] The token cache is being invalidate because project 8401746486cd43369ed2cca6a7ac7603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:32:22.323109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:22.323737 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1af77c8b-f6ad-4f3a-a61e-476562b8d159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:32:22.325219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4345/8728] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:32:22 2022] DELETE /identity/v3/projects/8401746486cd43369ed2cca6a7ac7603 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:22.331209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879265f5-91ad-47e2-a3c9-c8b1c1cb85c5 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.331662 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879265f5-91ad-47e2-a3c9-c8b1c1cb85c5 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.331781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879265f5-91ad-47e2-a3c9-c8b1c1cb85c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.351409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-879265f5-91ad-47e2-a3c9-c8b1c1cb85c5 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:22.352624 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-879265f5-91ad-47e2-a3c9-c8b1c1cb85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:22.397858 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:22.415177 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4384/8729] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:22.423548 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83b0c-7cf6-4a66-8502-44fd4bab76a9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.423842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83b0c-7cf6-4a66-8502-44fd4bab76a9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.424122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83b0c-7cf6-4a66-8502-44fd4bab76a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.444703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a9f83b0c-7cf6-4a66-8502-44fd4bab76a9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:22.445967 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a9f83b0c-7cf6-4a66-8502-44fd4bab76a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:22.453952 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4346/8730] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:32:22.458381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879265f5-91ad-47e2-a3c9-c8b1c1cb85c5 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.468218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879265f5-91ad-47e2-a3c9-c8b1c1cb85c5 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.472871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.473473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.473759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.474113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.474904 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:32:22.475215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.476820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.488923 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4385/8731] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:32:22.493721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83b0c-7cf6-4a66-8502-44fd4bab76a9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.503036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83b0c-7cf6-4a66-8502-44fd4bab76a9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.507829 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.508422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.508710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.508984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.509621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.511572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.540370 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4347/8732] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:32:22 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:32:22.544299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.554514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442b264-4646-40ac-a550-a2fff945406f admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.612873 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.613487 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.613800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.614088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.614751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.616716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.637419 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4386/8733] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:32:22 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:32:22.641332 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.653113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec4f6-7cba-4a51-9ff2-57495944d702 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.658142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.658740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.659022 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.659630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.660275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.662483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.672370 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4348/8734] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:32:22.676368 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.686718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b2101-607d-45bb-8456-b927306114f3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.691308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cacd1-e253-492d-817d-98a459d7578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.691893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cacd1-e253-492d-817d-98a459d7578e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.692173 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cacd1-e253-492d-817d-98a459d7578e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.692446 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cacd1-e253-492d-817d-98a459d7578e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.693903 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cacd1-e253-492d-817d-98a459d7578e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.695490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cacd1-e253-492d-817d-98a459d7578e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.706597 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4387/8735] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:32:22.711709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.721848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c1892-4fd4-4609-b6f1-81ab84afbf95 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.726557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.727195 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.727477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.727750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] PATH_INFO: `/v3/projects/3671b4249813424fb1afb48dcaacc44d/users/12f7ddc093ef459f97956ed5c7d74449/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.743197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3671b4249813424fb1afb48dcaacc44d, user_id=12f7ddc093ef459f97956ed5c7d74449, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.745877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.753596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:22.760430 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4349/8736] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:32:22 2022] PUT /identity/v3/projects/3671b4249813424fb1afb48dcaacc44d/users/12f7ddc093ef459f97956ed5c7d74449/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:32:22.765800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e61da-fe78-4e0c-9f36-9e808868c0bc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.766113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e61da-fe78-4e0c-9f36-9e808868c0bc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.766387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e61da-fe78-4e0c-9f36-9e808868c0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.785335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8d0e61da-fe78-4e0c-9f36-9e808868c0bc None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:22.787324 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8d0e61da-fe78-4e0c-9f36-9e808868c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:22.847209 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4388/8737] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:22.857848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:22.859802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:22.922561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c3487-a05d-4242-922a-1d82139ac65b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:22.928285 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6260756-7ee3-40f3-8623-854c55a7de5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:22.929036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6260756-7ee3-40f3-8623-854c55a7de5e service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:22.929324 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6260756-7ee3-40f3-8623-854c55a7de5e service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:22.929602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6260756-7ee3-40f3-8623-854c55a7de5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:22.939472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6260756-7ee3-40f3-8623-854c55a7de5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:22.940906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6260756-7ee3-40f3-8623-854c55a7de5e service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:22.950579 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4350/8738] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:27.402155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:27.402620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:27.402875 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:27.433763 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] MFA Rules not processed for user `12f7ddc093ef459f97956ed5c7d74449`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:27.435047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:27.494009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:27.519240 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4389/8739] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:27.531441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e86942-27aa-416d-9318-39bd0fff9f7e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:27.531985 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e86942-27aa-416d-9318-39bd0fff9f7e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:27.532084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e86942-27aa-416d-9318-39bd0fff9f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:27.555012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e5e86942-27aa-416d-9318-39bd0fff9f7e None None] MFA Rules not processed for user `12f7ddc093ef459f97956ed5c7d74449`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:32:27.556297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e5e86942-27aa-416d-9318-39bd0fff9f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:27.563484 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4351/8740] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:32:27.575693 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:27.577034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:27.688604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce20a14-6aac-4ffb-bb8c-a084f865de18 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:27.695350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:27.695980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:27.696264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:27.696540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:27.705322 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:27.706963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:27.717125 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4390/8741] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:37.547522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=30 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:37.712614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=41 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:40.417190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e86942-27aa-416d-9318-39bd0fff9f7e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:40.431148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e86942-27aa-416d-9318-39bd0fff9f7e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:40.434793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:40.435616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:40.435901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:40.436170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:40.438084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:40.451306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:40.452740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:40.464351 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4352/8742] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:40.552172 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:40.553634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:40.626573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f25cd3-5a1d-441c-887c-64f80a5e7cd1 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:40.632589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:40.633180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:40.633463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:40.633739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:40.635508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:40.650484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:40.652099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:40.666866 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4391/8743] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:40.842469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:32:40.843878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:40.857297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622f1f-80fd-41b9-a630-8eb515b20d74 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:32:40.861267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:32:40.861917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:32:40.862203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:32:40.862478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:32:40.864152 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:32:40.882884 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:32:40.884355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:32:40.898572 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4353/8744] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:32:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:32:50.658571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=32 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:32:50.890611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:33:22.519672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:22.555467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd2121-0c22-49a0-80a2-0c0188049d7d service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:22.560073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:22.560695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:22.560974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:22.561247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] PATH_INFO: `/v3/users/12f7ddc093ef459f97956ed5c7d74449` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:22.562555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f7ddc093ef459f97956ed5c7d74449)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:22.564621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:22.614484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f7ddc093ef459f97956ed5c7d74449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:22.626960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f7ddc093ef459f97956ed5c7d74449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:22.634929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f7ddc093ef459f97956ed5c7d74449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:22.644871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f7ddc093ef459f97956ed5c7d74449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:22.645672 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6edc84b3-491b-47f0-b24e-32069b71c2cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f7ddc093ef459f97956ed5c7d74449'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:22.652306 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4392/8745] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:33:22 2022] DELETE /identity/v3/users/12f7ddc093ef459f97956ed5c7d74449 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:33:22.831055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:22.868487 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab85ed-efce-4965-9b85-ebfbe1fc58df service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.083894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.084552 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.084835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.085113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] PATH_INFO: `/v3/projects/3671b4249813424fb1afb48dcaacc44d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.086276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3671b4249813424fb1afb48dcaacc44d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.088163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.134082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3671b4249813424fb1afb48dcaacc44d', 'request_id': 'req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:23.140798 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3671b4249813424fb1afb48dcaacc44d', 'request_id': 'req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:23.149340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3671b4249813424fb1afb48dcaacc44d', 'request_id': 'req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:23.172150 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] The token cache is being invalidate because project 3671b4249813424fb1afb48dcaacc44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:33:23.172440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:23.173061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0c27b62-015d-4ab5-9def-5f9b24d7fc3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:33:23.174551 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4354/8746] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:33:22 2022] DELETE /identity/v3/projects/3671b4249813424fb1afb48dcaacc44d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:33:23.179653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b2d1-bcee-4bd4-aa56-051ba1a7baa9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.179945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b2d1-bcee-4bd4-aa56-051ba1a7baa9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.180222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b2d1-bcee-4bd4-aa56-051ba1a7baa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.231765 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1501b2d1-bcee-4bd4-aa56-051ba1a7baa9 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:33:23.232917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1501b2d1-bcee-4bd4-aa56-051ba1a7baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:33:23.295254 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4393/8747] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:33:23.303824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a17ab9-8001-4d4d-8cca-a11438207014 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.304110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a17ab9-8001-4d4d-8cca-a11438207014 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.304384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a17ab9-8001-4d4d-8cca-a11438207014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.323664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a2a17ab9-8001-4d4d-8cca-a11438207014 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:33:23.324886 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2a17ab9-8001-4d4d-8cca-a11438207014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:33:23.333382 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4355/8748] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:33:23.337510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501b2d1-bcee-4bd4-aa56-051ba1a7baa9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:23.347961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501b2d1-bcee-4bd4-aa56-051ba1a7baa9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.352808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.353411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.353673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.353980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.354614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:33:23.354915 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.356492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.368453 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4394/8749] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:33:23.372692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a17ab9-8001-4d4d-8cca-a11438207014 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:23.382871 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a17ab9-8001-4d4d-8cca-a11438207014 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.387574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.388160 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.388438 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.388706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.389327 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.391236 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.418316 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4356/8750] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:33:23 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:33:23.422605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:23.432887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdedc32-e6e9-4a5b-a8d9-3cdaf83dc8c1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.440553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:33:23.492305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.492861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.493146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.493424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.494106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.496076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.516952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4395/8751] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:33:23 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:33:23.521795 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:23.532063 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520b11f-ad39-46ad-8bfd-dd8e80821ac9 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.536782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.537394 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.537672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.537980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.538606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.540178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.549747 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4357/8752] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:33:23.553905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:23.564061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59987-6f51-4d87-a9ec-7b28a87b995d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.569784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee1fc5-4f7d-470a-8830-3ff7d1b243a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.570426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1fc5-4f7d-470a-8830-3ff7d1b243a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.570712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1fc5-4f7d-470a-8830-3ff7d1b243a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.570994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1fc5-4f7d-470a-8830-3ff7d1b243a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.571631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee1fc5-4f7d-470a-8830-3ff7d1b243a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.573241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee1fc5-4f7d-470a-8830-3ff7d1b243a8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.584384 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4396/8753] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:33:23.588642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:23.599413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925aa2-b2df-4dc5-8450-e977b097b83a admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.604154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.604793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.605077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.605353 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] PATH_INFO: `/v3/projects/3b494efae18f494a8a8b1d6aa53c4327/users/cf26d0546ef846cb845be454c0e4ad76/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.607580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:33:23.623430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b494efae18f494a8a8b1d6aa53c4327, user_id=cf26d0546ef846cb845be454c0e4ad76, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.626420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.638755 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4358/8754] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:33:23 2022] PUT /identity/v3/projects/3b494efae18f494a8a8b1d6aa53c4327/users/cf26d0546ef846cb845be454c0e4ad76/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:33:23.643961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8dda3-256a-4a7b-868c-e6bad0fe988a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.644475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8dda3-256a-4a7b-868c-e6bad0fe988a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.644759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8dda3-256a-4a7b-868c-e6bad0fe988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.664430 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-18b8dda3-256a-4a7b-868c-e6bad0fe988a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:33:23.665617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-18b8dda3-256a-4a7b-868c-e6bad0fe988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:33:23.726606 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4397/8755] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:33:23.736526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:23.739040 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:33:23.826176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af007437-f895-476f-a26e-a580751016a8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:23.829946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac01e5-8e32-4107-a8f7-c58244f08b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:23.830547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac01e5-8e32-4107-a8f7-c58244f08b5a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:23.830832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac01e5-8e32-4107-a8f7-c58244f08b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:23.831109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac01e5-8e32-4107-a8f7-c58244f08b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:23.840860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac01e5-8e32-4107-a8f7-c58244f08b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:23.842338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac01e5-8e32-4107-a8f7-c58244f08b5a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:23.852411 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4359/8756] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:33:28.021981 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:28.022514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:28.022658 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:28.055372 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] MFA Rules not processed for user `cf26d0546ef846cb845be454c0e4ad76`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:33:28.056701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:33:28.125410 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4398/8757] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:33:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:33:28.136365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c353d94-9cd0-4fdc-b0de-cc6c98d95dfb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:28.136877 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c353d94-9cd0-4fdc-b0de-cc6c98d95dfb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:28.137076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c353d94-9cd0-4fdc-b0de-cc6c98d95dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:28.157657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4c353d94-9cd0-4fdc-b0de-cc6c98d95dfb None None] MFA Rules not processed for user `cf26d0546ef846cb845be454c0e4ad76`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:33:28.158897 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4c353d94-9cd0-4fdc-b0de-cc6c98d95dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:33:28.167092 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4360/8758] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:33:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:33:28.180569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:33:28.182480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:33:28.205448 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:33:28.264300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7d7dd-0569-4746-9ec9-8dddb8995b23 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:33:28.270863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:33:28.271485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:33:28.271769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:33:28.272044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:33:28.280033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:33:28.281605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:33:28.290573 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4399/8759] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:33:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:33:38.152411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=48,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:33:38.286568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=25 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:34:21.529693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340718a5-2309-4a20-9219-c4167c2758b3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:34:21.530319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340718a5-2309-4a20-9219-c4167c2758b3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:34:21.530319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340718a5-2309-4a20-9219-c4167c2758b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:34:21.549194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-340718a5-2309-4a20-9219-c4167c2758b3 None None] MFA Rules not processed for user `cf26d0546ef846cb845be454c0e4ad76`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:34:21.550431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-340718a5-2309-4a20-9219-c4167c2758b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:34:21.557258 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4361/8760] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:34:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:34:24.931848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:34:24.933683 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:34:24.953657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1306c-2c22-4e95-9374-7e48843cbbdc service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:34:24.960348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:34:24.961403 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:34:24.961690 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:34:24.962231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:34:24.964016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:34:24.982714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:34:24.992642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:34:25.008480 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4400/8761] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:34:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:34:31.543905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:34:35.001380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:35:17.321136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340718a5-2309-4a20-9219-c4167c2758b3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:17.333118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340718a5-2309-4a20-9219-c4167c2758b3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:17.337779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:17.338462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:17.338742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:17.339016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] PATH_INFO: `/v3/users/cf26d0546ef846cb845be454c0e4ad76` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:17.340169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf26d0546ef846cb845be454c0e4ad76)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:17.342057 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:17.389994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf26d0546ef846cb845be454c0e4ad76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.397422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf26d0546ef846cb845be454c0e4ad76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.398135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf26d0546ef846cb845be454c0e4ad76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.405639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf26d0546ef846cb845be454c0e4ad76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.406632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-344073c8-6a75-4191-b0cc-ef223083e9d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf26d0546ef846cb845be454c0e4ad76'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.423868 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4362/8762] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:35:17 2022] DELETE /identity/v3/users/cf26d0546ef846cb845be454c0e4ad76 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:35:17.599899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:17.613956 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdac6a-ae19-45a8-8e42-80b1fc6c1e14 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:17.672573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:17.673205 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:17.673486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:17.673785 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] PATH_INFO: `/v3/projects/3b494efae18f494a8a8b1d6aa53c4327` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:17.675982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b494efae18f494a8a8b1d6aa53c4327)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:17.677901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:17.725070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b494efae18f494a8a8b1d6aa53c4327', 'request_id': 'req-4986b780-3dc7-4922-92c7-91912c690b94'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.730605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b494efae18f494a8a8b1d6aa53c4327', 'request_id': 'req-4986b780-3dc7-4922-92c7-91912c690b94'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.737620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b494efae18f494a8a8b1d6aa53c4327', 'request_id': 'req-4986b780-3dc7-4922-92c7-91912c690b94'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.750043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] The token cache is being invalidate because project 3b494efae18f494a8a8b1d6aa53c4327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:35:17.750339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.750971 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-4986b780-3dc7-4922-92c7-91912c690b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:35:17.752585 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4401/8763] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:35:17 2022] DELETE /identity/v3/projects/3b494efae18f494a8a8b1d6aa53c4327 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:35:17.757965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035f1ba-8821-4731-acb5-9520d70a71a0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:17.758256 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035f1ba-8821-4731-acb5-9520d70a71a0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:17.758541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035f1ba-8821-4731-acb5-9520d70a71a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:17.778641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4035f1ba-8821-4731-acb5-9520d70a71a0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:35:17.779840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4035f1ba-8821-4731-acb5-9520d70a71a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:35:17.839447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4363/8764] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:35:17.848935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a8b4-c577-4415-9116-b5db8b0a6cfb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:17.849226 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a8b4-c577-4415-9116-b5db8b0a6cfb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:17.849507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a8b4-c577-4415-9116-b5db8b0a6cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:17.870112 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-0a42a8b4-c577-4415-9116-b5db8b0a6cfb None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:35:17.871323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a42a8b4-c577-4415-9116-b5db8b0a6cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:35:17.879370 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4402/8765] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:35:17.883639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035f1ba-8821-4731-acb5-9520d70a71a0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:17.893306 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035f1ba-8821-4731-acb5-9520d70a71a0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:17.898208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:17.898797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:17.899078 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:17.899350 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:17.899982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:35:17.900279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:17.901873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:17.913977 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4364/8766] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 21:35:17.918266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42a8b4-c577-4415-9116-b5db8b0a6cfb None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:17.928438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42a8b4-c577-4415-9116-b5db8b0a6cfb None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:17.933144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:17.933741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:17.934089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:17.934370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:17.935017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:17.936924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:17.967317 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4403/8767] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:35:17 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:35:17.968408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:17.979691 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf09b0-71ae-4ce4-bff0-6a412afa2f9c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:18.035909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:18.036515 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:18.036800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:18.037077 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:18.037706 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:18.039659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:18.047481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:35:18.062096 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4365/8768] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:35:17 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:35:18.067247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:18.079426 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3d6f5-cabe-4ac9-9e14-c9b3f86d1a5b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:18.085411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:18.086051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:18.086342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:18.086616 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:18.087256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:18.088854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:18.099369 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4404/8769] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:35:18.103497 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:18.114680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb939ca-07f1-43d0-a818-6791bd6ae69f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:18.119573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ac74b-778e-4645-b655-8130f37a6bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:18.120172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ac74b-778e-4645-b655-8130f37a6bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:18.120459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ac74b-778e-4645-b655-8130f37a6bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:18.120808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ac74b-778e-4645-b655-8130f37a6bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:18.121375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ac74b-778e-4645-b655-8130f37a6bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:18.124490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ac74b-778e-4645-b655-8130f37a6bb3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:18.136047 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4366/8770] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:35:18.140272 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:18.150906 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa7e98-9f02-475b-b54d-5e4ac1452dc9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:18.155634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:18.156289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:18.156570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:18.156847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] PATH_INFO: `/v3/projects/e767567d02b24a5eb0a01b9ead34449f/users/5e00195042734152b7ab7a71fc7b9e3b/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:18.172230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e767567d02b24a5eb0a01b9ead34449f, user_id=5e00195042734152b7ab7a71fc7b9e3b, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:18.175436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:18.187821 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4405/8771] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:35:18 2022] PUT /identity/v3/projects/e767567d02b24a5eb0a01b9ead34449f/users/5e00195042734152b7ab7a71fc7b9e3b/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:35:18.193477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24189348-4147-4408-a0d0-ffb80f748f7b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:18.193788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24189348-4147-4408-a0d0-ffb80f748f7b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:18.194100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24189348-4147-4408-a0d0-ffb80f748f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:18.213042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-24189348-4147-4408-a0d0-ffb80f748f7b None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:35:18.214312 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24189348-4147-4408-a0d0-ffb80f748f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:35:18.272352 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4367/8772] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:35:18.282477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:18.284384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:35:18.315739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:35:18.347212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ec45a-79dc-4df3-8a69-7545f8a97d1b admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:18.350995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:18.351540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:18.351823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:18.352101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:18.364913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:18.366394 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:18.375942 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4406/8773] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:35:22.113929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb9f33-f0dd-42a3-ab98-a4225327b8c6 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:22.114444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb9f33-f0dd-42a3-ab98-a4225327b8c6 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:22.114734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb9f33-f0dd-42a3-ab98-a4225327b8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:22.149092 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-55bb9f33-f0dd-42a3-ab98-a4225327b8c6 None None] MFA Rules not processed for user `5e00195042734152b7ab7a71fc7b9e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:35:22.150304 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-55bb9f33-f0dd-42a3-ab98-a4225327b8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:35:22.218151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4368/8774] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:35:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:35:22.232980 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:35:22.234744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:35:22.318230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f9092-9bec-4ea6-8990-3bed5f1bc572 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:35:22.323855 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7df34-1ad2-430a-b735-2005ee2eac46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:35:22.324469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7df34-1ad2-430a-b735-2005ee2eac46 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:22.324756 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7df34-1ad2-430a-b735-2005ee2eac46 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:22.325037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7df34-1ad2-430a-b735-2005ee2eac46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:22.333386 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7df34-1ad2-430a-b735-2005ee2eac46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:35:22.335009 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7df34-1ad2-430a-b735-2005ee2eac46 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:35:22.344111 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4407/8775] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:35:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:35:24.051061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ef86-db0a-43ab-9fea-d266d778fdcb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:35:24.051545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ef86-db0a-43ab-9fea-d266d778fdcb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:35:24.051643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ef86-db0a-43ab-9fea-d266d778fdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:35:24.070309 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f968ef86-db0a-43ab-9fea-d266d778fdcb None None] MFA Rules not processed for user `5e00195042734152b7ab7a71fc7b9e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:35:24.071500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f968ef86-db0a-43ab-9fea-d266d778fdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:35:24.078576 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4369/8776] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:35:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:35:32.339928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=48 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:35:34.065082 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:36:15.542011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228608-191b-454a-857e-320772fc95d2 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:15.542601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228608-191b-454a-857e-320772fc95d2 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:15.542601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228608-191b-454a-857e-320772fc95d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:15.590447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d9228608-191b-454a-857e-320772fc95d2 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:36:15.591661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d9228608-191b-454a-857e-320772fc95d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:15.598249 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4408/8777] 10.222.0.33 () {58 vars in 1062 bytes} [Sat Sep 10 21:36:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:36:15.604474 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968ef86-db0a-43ab-9fea-d266d778fdcb None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:36:15.614071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968ef86-db0a-43ab-9fea-d266d778fdcb None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:36:15.617766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:36:15.618455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:15.618733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:15.619000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:15.621594 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:15.638749 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:36:15.640180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:36:15.654435 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4370/8778] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:36:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:36:16.838050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:16.838602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:16.838602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:16.859067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:36:16.860252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:16.867715 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4409/8779] 10.222.0.33 () {58 vars in 1062 bytes} [Sat Sep 10 21:36:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:36:16.873680 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:36:16.882935 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df590f2d-9975-4a02-b09e-89295fbea1e2 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:36:16.886711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:36:16.887299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:16.887601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:16.887860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:16.889566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:16.906253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:36:16.907668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:36:16.921384 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4371/8780] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:36:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:36:26.854053 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=16 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:36:26.913815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:36:27.939306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:36:27.940804 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:28.028320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac525d5-60d5-457a-99b4-4d6047a95887 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:36:28.033515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046e91a-b3ef-4056-92a2-400aab0b9646 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:36:28.034189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e91a-b3ef-4056-92a2-400aab0b9646 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:28.034469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e91a-b3ef-4056-92a2-400aab0b9646 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:28.034741 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e91a-b3ef-4056-92a2-400aab0b9646 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:28.036454 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9046e91a-b3ef-4056-92a2-400aab0b9646 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:28.050730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046e91a-b3ef-4056-92a2-400aab0b9646 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:36:28.052289 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046e91a-b3ef-4056-92a2-400aab0b9646 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:36:28.065641 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4410/8781] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:36:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:36:31.061503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:36:31.071053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ead8a8-bbfe-4bad-995a-0777be5df9ca service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:36:31.074934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac2724-acfd-4885-ad2b-3a7ff4fa5629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:36:31.075800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac2724-acfd-4885-ad2b-3a7ff4fa5629 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:31.076084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac2724-acfd-4885-ad2b-3a7ff4fa5629 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:31.076381 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac2724-acfd-4885-ad2b-3a7ff4fa5629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:31.078168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-85ac2724-acfd-4885-ad2b-3a7ff4fa5629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:31.091583 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac2724-acfd-4885-ad2b-3a7ff4fa5629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:36:31.093025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac2724-acfd-4885-ad2b-3a7ff4fa5629 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:36:31.104088 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4372/8782] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:36:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:36:38.058845 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=37 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:36:41.097556 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:36:43.306018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:43.306018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:43.306018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:43.327074 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] MFA Rules not processed for user `5e00195042734152b7ab7a71fc7b9e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:36:43.328312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:43.335771 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4411/8783] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:36:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:36:43.956943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f080145-b7af-4c3a-82cd-f4d41dfaac4d None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:43.957388 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f080145-b7af-4c3a-82cd-f4d41dfaac4d None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:43.957713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f080145-b7af-4c3a-82cd-f4d41dfaac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:44.004029 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6f080145-b7af-4c3a-82cd-f4d41dfaac4d None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:36:44.005227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6f080145-b7af-4c3a-82cd-f4d41dfaac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:44.014275 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4373/8784] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:36:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:36:44.026531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:36:44.027924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:44.046149 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1ce67-32c6-4693-a9ac-a68f10764ef4 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:36:44.051072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:36:44.051666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:44.051944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:44.052221 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:44.061447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:36:44.063047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:36:44.073163 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4412/8785] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:36:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:36:54.001455 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=9 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:36:54.069229 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:36:55.761921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f080145-b7af-4c3a-82cd-f4d41dfaac4d None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:36:55.775440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f080145-b7af-4c3a-82cd-f4d41dfaac4d None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:36:55.780557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5397f-15dc-4e4e-ab23-2e03ca2d4785 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:36:55.781164 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5397f-15dc-4e4e-ab23-2e03ca2d4785 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:36:55.781450 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5397f-15dc-4e4e-ab23-2e03ca2d4785 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:36:55.781727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5397f-15dc-4e4e-ab23-2e03ca2d4785 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:36:55.783481 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-85c5397f-15dc-4e4e-ab23-2e03ca2d4785 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:36:55.798494 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5397f-15dc-4e4e-ab23-2e03ca2d4785 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:36:55.800183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5397f-15dc-4e4e-ab23-2e03ca2d4785 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:36:55.813433 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4374/8786] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:36:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:37:05.806444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:38:09.618619 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:09.629942 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff69e9-1a67-4454-8822-20a68db352d1 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:09.634567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:09.635191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:09.635480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:09.635759 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:09.637520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:09.651016 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:09.652515 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:09.663878 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4413/8787] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:38:09.756582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27dba4-c1ea-49c4-9b36-1f1d1b24f519 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:09.757194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27dba4-c1ea-49c4-9b36-1f1d1b24f519 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:09.757448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27dba4-c1ea-49c4-9b36-1f1d1b24f519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:09.777799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4e27dba4-c1ea-49c4-9b36-1f1d1b24f519 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:38:09.779032 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4e27dba4-c1ea-49c4-9b36-1f1d1b24f519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:09.787586 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4375/8788] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:38:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:38:09.799013 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:09.813601 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46f821-26da-41ea-97b4-8b56796f0bb1 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:09.819293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b670375-50c8-414d-a088-94ffcb6a3a45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:09.819887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670375-50c8-414d-a088-94ffcb6a3a45 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:09.820171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670375-50c8-414d-a088-94ffcb6a3a45 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:09.820502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670375-50c8-414d-a088-94ffcb6a3a45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:09.828725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b670375-50c8-414d-a088-94ffcb6a3a45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:09.830335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b670375-50c8-414d-a088-94ffcb6a3a45 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:09.840901 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4414/8789] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:38:10.019774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27dba4-c1ea-49c4-9b36-1f1d1b24f519 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:10.029437 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27dba4-c1ea-49c4-9b36-1f1d1b24f519 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:10.032971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:10.033561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:10.033918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:10.034196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:10.035908 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:10.051851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:10.053782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:10.067371 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4376/8790] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:38:10.125712 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389ebce-c38e-4c23-b17d-c0eaade68c0e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:10.126026 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389ebce-c38e-4c23-b17d-c0eaade68c0e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:10.126305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389ebce-c38e-4c23-b17d-c0eaade68c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:10.146355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4389ebce-c38e-4c23-b17d-c0eaade68c0e None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:38:10.147552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4389ebce-c38e-4c23-b17d-c0eaade68c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:10.156050 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4415/8791] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:38:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:38:10.167942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:10.182187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc62bb-4f6d-4ecb-9c9e-87b21ffa613b service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:10.187176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:10.187765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:10.188044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:10.188314 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:10.197391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:10.199003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:10.208467 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4377/8792] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:38:13.794160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389ebce-c38e-4c23-b17d-c0eaade68c0e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:13.804122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389ebce-c38e-4c23-b17d-c0eaade68c0e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:13.807737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:13.808331 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:13.808618 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:13.808894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:13.810642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:13.822640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:13.824084 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:13.835204 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4416/8793] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:38:14.240275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:14.256058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b7aad-c95e-4e72-b499-7b1fec09f6e2 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:14.262586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2e84d-2b2a-4db4-8a4c-3a59f13639f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:14.263194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2e84d-2b2a-4db4-8a4c-3a59f13639f8 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:14.263482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2e84d-2b2a-4db4-8a4c-3a59f13639f8 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:14.263776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2e84d-2b2a-4db4-8a4c-3a59f13639f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:14.266386 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-7eb2e84d-2b2a-4db4-8a4c-3a59f13639f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:14.297320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2e84d-2b2a-4db4-8a4c-3a59f13639f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:14.300336 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2e84d-2b2a-4db4-8a4c-3a59f13639f8 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:14.320243 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4378/8794] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:38:19.657734 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:19.659947 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:19.701120 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4da6b5-5fac-4d38-af6b-3eaa6f966d12 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:19.707389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:19.708008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:19.708293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:19.708569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:19.711189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:19.726886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:19.728862 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:19.747000 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4417/8795] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:38:24.311425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=21 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:38:27.339655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b02bb-808f-4fb2-ac0e-a5a6a92bf44e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:27.340220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b02bb-808f-4fb2-ac0e-a5a6a92bf44e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:27.340220 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b02bb-808f-4fb2-ac0e-a5a6a92bf44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:27.360754 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e49b02bb-808f-4fb2-ac0e-a5a6a92bf44e None None] MFA Rules not processed for user `5e00195042734152b7ab7a71fc7b9e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:38:27.361969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e49b02bb-808f-4fb2-ac0e-a5a6a92bf44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:27.368983 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4379/8796] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:38:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:38:27.680558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:38:27.690150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d7536-ceac-40b5-8136-b963359a6fb9 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:38:27.693623 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d33b6-8af3-4d01-a5e2-89ed90fc5a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:38:27.694235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d33b6-8af3-4d01-a5e2-89ed90fc5a74 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:38:27.694541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d33b6-8af3-4d01-a5e2-89ed90fc5a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:38:27.695048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d33b6-8af3-4d01-a5e2-89ed90fc5a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:38:27.696784 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-759d33b6-8af3-4d01-a5e2-89ed90fc5a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:38:27.709015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d33b6-8af3-4d01-a5e2-89ed90fc5a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:38:27.710865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d33b6-8af3-4d01-a5e2-89ed90fc5a74 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:38:27.721324 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4418/8797] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:38:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:38:37.355566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:38:37.715296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=33 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:40:05.688049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b02bb-808f-4fb2-ac0e-a5a6a92bf44e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:40:05.705916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b02bb-808f-4fb2-ac0e-a5a6a92bf44e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:40:05.711621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:40:05.712237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:40:05.712523 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:40:05.712852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:40:05.715471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:40:05.730964 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:40:05.732570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:40:05.747160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4380/8798] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:40:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:40:14.243921 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c945c4-8832-47b3-ade4-835b0235a2c0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:40:14.244385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c945c4-8832-47b3-ade4-835b0235a2c0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:40:14.244495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c945c4-8832-47b3-ade4-835b0235a2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:40:14.265423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-62c945c4-8832-47b3-ade4-835b0235a2c0 None None] MFA Rules not processed for user `5e00195042734152b7ab7a71fc7b9e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:40:14.266698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-62c945c4-8832-47b3-ade4-835b0235a2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:40:14.273714 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4419/8799] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:40:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:40:14.593468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:40:14.602501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b0621-9b98-4dde-b88b-f150428abd69 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:40:14.606061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:40:14.606648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:40:14.606928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:40:14.607197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:40:14.608878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:40:14.621375 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:40:14.622808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:40:14.634061 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4381/8800] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:40:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:40:22.899571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c945c4-8832-47b3-ade4-835b0235a2c0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:40:22.915138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c945c4-8832-47b3-ade4-835b0235a2c0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:40:22.921154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:40:22.921754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:40:22.922070 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:40:22.922349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:40:22.930125 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:40:22.931707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:40:22.941364 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4420/8801] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:40:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:40:24.628007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:40:32.937225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:11.050514 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:11.069087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5d4fd-855c-47c4-80e1-5947c6711fe3 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:11.072744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:11.073340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:11.073622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:11.073928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:11.082854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:11.084270 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:11.093990 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4382/8802] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:41:14.243444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:14.259462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8ed4e-19bf-4d38-8959-a178ee0de820 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:14.265116 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:14.265730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:14.266079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:14.266355 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:14.268066 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:14.285464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:14.287842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:14.311070 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4421/8803] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:14.323644 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:14.336239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50f15-1399-4c64-b70e-17c58184ad36 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:14.341702 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:14.342591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:14.343089 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:14.343708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] PATH_INFO: `/v3/users/5e00195042734152b7ab7a71fc7b9e3b` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:14.350641 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e00195042734152b7ab7a71fc7b9e3b)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:14.352696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:14.408025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e00195042734152b7ab7a71fc7b9e3b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.414971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e00195042734152b7ab7a71fc7b9e3b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.415643 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e00195042734152b7ab7a71fc7b9e3b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.423041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e00195042734152b7ab7a71fc7b9e3b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.423685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-796f98b4-34a9-43ab-9703-72eb99837f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e00195042734152b7ab7a71fc7b9e3b'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.441436 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4383/8804] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:41:14 2022] DELETE /identity/v3/users/5e00195042734152b7ab7a71fc7b9e3b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:41:14.616950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:14.628645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72a766-b8e6-48a2-970a-e1c98eb76e45 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:14.691568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:14.692238 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:14.692525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:14.692806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] PATH_INFO: `/v3/projects/e767567d02b24a5eb0a01b9ead34449f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:14.693963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e767567d02b24a5eb0a01b9ead34449f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:14.695949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:14.744614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e767567d02b24a5eb0a01b9ead34449f', 'request_id': 'req-97c3ca99-4ebd-4229-a455-61e8bf415419'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.750212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e767567d02b24a5eb0a01b9ead34449f', 'request_id': 'req-97c3ca99-4ebd-4229-a455-61e8bf415419'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.756346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e767567d02b24a5eb0a01b9ead34449f', 'request_id': 'req-97c3ca99-4ebd-4229-a455-61e8bf415419'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.769591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] The token cache is being invalidate because project e767567d02b24a5eb0a01b9ead34449f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:41:14.769919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.770555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-97c3ca99-4ebd-4229-a455-61e8bf415419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:14.772085 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4422/8805] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:41:14 2022] DELETE /identity/v3/projects/e767567d02b24a5eb0a01b9ead34449f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:41:14.777573 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190c97-109d-479f-92a7-a3089d1de30e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:14.777883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190c97-109d-479f-92a7-a3089d1de30e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:14.778156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190c97-109d-479f-92a7-a3089d1de30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:14.798384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-d4190c97-109d-479f-92a7-a3089d1de30e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:14.799574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d4190c97-109d-479f-92a7-a3089d1de30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:14.858651 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4384/8806] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:14.866692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba29bc-f7e2-47bc-a1b1-e7ce0ab23514 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:14.867015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba29bc-f7e2-47bc-a1b1-e7ce0ab23514 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:14.867293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba29bc-f7e2-47bc-a1b1-e7ce0ab23514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:14.888003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f9ba29bc-f7e2-47bc-a1b1-e7ce0ab23514 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:14.889208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f9ba29bc-f7e2-47bc-a1b1-e7ce0ab23514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:14.898083 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4423/8807] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:14.903228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190c97-109d-479f-92a7-a3089d1de30e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:14.918177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190c97-109d-479f-92a7-a3089d1de30e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:14.922794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:14.923395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:14.923675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:14.923945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:14.924592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:41:14.924889 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:14.926560 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:14.938652 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4385/8808] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:41:14.943236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba29bc-f7e2-47bc-a1b1-e7ce0ab23514 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:14.955608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba29bc-f7e2-47bc-a1b1-e7ce0ab23514 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:14.960608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:14.961218 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:14.961499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:14.961773 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:14.962494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:14.965044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:14.998171 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4424/8809] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:41:14 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:41:14.999207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:15.009553 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec13d1d-1317-4483-b8fe-1a76ed738085 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:15.044237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:15.069637 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:15.070276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:15.070555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:15.070826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:15.071451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:15.073682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:15.095219 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4386/8810] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:41:14 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:41:15.099491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:15.111516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef82f8-3b4c-46eb-accf-3b7148250a45 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:15.116479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:15.117079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:15.117361 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:15.117638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:15.118312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:15.120280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:15.130347 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4425/8811] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:41:15.134541 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:15.144614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f254c8-402e-4270-8310-8c86dd745a1d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:15.149288 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80726b-7843-4392-b4e5-4bcc15f070d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:15.149916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80726b-7843-4392-b4e5-4bcc15f070d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:15.150202 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80726b-7843-4392-b4e5-4bcc15f070d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:15.150477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80726b-7843-4392-b4e5-4bcc15f070d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:15.151721 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80726b-7843-4392-b4e5-4bcc15f070d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:15.153307 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80726b-7843-4392-b4e5-4bcc15f070d3 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:15.163263 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4387/8812] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:41:15.168145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:15.178408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89255e34-3779-4fd6-b4a3-28d212fe14ab admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:15.183359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:15.184001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:15.184281 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:15.184559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] PATH_INFO: `/v3/projects/9d69ccced5e14a43b40de853fcae7ae3/users/58891ebd588c439a8c02f7e26ac4b6ee/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:15.199332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d69ccced5e14a43b40de853fcae7ae3, user_id=58891ebd588c439a8c02f7e26ac4b6ee, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:15.202002 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:15.215145 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4426/8813] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:41:15 2022] PUT /identity/v3/projects/9d69ccced5e14a43b40de853fcae7ae3/users/58891ebd588c439a8c02f7e26ac4b6ee/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:41:15.220231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdba43-f9ed-4d08-a8f6-1fa4d9dc3e2a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:15.220542 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdba43-f9ed-4d08-a8f6-1fa4d9dc3e2a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:15.220819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdba43-f9ed-4d08-a8f6-1fa4d9dc3e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:15.240943 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-c2cdba43-f9ed-4d08-a8f6-1fa4d9dc3e2a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:15.242209 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c2cdba43-f9ed-4d08-a8f6-1fa4d9dc3e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:15.301129 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4388/8814] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:15.311032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:15.312820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:15.327366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:15.380178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454bd6a-d2d8-4019-bd0c-db6aed3511d1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:15.383886 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea616e7a-5c2c-46c7-8022-9a31581a1fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:15.384484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616e7a-5c2c-46c7-8022-9a31581a1fab service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:15.384767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616e7a-5c2c-46c7-8022-9a31581a1fab service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:15.385612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616e7a-5c2c-46c7-8022-9a31581a1fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:15.394987 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea616e7a-5c2c-46c7-8022-9a31581a1fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:15.396404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea616e7a-5c2c-46c7-8022-9a31581a1fab service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:15.406878 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4427/8815] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:19.637425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:19.638023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:19.638023 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:19.667130 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] MFA Rules not processed for user `58891ebd588c439a8c02f7e26ac4b6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:19.668533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:19.736161 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4389/8816] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:19.746588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4bcd-0489-4a77-9d74-2d26cb2525f4 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:19.747075 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4bcd-0489-4a77-9d74-2d26cb2525f4 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:19.747193 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4bcd-0489-4a77-9d74-2d26cb2525f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:19.767985 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1c5b4bcd-0489-4a77-9d74-2d26cb2525f4 None None] MFA Rules not processed for user `58891ebd588c439a8c02f7e26ac4b6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:19.769195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1c5b4bcd-0489-4a77-9d74-2d26cb2525f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:19.776378 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4428/8817] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:41:19.790320 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:19.792395 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:19.848446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:19.903675 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e2cee-c4b9-4ac6-8562-54806f26163f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:19.909185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:19.909785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:19.910094 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:19.910365 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:19.918065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:19.919648 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:19.928390 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4390/8818] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:19.960062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3421b6-aa32-4905-9087-f48ff61fc53f None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:19.960353 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3421b6-aa32-4905-9087-f48ff61fc53f None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:19.960636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3421b6-aa32-4905-9087-f48ff61fc53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:20.003778 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ae3421b6-aa32-4905-9087-f48ff61fc53f None None] MFA Rules not processed for user `5cdff5f5500040439f4acafa6c9ad948`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:20.004966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ae3421b6-aa32-4905-9087-f48ff61fc53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:20.059434 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4429/8819] 10.222.0.33 () {58 vars in 1063 bytes} [Sat Sep 10 21:41:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:20.065639 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:20.075372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ae318-0c54-441f-927f-148a0057d360 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:20.079028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a919d7-d107-4244-a82b-06956c462213 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:20.079640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a919d7-d107-4244-a82b-06956c462213 service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:20.079918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a919d7-d107-4244-a82b-06956c462213 service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:20.080189 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a919d7-d107-4244-a82b-06956c462213 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:20.081918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-45a919d7-d107-4244-a82b-06956c462213 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:20.098411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a919d7-d107-4244-a82b-06956c462213 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:20.099820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a919d7-d107-4244-a82b-06956c462213 service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:20.113517 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4391/8820] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:20.683695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3421b6-aa32-4905-9087-f48ff61fc53f None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:20.693384 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3421b6-aa32-4905-9087-f48ff61fc53f None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:20.697099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:20.698564 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:20.698973 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:20.699313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:20.701267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:20.719477 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:20.721148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:20.749086 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4430/8821] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:30.105820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=31 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:30.301146 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:30.301522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:30.301802 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:30.321605 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] MFA Rules not processed for user `58891ebd588c439a8c02f7e26ac4b6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:30.322870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:30.330190 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4392/8822] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:30.728243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=58 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:32.974900 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:32.977268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:32.989474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11951ca7-8946-4dee-a3a5-004bb88a2690 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:32.993346 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:32.994700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:32.995056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:32.995525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:32.997283 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:33.010597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:33.012185 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:33.023956 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4431/8823] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:36.882315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:36.883717 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:36.964740 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08317444-85b2-4774-aac4-b8e2ef8b6085 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:36.969801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:36.970448 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:36.970729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:36.971002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:36.972739 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:36.987183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:36.988766 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:37.001681 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4393/8824] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:43.018335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:46.994945 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=39 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:54.492772 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:54.522736 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b436c-4d44-49c6-ac96-dca9ffe7b03b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:54.527970 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:54.528596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:54.528878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:54.529155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] PATH_INFO: `/v3/users/58891ebd588c439a8c02f7e26ac4b6ee` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:54.530330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58891ebd588c439a8c02f7e26ac4b6ee)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:54.532243 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:54.578155 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58891ebd588c439a8c02f7e26ac4b6ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.590897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58891ebd588c439a8c02f7e26ac4b6ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.597743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58891ebd588c439a8c02f7e26ac4b6ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.605234 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58891ebd588c439a8c02f7e26ac4b6ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.605959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-0f10b857-ec56-476d-a345-e1644b9cc759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58891ebd588c439a8c02f7e26ac4b6ee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.612274 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4432/8825] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:41:54 2022] DELETE /identity/v3/users/58891ebd588c439a8c02f7e26ac4b6ee => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:41:54.785980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:54.797914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf46f3a-1b50-4635-9e78-a7a71abe02cc service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:54.855411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:54.856039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:54.856325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:54.856603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] PATH_INFO: `/v3/projects/9d69ccced5e14a43b40de853fcae7ae3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:54.857712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d69ccced5e14a43b40de853fcae7ae3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:54.859968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:54.906543 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d69ccced5e14a43b40de853fcae7ae3', 'request_id': 'req-12f291f5-fe02-4d94-aed7-9d6880c83c0e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.912712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d69ccced5e14a43b40de853fcae7ae3', 'request_id': 'req-12f291f5-fe02-4d94-aed7-9d6880c83c0e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.921425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d69ccced5e14a43b40de853fcae7ae3', 'request_id': 'req-12f291f5-fe02-4d94-aed7-9d6880c83c0e'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.933577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] The token cache is being invalidate because project 9d69ccced5e14a43b40de853fcae7ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:41:54.933937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.934752 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-12f291f5-fe02-4d94-aed7-9d6880c83c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:41:54.936474 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4394/8826] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:41:54 2022] DELETE /identity/v3/projects/9d69ccced5e14a43b40de853fcae7ae3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:41:54.941457 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5254-2469-4baf-90c8-eef742e4a680 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:54.941747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5254-2469-4baf-90c8-eef742e4a680 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:54.942058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd5254-2469-4baf-90c8-eef742e4a680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:54.987342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a9bd5254-2469-4baf-90c8-eef742e4a680 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:54.988568 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a9bd5254-2469-4baf-90c8-eef742e4a680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:55.051773 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4433/8827] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:55.060598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a083d0-f33c-4a9a-a551-20b96b3c7c61 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.060882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a083d0-f33c-4a9a-a551-20b96b3c7c61 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.061183 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a083d0-f33c-4a9a-a551-20b96b3c7c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.081786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-32a083d0-f33c-4a9a-a551-20b96b3c7c61 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:55.083017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32a083d0-f33c-4a9a-a551-20b96b3c7c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:55.091133 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4395/8828] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:55.095357 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd5254-2469-4baf-90c8-eef742e4a680 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:55.105626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd5254-2469-4baf-90c8-eef742e4a680 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:55.110440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:55.111032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.111314 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.111589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.112235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:41:55.112539 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:55.114187 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:55.126375 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4434/8829] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:41:55.130585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a083d0-f33c-4a9a-a551-20b96b3c7c61 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:55.139825 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a083d0-f33c-4a9a-a551-20b96b3c7c61 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:55.144472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:55.145060 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.145346 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.145622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.146345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:55.148225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:55.184576 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4396/8830] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:41:55 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:41:55.188606 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:55.199835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65085a04-14e8-49a2-8fd8-2a7d51a9c159 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:55.232256 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:55.258811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:55.260264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.260546 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.260817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.261429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:55.263374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:55.285239 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4435/8831] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:41:55 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:41:55.289291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:55.299696 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02354985-54fe-4dc0-bfde-0045b75e554b admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:55.304268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:55.304860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.305143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.305418 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.306190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:55.307788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:55.317458 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4397/8832] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:41:55.321928 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:55.332067 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e060f-c8f8-4288-a617-e4403933938d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:55.336699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41460097-5ae1-4031-ad83-0bce859e956c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:55.338012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41460097-5ae1-4031-ad83-0bce859e956c admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.338302 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41460097-5ae1-4031-ad83-0bce859e956c admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.338609 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41460097-5ae1-4031-ad83-0bce859e956c admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.339530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41460097-5ae1-4031-ad83-0bce859e956c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:55.341481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41460097-5ae1-4031-ad83-0bce859e956c admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:55.351416 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4436/8833] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:41:55.355587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:55.366142 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a970c-fbf2-4583-8e61-1831b5689263 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:55.370869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:55.371507 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.371801 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.372055 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] PATH_INFO: `/v3/projects/dea6ed1c990d4646a0e2ede3c744e955/users/f2034afea0fe42c998ee3dbc48a1b67f/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.386989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea6ed1c990d4646a0e2ede3c744e955, user_id=f2034afea0fe42c998ee3dbc48a1b67f, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:55.389606 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:55.402400 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4398/8834] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:41:55 2022] PUT /identity/v3/projects/dea6ed1c990d4646a0e2ede3c744e955/users/f2034afea0fe42c998ee3dbc48a1b67f/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:41:55.412467 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.412781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.413064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.442578 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] MFA Rules not processed for user `f2034afea0fe42c998ee3dbc48a1b67f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:41:55.443723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:55.509686 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4437/8835] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:41:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:41:55.519948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:55.522241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:55.543954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:41:55.582213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3ae48-1991-4a0e-8677-df55b4417983 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:55.585917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:55.586522 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:55.586807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:55.587623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:55.596000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:55.597431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:55.606875 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4399/8836] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:41:59.751552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:41:59.753798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:59.842559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e54b36-a433-40cb-8ce5-6972d7f8e670 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:41:59.848237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:41:59.848962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:41:59.849406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:41:59.849746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:41:59.854232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:41:59.891645 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:41:59.893391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:41:59.907558 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4438/8837] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:41:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:42:01.230430 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:01.250994 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610b45a-82cc-4aff-ae88-8a95520368bf service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:01.264274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc8712-8519-4687-8616-8afbea7b0988 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:01.265219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc8712-8519-4687-8616-8afbea7b0988 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:01.265578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc8712-8519-4687-8616-8afbea7b0988 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:01.267504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc8712-8519-4687-8616-8afbea7b0988 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:01.274519 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-07dc8712-8519-4687-8616-8afbea7b0988 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:01.298073 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc8712-8519-4687-8616-8afbea7b0988 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:01.302303 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc8712-8519-4687-8616-8afbea7b0988 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:01.318178 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4400/8838] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:42:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:42:08.823378 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:08.859848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:08.889726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f62a33-f010-4706-bc63-aab3da3f10df service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:08.894516 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:08.895148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:08.895431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:08.895708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] PATH_INFO: `/v3/users/f2034afea0fe42c998ee3dbc48a1b67f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:08.896829 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2034afea0fe42c998ee3dbc48a1b67f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:08.898782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:08.947798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2034afea0fe42c998ee3dbc48a1b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:08.960486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2034afea0fe42c998ee3dbc48a1b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:08.969004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2034afea0fe42c998ee3dbc48a1b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:08.983737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2034afea0fe42c998ee3dbc48a1b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:08.984370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2034afea0fe42c998ee3dbc48a1b67f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:08.991354 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4439/8839] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:42:08 2022] DELETE /identity/v3/users/f2034afea0fe42c998ee3dbc48a1b67f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:08.996682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:08.997190 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:08.997470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.017197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:09.018425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:09.078774 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4401/8840] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:09.088626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.090077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:09.147963 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91207b-fe59-4060-9b92-0cd68dbccc50 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.152950 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531eefd-e072-477b-aa12-fd39dc77ae07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.153550 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531eefd-e072-477b-aa12-fd39dc77ae07 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.153850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531eefd-e072-477b-aa12-fd39dc77ae07 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.154151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531eefd-e072-477b-aa12-fd39dc77ae07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.163099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531eefd-e072-477b-aa12-fd39dc77ae07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.164507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531eefd-e072-477b-aa12-fd39dc77ae07 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.173869 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4440/8841] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:42:09.347794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.349281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:09.364367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de491f09-1256-427f-9a6b-47290fa4f5f3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.369665 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.370328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.370611 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.370882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] PATH_INFO: `/v3/projects/dea6ed1c990d4646a0e2ede3c744e955` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.371993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea6ed1c990d4646a0e2ede3c744e955)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.375002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.421905 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea6ed1c990d4646a0e2ede3c744e955', 'request_id': 'req-b1da4d27-8c28-4562-8c4d-c8c35285117d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:09.428229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea6ed1c990d4646a0e2ede3c744e955', 'request_id': 'req-b1da4d27-8c28-4562-8c4d-c8c35285117d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:09.435828 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea6ed1c990d4646a0e2ede3c744e955', 'request_id': 'req-b1da4d27-8c28-4562-8c4d-c8c35285117d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:09.446906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] The token cache is being invalidate because project dea6ed1c990d4646a0e2ede3c744e955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:42:09.447293 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:09.447968 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b1da4d27-8c28-4562-8c4d-c8c35285117d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:09.449447 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4402/8842] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:42:09 2022] DELETE /identity/v3/projects/dea6ed1c990d4646a0e2ede3c744e955 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:09.454711 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f490-5d79-4622-9f1a-f4ce6e8dcd76 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.455279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f490-5d79-4622-9f1a-f4ce6e8dcd76 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.455563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298f490-5d79-4622-9f1a-f4ce6e8dcd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.480111 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c298f490-5d79-4622-9f1a-f4ce6e8dcd76 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:09.481579 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c298f490-5d79-4622-9f1a-f4ce6e8dcd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:09.532177 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:09.547081 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4441/8843] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:09.555705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501869ce-9b57-4a91-8bba-f0cb1e17947e None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.556008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501869ce-9b57-4a91-8bba-f0cb1e17947e None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.556532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501869ce-9b57-4a91-8bba-f0cb1e17947e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.576869 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-501869ce-9b57-4a91-8bba-f0cb1e17947e None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:09.578129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-501869ce-9b57-4a91-8bba-f0cb1e17947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:09.585928 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4403/8844] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:42:09.590798 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298f490-5d79-4622-9f1a-f4ce6e8dcd76 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.603655 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298f490-5d79-4622-9f1a-f4ce6e8dcd76 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.608259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.608857 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.609137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.609411 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.610425 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:42:09.610727 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.612306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.623988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4442/8845] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:42:09.628340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501869ce-9b57-4a91-8bba-f0cb1e17947e None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.638399 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501869ce-9b57-4a91-8bba-f0cb1e17947e None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.643011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.643599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.643878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.644154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.644769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.646729 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.674906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4404/8846] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:42:09 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:42:09.679086 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.689463 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a67b4-e0ff-40ec-b6cc-2ac5c573f0db admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.746848 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.747432 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.747716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.747994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.748633 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.750592 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.771765 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4443/8847] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:42:09 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:42:09.776243 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.781736 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:09.788033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc434bd0-8ea5-4b24-8d05-d321bde173ac admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.793732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.794374 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.794655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.794925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.795555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.797127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.807793 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4405/8848] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:42:09.811809 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.822916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a8c12-b3a0-4d85-bd06-13f560752f43 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.827620 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760f435-a67b-4ba8-bc10-03087f36dd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.828201 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760f435-a67b-4ba8-bc10-03087f36dd1a admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.828486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760f435-a67b-4ba8-bc10-03087f36dd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.828764 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760f435-a67b-4ba8-bc10-03087f36dd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.829404 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760f435-a67b-4ba8-bc10-03087f36dd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.831071 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760f435-a67b-4ba8-bc10-03087f36dd1a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.842074 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4444/8849] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:42:09.846176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:09.856251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9c6d0-e469-4c45-88a1-3035d95ae0de admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:09.861075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:09.861711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.862027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.862305 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] PATH_INFO: `/v3/projects/ac4fad0d8d164eefa7973417477d1f7e/users/d190e16ba888408cbf2d918df8ca0e6e/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.877249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4fad0d8d164eefa7973417477d1f7e, user_id=d190e16ba888408cbf2d918df8ca0e6e, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:09.879907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:09.892550 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4406/8850] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:42:09 2022] PUT /identity/v3/projects/ac4fad0d8d164eefa7973417477d1f7e/users/d190e16ba888408cbf2d918df8ca0e6e/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:09.904062 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fac1-729e-4668-9d4c-2ee2af965443 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:09.904379 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fac1-729e-4668-9d4c-2ee2af965443 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:09.904667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268fac1-729e-4668-9d4c-2ee2af965443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:09.932966 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4268fac1-729e-4668-9d4c-2ee2af965443 None None] MFA Rules not processed for user `d190e16ba888408cbf2d918df8ca0e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:09.934225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4268fac1-729e-4668-9d4c-2ee2af965443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:10.000876 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4445/8851] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:10.011013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:10.012751 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:10.069692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9178-ec71-4cb6-be0a-b532226c7a34 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:10.073624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aff7d-dccc-4b52-866f-50d83e08768c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:10.074267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aff7d-dccc-4b52-866f-50d83e08768c service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:10.074555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aff7d-dccc-4b52-866f-50d83e08768c service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:10.074832 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aff7d-dccc-4b52-866f-50d83e08768c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:10.083227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519aff7d-dccc-4b52-866f-50d83e08768c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:10.084657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519aff7d-dccc-4b52-866f-50d83e08768c service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:10.094151 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4407/8852] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:42:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:42:14.656748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268fac1-729e-4668-9d4c-2ee2af965443 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:14.708510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:14.722610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268fac1-729e-4668-9d4c-2ee2af965443 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:14.727646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:14.728277 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:14.728560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:14.728835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] PATH_INFO: `/v3/users/d190e16ba888408cbf2d918df8ca0e6e` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:14.730024 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d190e16ba888408cbf2d918df8ca0e6e)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:14.731957 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:14.778590 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd190e16ba888408cbf2d918df8ca0e6e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:14.790835 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd190e16ba888408cbf2d918df8ca0e6e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:14.797710 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd190e16ba888408cbf2d918df8ca0e6e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:14.804380 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd190e16ba888408cbf2d918df8ca0e6e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:14.805410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd190e16ba888408cbf2d918df8ca0e6e'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:14.811530 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4446/8853] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:42:14 2022] DELETE /identity/v3/users/d190e16ba888408cbf2d918df8ca0e6e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:14.816199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:14.816716 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:14.816847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:14.836319 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:14.838248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:14.894921 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4408/8854] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:14.906242 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:14.907654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:14.972762 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e791-8bef-49a0-ba91-2d4aa1b8f5f2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:14.977602 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc2175-d26d-45a8-8925-4ad0e5776803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:14.977908 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc2175-d26d-45a8-8925-4ad0e5776803 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:14.978195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc2175-d26d-45a8-8925-4ad0e5776803 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:14.978470 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc2175-d26d-45a8-8925-4ad0e5776803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:14.991480 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc2175-d26d-45a8-8925-4ad0e5776803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:14.994050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc2175-d26d-45a8-8925-4ad0e5776803 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.004408 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4447/8855] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:42:15.179087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.180898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:15.197567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77154adb-7b09-4778-a662-9bef7585aaae None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.202251 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.203427 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.203711 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.204148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] PATH_INFO: `/v3/projects/ac4fad0d8d164eefa7973417477d1f7e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.205274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4fad0d8d164eefa7973417477d1f7e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.207206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.256518 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4fad0d8d164eefa7973417477d1f7e', 'request_id': 'req-1e9df507-80a2-4b66-92aa-4379461effe6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:15.263009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4fad0d8d164eefa7973417477d1f7e', 'request_id': 'req-1e9df507-80a2-4b66-92aa-4379461effe6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:15.270417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4fad0d8d164eefa7973417477d1f7e', 'request_id': 'req-1e9df507-80a2-4b66-92aa-4379461effe6'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:15.275050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:15.283756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] The token cache is being invalidate because project ac4fad0d8d164eefa7973417477d1f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:42:15.284111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:15.285046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-1e9df507-80a2-4b66-92aa-4379461effe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:42:15.286818 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4409/8856] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:42:15 2022] DELETE /identity/v3/projects/ac4fad0d8d164eefa7973417477d1f7e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:15.293507 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd91f-578a-4468-9c82-df3bd794cc5a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.293814 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd91f-578a-4468-9c82-df3bd794cc5a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.294124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccd91f-578a-4468-9c82-df3bd794cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.313748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a2ccd91f-578a-4468-9c82-df3bd794cc5a None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:15.315010 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a2ccd91f-578a-4468-9c82-df3bd794cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:15.375160 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4448/8857] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:15.382988 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb989d24-728a-481d-be00-8d34862107e7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.383790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb989d24-728a-481d-be00-8d34862107e7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.384083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb989d24-728a-481d-be00-8d34862107e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.404162 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bb989d24-728a-481d-be00-8d34862107e7 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:15.405356 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bb989d24-728a-481d-be00-8d34862107e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:15.413197 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4410/8858] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:15.417498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccd91f-578a-4468-9c82-df3bd794cc5a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.427296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccd91f-578a-4468-9c82-df3bd794cc5a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.431959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.432558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.432844 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.433121 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.433761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:42:15.434171 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.436871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.445510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:15.449777 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4449/8859] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:42:15.454058 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb989d24-728a-481d-be00-8d34862107e7 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.464443 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb989d24-728a-481d-be00-8d34862107e7 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.469557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.470185 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.470471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.470745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.471378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.473298 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.500343 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4411/8860] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:42:15 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:42:15.504591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.516059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06358978-81a7-4231-9462-2a0921b6e0d8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.574409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.575015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.575300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.575577 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.576196 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.578150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.600409 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4450/8861] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:42:15 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:42:15.604657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.618480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb703-1bc8-48d3-963d-2719365711ab admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.623126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.623727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.624012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.624289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.624929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.626630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.636888 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4412/8862] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:42:15.641027 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.651689 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e660b-f64c-438d-9f02-e75145a78d92 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.656271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b7517-9e12-45ce-8174-c2c0b11eacf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.656864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7517-9e12-45ce-8174-c2c0b11eacf1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.657393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7517-9e12-45ce-8174-c2c0b11eacf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.657503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7517-9e12-45ce-8174-c2c0b11eacf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.658077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b7517-9e12-45ce-8174-c2c0b11eacf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.659668 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b7517-9e12-45ce-8174-c2c0b11eacf1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.670352 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4451/8863] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:42:15.674335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.684294 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba59e8-71dc-4ed7-a40d-fc48596d825c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.689064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.689709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.690100 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.690367 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] PATH_INFO: `/v3/projects/2bf5303213324a9591812e0b4debf54e/users/4086db48df2640109c3ef0f8720c36c8/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.705817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf5303213324a9591812e0b4debf54e, user_id=4086db48df2640109c3ef0f8720c36c8, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.708470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.721903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4413/8864] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:42:15 2022] PUT /identity/v3/projects/2bf5303213324a9591812e0b4debf54e/users/4086db48df2640109c3ef0f8720c36c8/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:15.727349 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1385f5-4392-4b44-80f1-24182837d61d None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.727825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1385f5-4392-4b44-80f1-24182837d61d None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.728101 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1385f5-4392-4b44-80f1-24182837d61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.747826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ab1385f5-4392-4b44-80f1-24182837d61d None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:15.749015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab1385f5-4392-4b44-80f1-24182837d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:15.808539 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4452/8865] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:42:15.818635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:15.820431 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:15.877984 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc96ba-1ec0-4117-9fff-10952c71f789 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:15.881526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:15.882214 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:15.882521 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:15.882794 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:15.892472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:15.893942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:15.903507 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4414/8866] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:42:19.903715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2766a-904a-43ae-8556-7606248a40dc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:19.904258 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2766a-904a-43ae-8556-7606248a40dc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:19.904362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2766a-904a-43ae-8556-7606248a40dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:19.933953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-4af2766a-904a-43ae-8556-7606248a40dc None None] MFA Rules not processed for user `4086db48df2640109c3ef0f8720c36c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:19.935397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-4af2766a-904a-43ae-8556-7606248a40dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:20.021026 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4453/8867] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:20.026233 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:20.038250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6789187-e0d7-45a1-a79f-bdf693e0fdbe service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:20.043918 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:20.044622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:20.045125 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:20.045477 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:20.046279 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:20.048389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:20.079746 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4415/8868] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:42:20 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:42:20.084006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2766a-904a-43ae-8556-7606248a40dc None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:20.096748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2766a-904a-43ae-8556-7606248a40dc None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:20.101665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:20.163072 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:20.163794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:20.165450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:20.165450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:20.165450 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:20.171452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:20.200988 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4454/8869] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:42:20 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:42:20.205010 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:20.216946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7600d3-baa7-4e67-a767-7601fe0e7ca4 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:20.223000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:20.223622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:20.223903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:20.224174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:20.224815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:20.226460 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:20.239288 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4416/8870] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:42:20.244068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:20.260156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f0015-00bb-45c1-b88c-0b1b4cdbb1c2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:20.266068 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:20.266747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:20.267037 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:20.267313 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] PATH_INFO: `/v3/projects/cfed2657f64f4f65a8de2f806e5cf983/users/feb90355846c4cfaa215fc88520c81ff/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:20.288167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfed2657f64f4f65a8de2f806e5cf983, user_id=feb90355846c4cfaa215fc88520c81ff, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:20.291953 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:20.307484 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4455/8871] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:42:20 2022] PUT /identity/v3/projects/cfed2657f64f4f65a8de2f806e5cf983/users/feb90355846c4cfaa215fc88520c81ff/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:20.311337 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:20.322960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae45f9-4202-48be-9d70-1e104f54ed3e admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:20.385597 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8b707-4691-41c6-9910-933b3a40ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:20.386425 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8b707-4691-41c6-9910-933b3a40ede4 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:20.386777 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8b707-4691-41c6-9910-933b3a40ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:20.387061 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8b707-4691-41c6-9910-933b3a40ede4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:20.387709 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8b707-4691-41c6-9910-933b3a40ede4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:20.389366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8b707-4691-41c6-9910-933b3a40ede4 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:20.391398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:20.404460 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4417/8872] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:42:20.408354 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:20.419872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe9589-f12f-41a1-acaf-00f796cdd2b6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:20.424652 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:20.425406 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:20.425653 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:20.426044 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] PATH_INFO: `/v3/projects/cfed2657f64f4f65a8de2f806e5cf983/users/feb90355846c4cfaa215fc88520c81ff/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:20.428165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfed2657f64f4f65a8de2f806e5cf983, user_id=feb90355846c4cfaa215fc88520c81ff, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:20.431008 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:20.442596 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4456/8873] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:42:20 2022] PUT /identity/v3/projects/cfed2657f64f4f65a8de2f806e5cf983/users/feb90355846c4cfaa215fc88520c81ff/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:42:24.875228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb738fe-67b6-4238-8902-269b0189504b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:24.875732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb738fe-67b6-4238-8902-269b0189504b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:24.875732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb738fe-67b6-4238-8902-269b0189504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:24.905799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-bbb738fe-67b6-4238-8902-269b0189504b None None] MFA Rules not processed for user `feb90355846c4cfaa215fc88520c81ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:24.907059 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-bbb738fe-67b6-4238-8902-269b0189504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:25.001876 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4418/8874] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:25.030635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:25.032421 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:25.115651 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc190e00-ce0e-4cb9-a1b7-a2b6f6be3d07 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:25.120442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56282ed0-5d6c-46e1-9139-29806d9814d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:25.121045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282ed0-5d6c-46e1-9139-29806d9814d6 service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:25.121323 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282ed0-5d6c-46e1-9139-29806d9814d6 service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:25.121995 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56282ed0-5d6c-46e1-9139-29806d9814d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:25.183274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:25.186833 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56282ed0-5d6c-46e1-9139-29806d9814d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:25.189014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56282ed0-5d6c-46e1-9139-29806d9814d6 service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:25.199987 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4457/8875] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:42:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:42:25.378965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73370c10-4e5d-49f7-94dd-927c7a6cf402 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:25.379347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73370c10-4e5d-49f7-94dd-927c7a6cf402 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:25.379630 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73370c10-4e5d-49f7-94dd-927c7a6cf402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:25.437335 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-73370c10-4e5d-49f7-94dd-927c7a6cf402 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:25.438911 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-73370c10-4e5d-49f7-94dd-927c7a6cf402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:25.551159 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4419/8876] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:42:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:25.603576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4681f-1c4c-4585-bf37-30bfb52c770a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:25.603864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4681f-1c4c-4585-bf37-30bfb52c770a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:25.604143 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4681f-1c4c-4585-bf37-30bfb52c770a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:25.664188 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-f0c4681f-1c4c-4585-bf37-30bfb52c770a None None] MFA Rules not processed for user `32f1fd57a99e4c0b9e50e997c5ee74f1`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:25.665599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-f0c4681f-1c4c-4585-bf37-30bfb52c770a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:25.746053 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4458/8877] 10.222.0.33 () {58 vars in 1040 bytes} [Sat Sep 10 21:42:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:25.752625 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73370c10-4e5d-49f7-94dd-927c7a6cf402 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:25.765446 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73370c10-4e5d-49f7-94dd-927c7a6cf402 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:25.770538 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ff4d6-6fdc-455a-b516-d8f138efaa52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:25.771154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff4d6-6fdc-455a-b516-d8f138efaa52 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:25.771440 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff4d6-6fdc-455a-b516-d8f138efaa52 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:25.771715 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff4d6-6fdc-455a-b516-d8f138efaa52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:25.780814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ff4d6-6fdc-455a-b516-d8f138efaa52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:25.782476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ff4d6-6fdc-455a-b516-d8f138efaa52 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:25.791752 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4420/8878] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:42:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:42:34.737943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ba6e-bc96-4251-a138-358d643c3897 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:34.738567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ba6e-bc96-4251-a138-358d643c3897 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:34.738567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4ba6e-bc96-4251-a138-358d643c3897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:34.757865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-93d4ba6e-bc96-4251-a138-358d643c3897 None None] MFA Rules not processed for user `4086db48df2640109c3ef0f8720c36c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:34.759085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-93d4ba6e-bc96-4251-a138-358d643c3897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:34.765957 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4459/8879] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:42:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:35.787811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=82 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:36.908863 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37f1bc-7ea1-494c-b44f-2c54b0e72ca1 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:36.909596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37f1bc-7ea1-494c-b44f-2c54b0e72ca1 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:36.909596 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37f1bc-7ea1-494c-b44f-2c54b0e72ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:36.931849 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ad37f1bc-7ea1-494c-b44f-2c54b0e72ca1 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:42:36.933436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ad37f1bc-7ea1-494c-b44f-2c54b0e72ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:42:36.941044 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4421/8880] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:42:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:42:36.959528 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4ba6e-bc96-4251-a138-358d643c3897 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:42:36.981391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4ba6e-bc96-4251-a138-358d643c3897 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:42:36.987768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:42:36.988566 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:42:36.989028 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:42:36.989397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:42:36.998962 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:42:37.000663 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:42:37.010443 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4460/8881] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:42:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:42:46.928087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:42:47.006642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=55 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:00.103345 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d300b1-10d9-48ed-813d-453a867cd4b0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:00.103803 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d300b1-10d9-48ed-813d-453a867cd4b0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:00.103900 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d300b1-10d9-48ed-813d-453a867cd4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:00.124169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-66d300b1-10d9-48ed-813d-453a867cd4b0 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:00.125357 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-66d300b1-10d9-48ed-813d-453a867cd4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:00.132388 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4422/8882] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:43:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:00.145103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:00.155583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a3e6c-7284-49f2-b309-cc42163f7fc6 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:00.160643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:00.161235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:00.161514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:00.161787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:00.170151 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:00.171724 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:00.180828 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4461/8883] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:43:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:43:00.681852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829ff83-89cf-43e8-995b-62329a94a7eb None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:00.682154 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829ff83-89cf-43e8-995b-62329a94a7eb None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:00.682435 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829ff83-89cf-43e8-995b-62329a94a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:00.702257 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1829ff83-89cf-43e8-995b-62329a94a7eb None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:00.703503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1829ff83-89cf-43e8-995b-62329a94a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:00.711391 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4423/8884] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:43:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:00.723588 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:00.734673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4698065-e08f-44e2-adac-238d0dcfdb2d service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:00.740636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:00.741244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:00.741529 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:00.741837 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:00.750817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:00.752481 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:00.760774 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4462/8885] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:43:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:43:09.338807 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf1149-8c6c-4314-b6c8-17ef1afdb7b4 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:09.339313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf1149-8c6c-4314-b6c8-17ef1afdb7b4 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:09.339414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf1149-8c6c-4314-b6c8-17ef1afdb7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:09.359283 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-75bf1149-8c6c-4314-b6c8-17ef1afdb7b4 None None] MFA Rules not processed for user `4086db48df2640109c3ef0f8720c36c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:09.360742 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-75bf1149-8c6c-4314-b6c8-17ef1afdb7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:09.367825 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4424/8886] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:43:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:09.952914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:09.954554 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:10.040725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74f05-20e3-4f74-8de5-afbb86a5b8b8 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:10.045698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:10.046366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:10.046646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:10.046919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:10.048654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:10.060989 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:10.062598 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:10.073247 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4463/8887] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:11.320155 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:11.320649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:11.320881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:11.340558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:11.341773 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:11.349233 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4425/8888] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:43:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:11.363486 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:11.375744 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212572f-32c7-485d-b720-988971c53cbb service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:11.380696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:11.381297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:11.381582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:11.381914 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:11.391138 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:11.392713 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:11.402460 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4464/8889] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:43:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:43:20.284745 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:20.286272 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:20.345423 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1dd26-becf-43a6-bdeb-767ffa95fc93 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:20.349113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:20.349724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:20.350025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:20.350299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:20.352024 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:20.364698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:20.366153 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:20.376948 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4426/8890] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:21.110885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:21.113793 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:21.199920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a5dea-de89-475a-8253-c6502419f766 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:21.211593 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9e297-2979-410d-8020-c5abb076e798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:21.212224 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9e297-2979-410d-8020-c5abb076e798 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:21.212508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9e297-2979-410d-8020-c5abb076e798 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:21.212842 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9e297-2979-410d-8020-c5abb076e798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:21.214614 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-e5e9e297-2979-410d-8020-c5abb076e798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:21.230739 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9e297-2979-410d-8020-c5abb076e798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:21.232367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9e297-2979-410d-8020-c5abb076e798 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:21.247059 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4465/8891] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:26.809768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:26.864567 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf9699-84b6-470f-921f-51adc9c9e9c0 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:26.882772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:26.883412 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:26.883695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:26.883969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:26.886340 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:26.924888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:26.926539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:26.932993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=66 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:26.941508 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4427/8892] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:43:31.239162 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=84 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:34.843118 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfa9ea-f805-4246-a438-093f870fb607 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:34.843700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfa9ea-f805-4246-a438-093f870fb607 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:34.843700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfa9ea-f805-4246-a438-093f870fb607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:34.863718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dfbfa9ea-f805-4246-a438-093f870fb607 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:34.864926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dfbfa9ea-f805-4246-a438-093f870fb607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:34.872659 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4466/8893] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:43:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:34.895593 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:34.906772 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328739f7-4b9a-49c6-9771-b21cecc6fc07 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:34.911960 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:34.912555 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:34.912853 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:34.913129 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:34.921416 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:34.923013 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:34.931588 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4428/8894] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:43:41.467558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfa9ea-f805-4246-a438-093f870fb607 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:41.551153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfa9ea-f805-4246-a438-093f870fb607 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:41.556274 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:41.557522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:41.557815 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:41.558134 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] PATH_INFO: `/v3/users/4086db48df2640109c3ef0f8720c36c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:41.559280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4086db48df2640109c3ef0f8720c36c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:41.561232 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:41.610740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4086db48df2640109c3ef0f8720c36c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.623325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4086db48df2640109c3ef0f8720c36c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.630251 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4086db48df2640109c3ef0f8720c36c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.637544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4086db48df2640109c3ef0f8720c36c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.638215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4086db48df2640109c3ef0f8720c36c8'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.644069 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4467/8895] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:43:41 2022] DELETE /identity/v3/users/4086db48df2640109c3ef0f8720c36c8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:43:41.818042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:41.829506 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc6f4a-e056-4bb9-b4b1-3c9cc55263d8 service swift] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:41.887359 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:41.887992 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:41.888276 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:41.888558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] PATH_INFO: `/v3/projects/2bf5303213324a9591812e0b4debf54e` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:41.889698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf5303213324a9591812e0b4debf54e)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:41.891649 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:41.942442 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf5303213324a9591812e0b4debf54e', 'request_id': 'req-593f7356-7bb3-4bed-b463-7e6358781011'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.950156 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf5303213324a9591812e0b4debf54e', 'request_id': 'req-593f7356-7bb3-4bed-b463-7e6358781011'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.957509 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf5303213324a9591812e0b4debf54e', 'request_id': 'req-593f7356-7bb3-4bed-b463-7e6358781011'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.981878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] The token cache is being invalidate because project 2bf5303213324a9591812e0b4debf54e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:43:41.982184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.982830 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:41.984312 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4429/8896] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:43:41 2022] DELETE /identity/v3/projects/2bf5303213324a9591812e0b4debf54e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:43:41.988551 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:41.990063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:42.003846 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33149ece-0218-4061-96cf-124ffe0048b8 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.060076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.060702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.060984 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.061261 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] PATH_INFO: `/v3/users/feb90355846c4cfaa215fc88520c81ff` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.062416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb90355846c4cfaa215fc88520c81ff)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.064340 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.068916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:42.113800 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb90355846c4cfaa215fc88520c81ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.128106 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb90355846c4cfaa215fc88520c81ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.135312 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb90355846c4cfaa215fc88520c81ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.142707 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb90355846c4cfaa215fc88520c81ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.143370 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-30595dd3-bcda-4cdd-9759-902a67144457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb90355846c4cfaa215fc88520c81ff'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.149232 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4468/8897] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:43:41 2022] DELETE /identity/v3/users/feb90355846c4cfaa215fc88520c81ff => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:43:42.327224 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:42.328671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:42.342760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7356-7bb3-4bed-b463-7e6358781011 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.401237 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.401912 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.402186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.402464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] PATH_INFO: `/v3/projects/cfed2657f64f4f65a8de2f806e5cf983` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.403741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfed2657f64f4f65a8de2f806e5cf983)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.405658 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.413693 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:42.454138 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfed2657f64f4f65a8de2f806e5cf983', 'request_id': 'req-4783a303-a3e9-4c1f-8146-4c56466643ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.460275 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfed2657f64f4f65a8de2f806e5cf983', 'request_id': 'req-4783a303-a3e9-4c1f-8146-4c56466643ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.467973 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfed2657f64f4f65a8de2f806e5cf983', 'request_id': 'req-4783a303-a3e9-4c1f-8146-4c56466643ca'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.479549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] The token cache is being invalidate because project cfed2657f64f4f65a8de2f806e5cf983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:43:42.479840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.480475 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-4783a303-a3e9-4c1f-8146-4c56466643ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:43:42.481968 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4430/8898] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:43:42 2022] DELETE /identity/v3/projects/cfed2657f64f4f65a8de2f806e5cf983 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:43:42.487439 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cb0e2-6d04-4a53-ab28-29c9a4e34807 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.487939 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cb0e2-6d04-4a53-ab28-29c9a4e34807 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.488047 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cb0e2-6d04-4a53-ab28-29c9a4e34807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.526982 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a75cb0e2-6d04-4a53-ab28-29c9a4e34807 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:42.528222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a75cb0e2-6d04-4a53-ab28-29c9a4e34807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:42.592963 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4469/8899] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:42.600949 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42cc6-cbaa-46bc-8a3c-98920002aad0 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.601241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42cc6-cbaa-46bc-8a3c-98920002aad0 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.601520 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42cc6-cbaa-46bc-8a3c-98920002aad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.621971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ead42cc6-cbaa-46bc-8a3c-98920002aad0 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:42.623169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ead42cc6-cbaa-46bc-8a3c-98920002aad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:42.630936 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4431/8900] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:42.635761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cb0e2-6d04-4a53-ab28-29c9a4e34807 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:42.645702 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cb0e2-6d04-4a53-ab28-29c9a4e34807 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.650442 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.651042 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.651329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.651611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.652286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:43:42.652594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.654235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.666039 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4470/8901] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:43:42.670241 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42cc6-cbaa-46bc-8a3c-98920002aad0 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:42.679478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42cc6-cbaa-46bc-8a3c-98920002aad0 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.684826 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.685441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.685728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.686030 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.686657 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.688602 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.716287 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4432/8902] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:43:42 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:43:42.720608 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:42.731642 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211cfa-ee54-4810-9fd3-e5ae0be44be6 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.782472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:42.791596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.792325 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.792640 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.792931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.793761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.796107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.817019 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4471/8903] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:43:42 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:43:42.821172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:42.832003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d6818-7408-442f-a9b3-62bb1d1a9591 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.836726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.837415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.837697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.838033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.838677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.840261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.850055 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4433/8904] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:43:42.854583 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:42.865449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4937d6-b26b-4a74-b7e8-de88a36f82f0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.871133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b368c7-608d-4f67-96e5-a1dfbd23686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.871735 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b368c7-608d-4f67-96e5-a1dfbd23686e admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.872018 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b368c7-608d-4f67-96e5-a1dfbd23686e admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.872291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b368c7-608d-4f67-96e5-a1dfbd23686e admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.872920 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b368c7-608d-4f67-96e5-a1dfbd23686e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.874532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b368c7-608d-4f67-96e5-a1dfbd23686e admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.885905 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4472/8905] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:43:42.890122 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:42.900329 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32306744-540d-42b2-b308-7a197d9a1b1d admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:42.905525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:42.906222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.906508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.906786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] PATH_INFO: `/v3/projects/89de7e77cc234d14943dc99d02388c45/users/2db198ac8f2842638e0edd931ca8d26d/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.921981 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89de7e77cc234d14943dc99d02388c45, user_id=2db198ac8f2842638e0edd931ca8d26d, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:42.924633 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:42.937722 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4434/8906] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:43:42 2022] PUT /identity/v3/projects/89de7e77cc234d14943dc99d02388c45/users/2db198ac8f2842638e0edd931ca8d26d/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:43:42.949133 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8332-c6c4-41a0-98af-a9dfff4a309e None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:42.949424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8332-c6c4-41a0-98af-a9dfff4a309e None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:42.949700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8332-c6c4-41a0-98af-a9dfff4a309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:42.979359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9ada8332-c6c4-41a0-98af-a9dfff4a309e None None] MFA Rules not processed for user `2db198ac8f2842638e0edd931ca8d26d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:42.980591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9ada8332-c6c4-41a0-98af-a9dfff4a309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:43.048985 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4473/8907] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:43:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:43.053965 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:43.117662 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91df13e-081b-4ae6-a3d0-0a3015732e93 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:43.123018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:43.123613 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.123894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.124172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.124804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:43.126781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:43.153595 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4435/8908] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:43:43 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:43:43.157938 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada8332-c6c4-41a0-98af-a9dfff4a309e None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:43.169006 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada8332-c6c4-41a0-98af-a9dfff4a309e None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:43.226716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:43.227374 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.227635 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.227975 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.228594 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:43.230562 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:43.252766 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4474/8909] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:43:43 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:43:43.256629 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:43.267663 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ca51a-9a20-4e93-8590-3cf03c533cf0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:43.272310 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:43.272902 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.273187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.273465 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.274133 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:43.275874 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:43.286284 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4436/8910] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:43:43.290264 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:43.300444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333660dc-0a7e-4bb5-9ee8-e5018dbc53b0 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:43.305094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:43.306306 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.306820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.307103 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] PATH_INFO: `/v3/projects/efc845d6525d4fa391f72a1609fdbdc1/users/f445d1ce927c4558aaf5661d7a009e9f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.322894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc845d6525d4fa391f72a1609fdbdc1, user_id=f445d1ce927c4558aaf5661d7a009e9f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:43.325540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:43.338007 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4475/8911] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:43:43 2022] PUT /identity/v3/projects/efc845d6525d4fa391f72a1609fdbdc1/users/f445d1ce927c4558aaf5661d7a009e9f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:43:43.341954 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:43.352064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b7482-9f08-441f-99bb-e59bcdb1d3e0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:43.355847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:43.412184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914483c-182b-41ef-a1f8-8984c876042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:43.412792 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914483c-182b-41ef-a1f8-8984c876042f admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.413111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914483c-182b-41ef-a1f8-8984c876042f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.413387 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914483c-182b-41ef-a1f8-8984c876042f admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.414046 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914483c-182b-41ef-a1f8-8984c876042f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:43.415896 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914483c-182b-41ef-a1f8-8984c876042f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:43.425596 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4437/8912] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:43:43.429757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:43.440899 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a021af7-8de6-46da-a7b7-6e447260eedf admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:43.445638 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:43.446343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.447293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.447576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] PATH_INFO: `/v3/projects/efc845d6525d4fa391f72a1609fdbdc1/users/f445d1ce927c4558aaf5661d7a009e9f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.449498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc845d6525d4fa391f72a1609fdbdc1, user_id=f445d1ce927c4558aaf5661d7a009e9f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:43.452165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:43.462250 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4476/8913] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:43:43 2022] PUT /identity/v3/projects/efc845d6525d4fa391f72a1609fdbdc1/users/f445d1ce927c4558aaf5661d7a009e9f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:43:43.471820 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.472110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.472415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.505532 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] MFA Rules not processed for user `f445d1ce927c4558aaf5661d7a009e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:43.506682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:43.575045 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4438/8914] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:43:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:43.589388 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:43.591130 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:43.611113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:43.670641 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8201d08-9e9f-4b35-a338-271668c394fc admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:43.676178 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4286930-86d8-4623-a4d7-55e04c77e9bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:43.676791 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4286930-86d8-4623-a4d7-55e04c77e9bb service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:43.677060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4286930-86d8-4623-a4d7-55e04c77e9bb service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:43.677330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4286930-86d8-4623-a4d7-55e04c77e9bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:43.733391 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4286930-86d8-4623-a4d7-55e04c77e9bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:43.735244 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4286930-86d8-4623-a4d7-55e04c77e9bb service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:43.745082 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4477/8915] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:47.142719 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:47.146937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:47.227750 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c968-6de7-4e9d-8de8-5b540d4befc5 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:47.232231 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:47.232840 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:47.233119 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:47.233391 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:47.242835 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:47.244263 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:47.254931 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4439/8916] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:50.456830 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:50.457297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:50.457395 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:50.480632 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] MFA Rules not processed for user `2db198ac8f2842638e0edd931ca8d26d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:50.481895 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:50.493916 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4478/8917] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:43:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:50.867549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:50.877501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb90ba-2842-4ad6-b886-73bea9a915b4 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:50.881623 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26285296-e44a-4e8d-8c82-8e8c0a4173d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:50.882439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285296-e44a-4e8d-8c82-8e8c0a4173d4 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:50.882760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285296-e44a-4e8d-8c82-8e8c0a4173d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:50.883131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285296-e44a-4e8d-8c82-8e8c0a4173d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:50.885756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-26285296-e44a-4e8d-8c82-8e8c0a4173d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:50.899610 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26285296-e44a-4e8d-8c82-8e8c0a4173d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:50.901642 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26285296-e44a-4e8d-8c82-8e8c0a4173d4 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:50.915933 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4440/8918] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:51.325339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:51.327085 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:51.461600 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f882b72-8035-4cb5-847f-6552ab8d769a None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:51.468510 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161b99f-2e00-44e8-b539-496b00252909 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:51.469207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b99f-2e00-44e8-b539-496b00252909 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:51.469676 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b99f-2e00-44e8-b539-496b00252909 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:51.470017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b99f-2e00-44e8-b539-496b00252909 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:51.472014 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-2161b99f-2e00-44e8-b539-496b00252909 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:51.486558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161b99f-2e00-44e8-b539-496b00252909 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:51.488192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161b99f-2e00-44e8-b539-496b00252909 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:51.500258 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4479/8919] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:52.028261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f9580-a122-463d-b629-b24536d757df None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:52.028734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f9580-a122-463d-b629-b24536d757df None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:52.028843 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f9580-a122-463d-b629-b24536d757df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:52.044187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=100 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:43:52.050234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ae4f9580-a122-463d-b629-b24536d757df None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:52.053211 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ae4f9580-a122-463d-b629-b24536d757df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:52.065298 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4441/8920] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Sep 10 21:43:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:52.094192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beac563-cf62-4879-872b-d18430d53ecf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:52.094720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beac563-cf62-4879-872b-d18430d53ecf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:52.095094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beac563-cf62-4879-872b-d18430d53ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:52.115911 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-8beac563-cf62-4879-872b-d18430d53ecf None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:52.117266 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8beac563-cf62-4879-872b-d18430d53ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:52.131188 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4480/8921] 10.222.0.33 () {58 vars in 1058 bytes} [Sat Sep 10 21:43:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:43:52.137934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f9580-a122-463d-b629-b24536d757df None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:52.149860 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f9580-a122-463d-b629-b24536d757df None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:52.156292 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:52.157062 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:52.157472 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:52.157756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:52.167800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:52.169410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:52.180284 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4442/8922] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:43:59.722905 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da3c5-b911-4752-b239-1bf376bc6abb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:59.723485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da3c5-b911-4752-b239-1bf376bc6abb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:59.723485 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da3c5-b911-4752-b239-1bf376bc6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:59.745684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b55da3c5-b911-4752-b239-1bf376bc6abb None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:43:59.746994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b55da3c5-b911-4752-b239-1bf376bc6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:43:59.755890 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4481/8923] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 10 21:43:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:43:59.768039 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:43:59.782149 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035980a-a745-4b83-a113-bf9f3ed28bf6 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:43:59.787207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:43:59.787817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:43:59.788083 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:43:59.788360 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:43:59.796672 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:43:59.798297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:43:59.808442 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4443/8924] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:43:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:44:09.740555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=90 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:44:09.803870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:44:58.298706 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cafb27-ee77-4de8-84d6-f9f21a24193c None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:44:58.299864 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cafb27-ee77-4de8-84d6-f9f21a24193c None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:44:58.300369 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cafb27-ee77-4de8-84d6-f9f21a24193c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:44:58.333464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c8cafb27-ee77-4de8-84d6-f9f21a24193c None None] MFA Rules not processed for user `2db198ac8f2842638e0edd931ca8d26d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:44:58.334719 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c8cafb27-ee77-4de8-84d6-f9f21a24193c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:44:58.344335 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4482/8925] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:44:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:45:08.327249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:45:11.811744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:45:11.814208 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:45:11.836126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5c5ed-a83e-44b6-99a4-8a0696153a1b service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:45:11.842906 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6d6a8-9b01-4d58-8444-84c8badc3f40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:45:11.843774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d6a8-9b01-4d58-8444-84c8badc3f40 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:45:11.844246 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d6a8-9b01-4d58-8444-84c8badc3f40 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:45:11.844938 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d6a8-9b01-4d58-8444-84c8badc3f40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:45:11.847814 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c2a6d6a8-9b01-4d58-8444-84c8badc3f40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:45:11.866362 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6d6a8-9b01-4d58-8444-84c8badc3f40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:45:11.868315 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6d6a8-9b01-4d58-8444-84c8badc3f40 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:45:11.887147 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4444/8926] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:45:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:45:21.875652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:46:15.971464 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baacb2-57ce-4a39-b397-e951e3a889d9 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:15.972050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baacb2-57ce-4a39-b397-e951e3a889d9 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:15.972050 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baacb2-57ce-4a39-b397-e951e3a889d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:16.022252 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-b8baacb2-57ce-4a39-b397-e951e3a889d9 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:46:16.023482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-b8baacb2-57ce-4a39-b397-e951e3a889d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:46:16.032991 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4483/8927] 10.222.0.33 () {58 vars in 1059 bytes} [Sat Sep 10 21:46:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:46:16.039048 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61f61-8a11-4d94-a9e7-2b9d5054c8b9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:16.039589 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61f61-8a11-4d94-a9e7-2b9d5054c8b9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:16.039687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61f61-8a11-4d94-a9e7-2b9d5054c8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:16.060924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1da61f61-8a11-4d94-a9e7-2b9d5054c8b9 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:46:16.062180 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1da61f61-8a11-4d94-a9e7-2b9d5054c8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:46:16.072280 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4445/8928] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:46:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:46:16.112124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baacb2-57ce-4a39-b397-e951e3a889d9 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:46:16.121656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baacb2-57ce-4a39-b397-e951e3a889d9 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:46:16.125482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0635c7-3a46-4bfd-b629-4e1672a38deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:46:16.126137 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0635c7-3a46-4bfd-b629-4e1672a38deb service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:16.126424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0635c7-3a46-4bfd-b629-4e1672a38deb service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:16.126703 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0635c7-3a46-4bfd-b629-4e1672a38deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:16.138122 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0635c7-3a46-4bfd-b629-4e1672a38deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:46:16.139591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0635c7-3a46-4bfd-b629-4e1672a38deb service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:46:16.150832 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4484/8929] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:46:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:46:17.416118 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:17.416628 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:17.416761 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:17.436545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:46:17.437741 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:46:17.447124 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4446/8930] 10.222.0.33 () {58 vars in 1059 bytes} [Sat Sep 10 21:46:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:46:17.453359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8449d-09b2-4e8a-9357-22bced9e6d34 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:17.454011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8449d-09b2-4e8a-9357-22bced9e6d34 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:17.454011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8449d-09b2-4e8a-9357-22bced9e6d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:17.474041 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-75a8449d-09b2-4e8a-9357-22bced9e6d34 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:46:17.475260 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-75a8449d-09b2-4e8a-9357-22bced9e6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:46:17.484740 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4485/8931] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:46:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:46:17.496199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:46:17.497568 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:46:17.508944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeed86-a717-4d3b-9664-8abf9ff03954 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:46:17.513266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:46:17.513882 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:17.514168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:17.514445 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:17.524673 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:46:17.526128 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:46:17.537547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4447/8932] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:46:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:46:27.469231 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=22 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:46:27.532901 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:46:45.472576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e88631-391e-4a1f-8d9f-f091007fa499 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:45.473128 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e88631-391e-4a1f-8d9f-f091007fa499 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:45.473267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e88631-391e-4a1f-8d9f-f091007fa499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:45.512567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-65e88631-391e-4a1f-8d9f-f091007fa499 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:46:45.513776 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-65e88631-391e-4a1f-8d9f-f091007fa499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:46:45.520502 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4486/8933] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 10 21:46:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:46:45.533145 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:46:45.541621 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade1b12-3c21-4700-a39e-e47434a8d76f service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:46:45.546188 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:46:45.546791 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:46:45.547076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:46:45.547355 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:46:45.554549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:46:45.555980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:46:45.564354 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4448/8934] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:46:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:46:55.509341 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=9 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:46:55.560924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:47:17.843414 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a7ab9-aa53-4058-8879-a50e5715f6cc None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:17.843923 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a7ab9-aa53-4058-8879-a50e5715f6cc None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:17.844020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a7ab9-aa53-4058-8879-a50e5715f6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:17.865377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a73a7ab9-aa53-4058-8879-a50e5715f6cc None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:47:17.866697 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a73a7ab9-aa53-4058-8879-a50e5715f6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:17.878248 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4487/8935] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:47:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:47:17.894451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:47:17.909684 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc67c5-2968-48e5-af98-65c58c11fed2 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:47:17.915549 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:47:17.916242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:17.917054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:17.917054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:17.926603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:47:17.928289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:47:17.941039 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4449/8936] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:47:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:47:24.645198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331f40d-24a5-4db8-a69f-77c7fd235faf None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:24.645657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331f40d-24a5-4db8-a69f-77c7fd235faf None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:24.645766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331f40d-24a5-4db8-a69f-77c7fd235faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:24.705596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9331f40d-24a5-4db8-a69f-77c7fd235faf None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:47:24.706882 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9331f40d-24a5-4db8-a69f-77c7fd235faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:24.717204 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4488/8937] 10.222.0.33 () {58 vars in 1058 bytes} [Sat Sep 10 21:47:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 21:47:24.723545 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:47:24.735632 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91683c3e-b5a0-43f9-ba22-f520e47c1717 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:47:24.741191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1302f-f1a5-4cb0-ae90-e0119b07f3ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:47:24.741793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1302f-f1a5-4cb0-ae90-e0119b07f3ce service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:24.742102 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1302f-f1a5-4cb0-ae90-e0119b07f3ce service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:24.742378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1302f-f1a5-4cb0-ae90-e0119b07f3ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:24.744109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ddf1302f-f1a5-4cb0-ae90-e0119b07f3ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:24.760604 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1302f-f1a5-4cb0-ae90-e0119b07f3ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:47:24.762297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1302f-f1a5-4cb0-ae90-e0119b07f3ce service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:47:24.775361 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4450/8938] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:47:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:47:32.583865 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5be732-c57c-403e-9eac-117cfe1bc206 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:32.584359 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5be732-c57c-403e-9eac-117cfe1bc206 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:32.584472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5be732-c57c-403e-9eac-117cfe1bc206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:32.606127 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-eb5be732-c57c-403e-9eac-117cfe1bc206 None None] MFA Rules not processed for user `2db198ac8f2842638e0edd931ca8d26d`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:47:32.607366 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eb5be732-c57c-403e-9eac-117cfe1bc206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:32.615468 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4489/8939] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:47:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:47:32.983581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a174c9-b7c2-4023-903e-9bf4929842ec None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:32.984169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a174c9-b7c2-4023-903e-9bf4929842ec None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:32.984169 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a174c9-b7c2-4023-903e-9bf4929842ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:33.010793 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e2a174c9-b7c2-4023-903e-9bf4929842ec None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:47:33.012535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e2a174c9-b7c2-4023-903e-9bf4929842ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:33.019711 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4451/8940] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:47:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:47:33.031782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5be732-c57c-403e-9eac-117cfe1bc206 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:47:33.046856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5be732-c57c-403e-9eac-117cfe1bc206 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:47:33.050933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:47:33.051656 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:33.052005 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:33.052363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:33.075714 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:47:33.077475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:47:33.091076 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4490/8941] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:47:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:47:35.648823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd43fa-8c90-4c50-8c55-e0faae1b2814 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:35.649297 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd43fa-8c90-4c50-8c55-e0faae1b2814 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:35.649577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd43fa-8c90-4c50-8c55-e0faae1b2814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:35.673708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-a2cd43fa-8c90-4c50-8c55-e0faae1b2814 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:47:35.675084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-a2cd43fa-8c90-4c50-8c55-e0faae1b2814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:35.683547 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4452/8942] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Sep 10 21:47:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:47:35.698493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:47:35.708409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f6a6a-ba37-4eea-87b8-eae0712884e6 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:47:35.712523 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:47:35.713141 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:35.713423 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:35.713700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:35.723230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:47:35.724929 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:47:35.734365 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4491/8943] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:47:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:47:36.693925 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0ec25-8733-4673-a7c5-5bd2ba187fc3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:36.694480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0ec25-8733-4673-a7c5-5bd2ba187fc3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:36.694480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0ec25-8733-4673-a7c5-5bd2ba187fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:36.718111 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-6fa0ec25-8733-4673-a7c5-5bd2ba187fc3 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:47:36.719334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6fa0ec25-8733-4673-a7c5-5bd2ba187fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:36.729406 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4453/8944] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:47:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:47:36.754949 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:47:36.770503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b8fd0-38c2-4e31-832f-b3a6aaa83308 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:47:36.776466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:47:36.777738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:36.778056 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:36.778332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:36.787570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:47:36.789154 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:47:36.800179 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4492/8945] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:47:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:47:42.078586 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731adda6-15a5-4aab-bf9d-0f7424306d2b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:42.079103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731adda6-15a5-4aab-bf9d-0f7424306d2b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:42.079206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731adda6-15a5-4aab-bf9d-0f7424306d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:42.100939 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-731adda6-15a5-4aab-bf9d-0f7424306d2b None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:47:42.102177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-731adda6-15a5-4aab-bf9d-0f7424306d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:47:42.112939 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4454/8946] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:47:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:47:42.128811 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:47:42.140058 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daf612-cd27-43dd-8c4c-e3a6c8bbad3d service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:47:42.145932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:47:42.146558 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:47:42.146856 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:47:42.147102 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:47:42.158069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:47:42.159646 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:47:42.170075 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4493/8947] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:47:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:47:52.095075 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=24 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:47:52.165692 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=44 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:48:20.956174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725796-2f2b-467e-8500-e71fcdcc01d7 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:48:20.956898 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725796-2f2b-467e-8500-e71fcdcc01d7 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:48:20.957252 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725796-2f2b-467e-8500-e71fcdcc01d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:48:20.981219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-21725796-2f2b-467e-8500-e71fcdcc01d7 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:48:20.982468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-21725796-2f2b-467e-8500-e71fcdcc01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:48:20.989490 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4455/8948] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:48:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:48:21.002107 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:48:21.011716 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099e2db-2253-4e0f-a2e2-0b18e43e63eb service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:48:21.015532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:48:21.016140 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:48:21.016484 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:48:21.016825 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:48:21.024385 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:48:21.025834 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:48:21.034425 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4494/8949] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:48:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:48:21.117816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc53a6-8ca0-46fa-ab72-333095d0964f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:48:21.118136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc53a6-8ca0-46fa-ab72-333095d0964f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:48:21.118417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc53a6-8ca0-46fa-ab72-333095d0964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:48:21.138746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-cbbc53a6-8ca0-46fa-ab72-333095d0964f None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:48:21.139946 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cbbc53a6-8ca0-46fa-ab72-333095d0964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:48:21.148473 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4456/8950] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:48:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:48:21.159927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:48:21.172195 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e20fc7-0fe5-471f-93e7-56e33b8c9b4b service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:48:21.177300 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:48:21.177924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:48:21.178213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:48:21.178492 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:48:21.187894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:48:21.189493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:48:21.199969 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4495/8951] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:48:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:48:21.386338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc53a6-8ca0-46fa-ab72-333095d0964f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:48:21.395891 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc53a6-8ca0-46fa-ab72-333095d0964f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:48:21.399577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:48:21.400178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:48:21.400459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:48:21.400735 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:48:21.403261 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:48:21.420595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:48:21.422044 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:48:21.435668 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4457/8952] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:48:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:48:21.776560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:48:21.786955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0f358-b0e9-41a2-a0b0-4004df9d2d28 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:48:21.792542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:48:21.793129 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:48:21.793440 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:48:21.793718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:48:21.795461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:48:21.806615 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:48:21.808817 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:48:21.818937 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4496/8953] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:48:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:48:31.428015 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:48:31.814580 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=18 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:48:43.425415 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:48:43.436612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b79168-ca49-490d-8bdd-8b49afe68717 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:48:43.442369 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d59698-a407-4acc-9a61-7eb9437915b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:48:43.442966 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d59698-a407-4acc-9a61-7eb9437915b6 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:48:43.443861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d59698-a407-4acc-9a61-7eb9437915b6 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:48:43.444147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d59698-a407-4acc-9a61-7eb9437915b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:48:43.452934 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d59698-a407-4acc-9a61-7eb9437915b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:48:43.454570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d59698-a407-4acc-9a61-7eb9437915b6 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:48:43.464416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4458/8954] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:48:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:48:53.459408 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:49:07.800233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:49:07.811599 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2719a-aee6-4d57-a294-1c31a8b887d4 service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:49:07.816617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:49:07.817215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:49:07.817494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:49:07.817770 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:49:07.826757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:49:07.828351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:49:07.838208 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4497/8955] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:49:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:49:17.834220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:49:41.930054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288be13a-ce8d-48b1-a744-5d549dc8d4c9 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:49:41.930924 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288be13a-ce8d-48b1-a744-5d549dc8d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:49:41.931201 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288be13a-ce8d-48b1-a744-5d549dc8d4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:49:41.952909 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-288be13a-ce8d-48b1-a744-5d549dc8d4c9 None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:49:41.954165 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-288be13a-ce8d-48b1-a744-5d549dc8d4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:49:41.963131 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4459/8956] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:49:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:49:41.981670 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:49:41.993453 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f8281-e36a-456b-8939-6fad8277a1cd service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:49:42.000634 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:49:42.001200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:49:42.001466 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:49:42.001743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:49:42.010902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:49:42.013166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:49:42.023783 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4498/8957] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:49:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:49:45.169760 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288be13a-ce8d-48b1-a744-5d549dc8d4c9 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:49:45.180867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288be13a-ce8d-48b1-a744-5d549dc8d4c9 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:49:45.185198 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc5aa-8308-481e-a7fb-82a63fa25509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:49:45.185854 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc5aa-8308-481e-a7fb-82a63fa25509 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:49:45.186385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc5aa-8308-481e-a7fb-82a63fa25509 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:49:45.186867 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc5aa-8308-481e-a7fb-82a63fa25509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:49:45.198053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc5aa-8308-481e-a7fb-82a63fa25509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:49:45.199695 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc5aa-8308-481e-a7fb-82a63fa25509 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:49:45.210723 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4460/8958] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:49:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:49:45.937284 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:49:45.951799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2346-c300-4a83-984d-f126e36847fb service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:49:45.960237 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:49:45.961227 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:49:45.961746 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:49:45.962299 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:49:45.973293 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:49:45.975983 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:49:45.990957 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4499/8959] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:49:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:49:47.896534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0a80-4ba7-4e34-8c2d-f6b84bcd4751 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:49:47.897009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0a80-4ba7-4e34-8c2d-f6b84bcd4751 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:49:47.897110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0a80-4ba7-4e34-8c2d-f6b84bcd4751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:49:47.917168 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-ae0e0a80-4ba7-4e34-8c2d-f6b84bcd4751 None None] MFA Rules not processed for user `2db198ac8f2842638e0edd931ca8d26d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:49:47.918462 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-ae0e0a80-4ba7-4e34-8c2d-f6b84bcd4751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:49:47.925559 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4461/8960] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:49:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:49:49.691699 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:49:49.703927 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab1844-dff0-4bff-81f6-36953adc86fe service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:49:49.709603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:49:49.710367 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:49:49.710726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:49:49.711073 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:49:49.720922 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:49:49.722708 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:49:49.734404 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4500/8961] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:49:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:49:57.911861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=12 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:49:59.729916 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=18 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:51:26.385683 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0a80-4ba7-4e34-8c2d-f6b84bcd4751 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:51:26.408944 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0a80-4ba7-4e34-8c2d-f6b84bcd4751 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:51:26.413229 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eec8c-a4d1-4193-ba1c-7e4f0de48d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:51:26.413859 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eec8c-a4d1-4193-ba1c-7e4f0de48d73 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:26.414176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eec8c-a4d1-4193-ba1c-7e4f0de48d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:26.414454 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eec8c-a4d1-4193-ba1c-7e4f0de48d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:26.434490 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6eec8c-a4d1-4193-ba1c-7e4f0de48d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:51:26.436653 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6eec8c-a4d1-4193-ba1c-7e4f0de48d73 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:51:26.460045 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4462/8962] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:51:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:51:27.646840 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:51:27.657110 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6df299-5f2b-4bf2-bffc-6723ba37891e service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:51:27.660849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:51:27.661444 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:27.661732 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:27.662043 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:27.672854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:51:27.674337 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:51:27.685205 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4501/8963] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:51:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:51:33.043217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314034-2b0a-413a-a3bc-f579bc4416bd None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:33.043787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314034-2b0a-413a-a3bc-f579bc4416bd None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:33.044072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314034-2b0a-413a-a3bc-f579bc4416bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:33.064768 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4f314034-2b0a-413a-a3bc-f579bc4416bd None None] MFA Rules not processed for user `b7ea3c7b376b40cfa12b65cd62046345`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:51:33.066002 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4f314034-2b0a-413a-a3bc-f579bc4416bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:51:33.076084 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4463/8964] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:51:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:51:33.092705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:51:33.110501 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b13fc-10d7-40e1-a9c7-072b0ed06b1e service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:51:33.116460 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417845e-433c-452a-977f-84283bf5a5fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:51:33.119737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417845e-433c-452a-977f-84283bf5a5fe service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:33.119737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417845e-433c-452a-977f-84283bf5a5fe service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:33.119737 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417845e-433c-452a-977f-84283bf5a5fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:33.128660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417845e-433c-452a-977f-84283bf5a5fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:51:33.130291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417845e-433c-452a-977f-84283bf5a5fe service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:51:33.140989 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4502/8965] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:51:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:51:38.745659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168a03-021d-490c-94fb-ff85b3f5b50b None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:38.746163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168a03-021d-490c-94fb-ff85b3f5b50b None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:38.746503 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168a03-021d-490c-94fb-ff85b3f5b50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:38.768929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-47168a03-021d-490c-94fb-ff85b3f5b50b None None] MFA Rules not processed for user `2db198ac8f2842638e0edd931ca8d26d`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:51:38.770199 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-47168a03-021d-490c-94fb-ff85b3f5b50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:51:38.778078 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4464/8966] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:51:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:51:39.123472 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1869fb-6f56-4ebf-a3d4-e285682eb148 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:39.123945 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1869fb-6f56-4ebf-a3d4-e285682eb148 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:39.124150 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1869fb-6f56-4ebf-a3d4-e285682eb148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:39.144215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-9e1869fb-6f56-4ebf-a3d4-e285682eb148 None None] MFA Rules not processed for user `ed817faea9e84f49bb4bfb065fe4adbd`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:51:39.145429 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-9e1869fb-6f56-4ebf-a3d4-e285682eb148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:51:39.152047 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4503/8967] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:51:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:51:39.164186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168a03-021d-490c-94fb-ff85b3f5b50b None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:51:39.174782 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168a03-021d-490c-94fb-ff85b3f5b50b None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:51:39.178300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:51:39.179192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:39.179471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:39.179763 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:39.189115 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:51:39.190584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:51:39.198892 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4465/8968] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:51:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:51:45.179959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1869fb-6f56-4ebf-a3d4-e285682eb148 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:51:45.190271 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1869fb-6f56-4ebf-a3d4-e285682eb148 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:51:45.194390 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:51:45.194994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:51:45.195279 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:51:45.195559 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:51:45.203126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:51:45.204563 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:51:45.212900 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4504/8969] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:51:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:51:49.195816 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=29 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:51:55.209270 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=21 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:02.068956 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:02.083370 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dc9ca-1038-4f36-94b3-b2a7b072f47a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:02.092682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:02.094601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:02.094601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:02.094937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:02.106174 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:02.107784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:02.126003 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4466/8970] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:04.512965 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:04.527867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7327d-2fef-48e5-bf94-09d83c0398c7 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:04.534181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c03ad-c280-4166-b006-bfe645b8bfd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:04.534850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c03ad-c280-4166-b006-bfe645b8bfd7 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:04.535197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c03ad-c280-4166-b006-bfe645b8bfd7 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:04.535514 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c03ad-c280-4166-b006-bfe645b8bfd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:04.546389 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c03ad-c280-4166-b006-bfe645b8bfd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:04.548060 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c03ad-c280-4166-b006-bfe645b8bfd7 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:04.559206 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4505/8971] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:09.005363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:09.084601 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663406e8-7857-4cf4-8f5c-ce29be80e78d service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:09.089287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:09.089963 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:09.090250 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:09.090526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] PATH_INFO: `/v3/users/2db198ac8f2842638e0edd931ca8d26d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:09.091664 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db198ac8f2842638e0edd931ca8d26d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:09.094368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:09.150888 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db198ac8f2842638e0edd931ca8d26d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.158654 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db198ac8f2842638e0edd931ca8d26d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.159325 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db198ac8f2842638e0edd931ca8d26d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.166620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db198ac8f2842638e0edd931ca8d26d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.167260 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db198ac8f2842638e0edd931ca8d26d'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.184878 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4467/8972] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:53:09 2022] DELETE /identity/v3/users/2db198ac8f2842638e0edd931ca8d26d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:53:09.189498 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:09.190029 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:09.190160 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:09.209958 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:53:09.211147 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:09.272438 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4506/8973] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:53:09.282616 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:09.284028 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:09.339842 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821db47-21be-4946-80dd-2dff7f50d512 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:09.344269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:09.344875 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:09.345158 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:09.345436 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:09.354526 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:09.355958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:09.361194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:09.366926 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4468/8974] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:09.557611 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:09.559495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:09.572572 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2dbcb-d62b-4bd8-b05f-149e45593eb1 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:09.577871 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:09.578731 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:09.579208 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:09.579499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] PATH_INFO: `/v3/projects/89de7e77cc234d14943dc99d02388c45` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:09.580718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=89de7e77cc234d14943dc99d02388c45)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:09.582861 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:09.632348 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89de7e77cc234d14943dc99d02388c45', 'request_id': 'req-ff681e10-a134-4f4a-9234-8622602f9fbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.638167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89de7e77cc234d14943dc99d02388c45', 'request_id': 'req-ff681e10-a134-4f4a-9234-8622602f9fbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.645199 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89de7e77cc234d14943dc99d02388c45', 'request_id': 'req-ff681e10-a134-4f4a-9234-8622602f9fbd'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.657990 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] The token cache is being invalidate because project 89de7e77cc234d14943dc99d02388c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:53:09.658315 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.658948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.660456 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4507/8975] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:53:09 2022] DELETE /identity/v3/projects/89de7e77cc234d14943dc99d02388c45 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:53:09.664464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:09.674894 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00031115-f048-4a76-8255-972ac85b681f service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:09.730414 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:09.731051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:09.731333 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:09.731612 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] PATH_INFO: `/v3/users/f445d1ce927c4558aaf5661d7a009e9f` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:09.732726 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f445d1ce927c4558aaf5661d7a009e9f)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:09.734851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:09.781271 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f445d1ce927c4558aaf5661d7a009e9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.788163 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f445d1ce927c4558aaf5661d7a009e9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.788836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f445d1ce927c4558aaf5661d7a009e9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.796248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f445d1ce927c4558aaf5661d7a009e9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.796950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-08cb0bec-840e-4052-ac8a-c871e3e5214a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f445d1ce927c4558aaf5661d7a009e9f'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:09.814933 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4469/8976] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:53:09 2022] DELETE /identity/v3/users/f445d1ce927c4558aaf5661d7a009e9f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:53:10.095077 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.096621 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:10.110474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff681e10-a134-4f4a-9234-8622602f9fbd admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.172452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.173096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.173381 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.173666 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] PATH_INFO: `/v3/projects/efc845d6525d4fa391f72a1609fdbdc1` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.174847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc845d6525d4fa391f72a1609fdbdc1)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.176771 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.188649 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:10.224960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc845d6525d4fa391f72a1609fdbdc1', 'request_id': 'req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:10.230456 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc845d6525d4fa391f72a1609fdbdc1', 'request_id': 'req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:10.237715 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc845d6525d4fa391f72a1609fdbdc1', 'request_id': 'req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:10.250511 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] The token cache is being invalidate because project efc845d6525d4fa391f72a1609fdbdc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:53:10.250805 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:10.251441 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-ad48e8dc-2f60-4b51-8d1a-18f1ad9ebe64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:53:10.252986 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4508/8977] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:53:10 2022] DELETE /identity/v3/projects/efc845d6525d4fa391f72a1609fdbdc1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:53:10.258508 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7a796-42bd-4ade-8f0e-c9d38c9bb056 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.259009 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7a796-42bd-4ade-8f0e-c9d38c9bb056 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.259120 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7a796-42bd-4ade-8f0e-c9d38c9bb056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.298635 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-32f7a796-42bd-4ade-8f0e-c9d38c9bb056 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:53:10.299796 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-32f7a796-42bd-4ade-8f0e-c9d38c9bb056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:10.361249 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4470/8978] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:53:10.369531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0647-5dff-4716-9af3-a792e10287b3 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.369850 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0647-5dff-4716-9af3-a792e10287b3 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.370132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0647-5dff-4716-9af3-a792e10287b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.392574 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-6f5d0647-5dff-4716-9af3-a792e10287b3 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:53:10.393802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f5d0647-5dff-4716-9af3-a792e10287b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:10.401952 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4509/8979] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:53:10.406405 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7a796-42bd-4ade-8f0e-c9d38c9bb056 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.412535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:10.416982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7a796-42bd-4ade-8f0e-c9d38c9bb056 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.423103 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.423723 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.424117 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.424581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.425228 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:53:10.425534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.427249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.439510 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4471/8980] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:53:10.444410 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0647-5dff-4716-9af3-a792e10287b3 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.454488 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0647-5dff-4716-9af3-a792e10287b3 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.459156 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.459752 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.460061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.460336 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.460961 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.462913 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.500108 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4510/8981] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:53:10 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:53:10.504334 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.514571 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d198c-ff03-47e4-b99b-35bf1aed5b00 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.574434 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.575042 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.575326 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.575603 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.576219 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.578804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.599598 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4472/8982] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:53:10 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:53:10.603766 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.615100 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe7fc3-9d4b-490f-ad1f-dba9ebea3f8d admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.620011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.620648 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.620932 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.621207 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.621907 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.623496 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.633574 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4511/8983] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:53:10.638652 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.649277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c2e1d-fd15-4033-a34f-a405adb03735 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.660485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd4e5b-f99c-4919-b6f0-2e2581e8da15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.661076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4e5b-f99c-4919-b6f0-2e2581e8da15 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.661354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4e5b-f99c-4919-b6f0-2e2581e8da15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.661624 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4e5b-f99c-4919-b6f0-2e2581e8da15 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.662287 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd4e5b-f99c-4919-b6f0-2e2581e8da15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.663969 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd4e5b-f99c-4919-b6f0-2e2581e8da15 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.675624 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4473/8984] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:53:10.679760 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.690020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bca477-586b-409c-8947-2a85cbbe6224 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.694677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.695332 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.695617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.695898 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] PATH_INFO: `/v3/projects/f6fed94219e94a079adb19e226519d93/users/e8f20ab4946f4ad6ac5637b10b9b7e2a/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.711088 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fed94219e94a079adb19e226519d93, user_id=e8f20ab4946f4ad6ac5637b10b9b7e2a, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.713747 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.727125 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4512/8985] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:53:10 2022] PUT /identity/v3/projects/f6fed94219e94a079adb19e226519d93/users/e8f20ab4946f4ad6ac5637b10b9b7e2a/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:53:10.732179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ead56e-ee09-4eba-8582-42126eeb9dad None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.732469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ead56e-ee09-4eba-8582-42126eeb9dad None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.732746 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ead56e-ee09-4eba-8582-42126eeb9dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.752341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-56ead56e-ee09-4eba-8582-42126eeb9dad None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:53:10.753531 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-56ead56e-ee09-4eba-8582-42126eeb9dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:10.812422 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4474/8986] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:53:10.821799 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:10.823542 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:10.880831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c85a2-89f2-4ea6-a613-cd754b872bc3 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:10.884459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:10.885061 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:10.885342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:10.885643 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:10.895617 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:10.897035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:10.907304 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4513/8987] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:15.129819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4cc5b-86d9-462a-96d3-a93d10f44143 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:15.130378 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4cc5b-86d9-462a-96d3-a93d10f44143 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:15.130483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4cc5b-86d9-462a-96d3-a93d10f44143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:15.164012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-36b4cc5b-86d9-462a-96d3-a93d10f44143 None None] MFA Rules not processed for user `e8f20ab4946f4ad6ac5637b10b9b7e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:53:15.165727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-36b4cc5b-86d9-462a-96d3-a93d10f44143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:15.237547 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:15.242134 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4475/8988] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:53:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:53:15.246416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:15.257868 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da1543-309d-44f4-b054-bf1485ae8492 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:15.262591 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:15.263206 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:15.263491 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:15.263768 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:15.264408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:15.266545 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:15.298310 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4514/8989] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:53:15 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:53:15.303267 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4cc5b-86d9-462a-96d3-a93d10f44143 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:15.325268 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4cc5b-86d9-462a-96d3-a93d10f44143 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:15.397722 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:15.398382 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:15.398668 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:15.398942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:15.399585 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:15.401655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:15.425682 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4476/8990] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:53:15 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:53:15.429751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:15.441570 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682dca7-1349-4d5d-b57c-0a8c2be2fb0a admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:15.448233 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:15.448722 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:15.449001 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:15.449292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:15.449935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:15.452885 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:15.456011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:15.466925 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4515/8991] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:53:15.472003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:15.482480 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11179-3aa0-4cc8-bcbf-8a984185ff54 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:15.488135 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:15.488790 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:15.489072 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:15.489351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] PATH_INFO: `/v3/projects/9370ba9f489e40e0ae6b23ba7f14f579/users/4a35ba9a479d43a9b20cae84d5d9bad3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:15.505785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9370ba9f489e40e0ae6b23ba7f14f579, user_id=4a35ba9a479d43a9b20cae84d5d9bad3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:15.508489 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:15.522667 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4477/8992] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:53:15 2022] PUT /identity/v3/projects/9370ba9f489e40e0ae6b23ba7f14f579/users/4a35ba9a479d43a9b20cae84d5d9bad3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:53:15.526698 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:15.537338 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679fc7b-7096-4052-97a7-19c7c7fffda1 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:15.604541 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2dc8d-1449-4da4-979f-6dfb9ed78fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:15.605165 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2dc8d-1449-4da4-979f-6dfb9ed78fab admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:15.605505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2dc8d-1449-4da4-979f-6dfb9ed78fab admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:15.605790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2dc8d-1449-4da4-979f-6dfb9ed78fab admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:15.606471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2dc8d-1449-4da4-979f-6dfb9ed78fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:15.608091 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2dc8d-1449-4da4-979f-6dfb9ed78fab admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:15.619951 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4516/8993] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:53:15.623948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:15.634848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324dcaf-9a0a-461f-abcb-bba099de6dd8 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:15.639598 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:15.640512 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:15.640661 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:15.640824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] PATH_INFO: `/v3/projects/9370ba9f489e40e0ae6b23ba7f14f579/users/4a35ba9a479d43a9b20cae84d5d9bad3/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:15.642781 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9370ba9f489e40e0ae6b23ba7f14f579, user_id=4a35ba9a479d43a9b20cae84d5d9bad3, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:15.645439 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:15.656906 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4478/8994] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:53:15 2022] PUT /identity/v3/projects/9370ba9f489e40e0ae6b23ba7f14f579/users/4a35ba9a479d43a9b20cae84d5d9bad3/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:53:20.187893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d5ff4-f223-4ac8-9cb5-76915b6a9fd0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:20.188959 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d5ff4-f223-4ac8-9cb5-76915b6a9fd0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:20.189567 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d5ff4-f223-4ac8-9cb5-76915b6a9fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:20.236247 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-c90d5ff4-f223-4ac8-9cb5-76915b6a9fd0 None None] MFA Rules not processed for user `4a35ba9a479d43a9b20cae84d5d9bad3`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:53:20.238144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c90d5ff4-f223-4ac8-9cb5-76915b6a9fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:20.324418 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4517/8995] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:53:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:53:20.352861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:20.355012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:20.476170 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617d83a-00f1-42c6-a15f-8db2e9edc090 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:20.481917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:20.482537 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:20.482817 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:20.483113 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:20.529987 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:20.559110 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:20.560799 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:20.572335 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4479/8996] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:21.640159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c39ec-4a4a-4ed8-8d36-038f1eb3cd26 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:21.640159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c39ec-4a4a-4ed8-8d36-038f1eb3cd26 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:21.640159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c39ec-4a4a-4ed8-8d36-038f1eb3cd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:21.661901 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-dd0c39ec-4a4a-4ed8-8d36-038f1eb3cd26 None None] MFA Rules not processed for user `e8f20ab4946f4ad6ac5637b10b9b7e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:53:21.663190 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-dd0c39ec-4a4a-4ed8-8d36-038f1eb3cd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:21.670628 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4518/8997] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:53:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:53:24.337671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:24.397947 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86741835-fd1c-4b9e-8802-41413b24323a service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:24.401712 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:24.402366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:24.402647 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:24.402923 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:24.404688 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:24.415008 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:24.416451 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:24.425416 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4480/8998] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:31.658751 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=67 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:31.727742 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c39ec-4a4a-4ed8-8d36-038f1eb3cd26 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:31.785849 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c39ec-4a4a-4ed8-8d36-038f1eb3cd26 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:31.792661 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:31.794531 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:31.794820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:31.795126 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:31.797581 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:53:31.831343 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:31.832847 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:31.843807 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4519/8999] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:34.417484 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:53:34.437338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4449722-a160-45b9-b298-c0eae6201616 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:53:34.441000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:53:34.441587 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:53:34.441921 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:53:34.442197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:53:34.451001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:53:34.453517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:53:34.462840 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4481/9000] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:53:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:53:41.839048 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=28 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:53:44.458677 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=47 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:54:13.742436 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:54:13.743872 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:54:13.821482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125b6c8-ce2e-42bb-a604-167ab6a33012 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:54:13.826517 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ba72f-b3da-44a6-9dd4-aec8519a095a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:54:13.827113 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba72f-b3da-44a6-9dd4-aec8519a095a service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:54:13.827393 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba72f-b3da-44a6-9dd4-aec8519a095a service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:54:13.827667 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba72f-b3da-44a6-9dd4-aec8519a095a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:54:13.830132 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c08ba72f-b3da-44a6-9dd4-aec8519a095a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:54:13.841416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ba72f-b3da-44a6-9dd4-aec8519a095a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:54:13.843017 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ba72f-b3da-44a6-9dd4-aec8519a095a service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:54:13.853632 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4520/9001] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:54:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:54:23.849586 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=36 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:54:48.909720 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:54:48.921660 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2fec4-f230-4c32-99d3-55758e3e3977 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:54:48.927027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:54:48.927655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:54:48.927937 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:54:48.928217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:54:48.930007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:54:48.941179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:54:48.943166 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:54:48.952747 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4482/9002] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:54:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:54:58.948788 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:55:46.978681 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:55:46.979209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:55:46.979329 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:55:47.031139 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:55:47.032339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:55:47.118139 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4521/9003] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:55:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:55:47.170194 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:55:47.171676 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:55:47.266876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5367a9-4cb4-47f3-a4cd-231310b6daa5 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:55:47.272131 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a2457-f8f0-4c57-abc3-a251ee8620ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:55:47.272727 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2457-f8f0-4c57-abc3-a251ee8620ec service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:55:47.273003 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2457-f8f0-4c57-abc3-a251ee8620ec service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:55:47.273273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a2457-f8f0-4c57-abc3-a251ee8620ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:55:47.281313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a2457-f8f0-4c57-abc3-a251ee8620ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:55:47.282916 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a2457-f8f0-4c57-abc3-a251ee8620ec service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:55:47.290828 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4483/9004] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:55:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:55:57.114877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=40 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:55:57.287779 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=41 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:56:00.511452 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:00.513176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:00.538280 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc126e61-215e-4b7f-b713-c242618f79f0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:00.541926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:00.542540 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:00.542823 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:00.543960 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:00.545878 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:00.555740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:00.557181 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:00.576639 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4522/9005] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:56:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:56:01.704852 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb66e11-d42e-4348-9347-367c016760d2 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:01.705373 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb66e11-d42e-4348-9347-367c016760d2 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:01.705501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb66e11-d42e-4348-9347-367c016760d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:01.735976 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-4eb66e11-d42e-4348-9347-367c016760d2 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:56:01.737171 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-4eb66e11-d42e-4348-9347-367c016760d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:01.747243 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4484/9006] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:56:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:56:01.758977 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:01.760401 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:01.843167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c7f56-a3da-4472-827d-9028dda5dfa2 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:01.846859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048b12e-03b9-4e6f-b712-44d70fded80e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:01.847458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b12e-03b9-4e6f-b712-44d70fded80e service glance] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:01.847745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b12e-03b9-4e6f-b712-44d70fded80e service glance] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:01.848022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b12e-03b9-4e6f-b712-44d70fded80e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:01.858212 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048b12e-03b9-4e6f-b712-44d70fded80e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:01.859630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048b12e-03b9-4e6f-b712-44d70fded80e service glance] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:01.870716 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4523/9007] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:56:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:56:01.937697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb66e11-d42e-4348-9347-367c016760d2 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:01.947500 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb66e11-d42e-4348-9347-367c016760d2 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:01.952000 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:01.952681 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:01.953143 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:01.953483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:01.955417 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:01.966066 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:01.968104 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:01.977394 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4485/9008] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:56:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:56:02.047451 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ff25-4fd6-4b14-887a-040344f99756 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:02.047740 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ff25-4fd6-4b14-887a-040344f99756 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:02.048035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ff25-4fd6-4b14-887a-040344f99756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:02.068458 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-d6e6ff25-4fd6-4b14-887a-040344f99756 None None] MFA Rules not processed for user `6307d15dca3043b987a28fc41515acc2`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:56:02.069660 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d6e6ff25-4fd6-4b14-887a-040344f99756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:02.079444 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4524/9009] 10.222.0.33 () {58 vars in 1018 bytes} [Sat Sep 10 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:56:02.089227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:02.098385 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ffe9-08c0-4f19-b8d2-f674f26c1f1a service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:02.102148 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875e219-eae6-4e50-8182-d3852ad7e9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:02.102778 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875e219-eae6-4e50-8182-d3852ad7e9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:02.103065 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875e219-eae6-4e50-8182-d3852ad7e9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:02.103347 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875e219-eae6-4e50-8182-d3852ad7e9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:02.114027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875e219-eae6-4e50-8182-d3852ad7e9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:02.115458 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875e219-eae6-4e50-8182-d3852ad7e9d3 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:02.126208 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4486/9010] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:56:02.191782 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ff25-4fd6-4b14-887a-040344f99756 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:02.203204 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ff25-4fd6-4b14-887a-040344f99756 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:02.208320 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:02.208918 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:02.209202 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:02.209479 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:02.211245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:02.222677 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:02.224265 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:02.234588 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4525/9011] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:56:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:56:04.908824 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db5bf-6ce9-4d25-af29-7a6b51c9792c None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:04.909358 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db5bf-6ce9-4d25-af29-7a6b51c9792c None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:04.909471 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003db5bf-6ce9-4d25-af29-7a6b51c9792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:04.929376 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-003db5bf-6ce9-4d25-af29-7a6b51c9792c None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:56:04.930698 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-003db5bf-6ce9-4d25-af29-7a6b51c9792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:04.939160 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4487/9012] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 21:56:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:56:04.951673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:04.963022 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcf23e-8a2a-4faa-a581-a2ed09b7259a service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:04.970471 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:04.971167 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:04.971513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:04.971859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:04.980522 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:04.982613 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:04.991323 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4526/9013] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:56:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:56:14.923819 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=22 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:56:14.987576 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=61 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:56:24.579666 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3ab1-10e2-431e-b512-59c7d015fd2f None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:24.580234 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3ab1-10e2-431e-b512-59c7d015fd2f None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:24.580354 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3ab1-10e2-431e-b512-59c7d015fd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:24.609516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-b83a3ab1-10e2-431e-b512-59c7d015fd2f None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:56:24.610747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-b83a3ab1-10e2-431e-b512-59c7d015fd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:24.618216 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4488/9014] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 21:56:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:56:24.631057 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:24.641573 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bb271-8ec8-4b40-a2c6-9f0d3d43a37b service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:24.646626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:24.647213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:24.647494 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:24.647767 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:24.655816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:24.657553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:24.665469 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4527/9015] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 21:56:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 21:56:34.604051 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=7 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:56:34.662350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:56:54.137277 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3ab1-10e2-431e-b512-59c7d015fd2f None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:56:54.146713 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3ab1-10e2-431e-b512-59c7d015fd2f None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:56:54.150266 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:56:54.150858 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:56:54.151134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:56:54.151401 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:56:54.153076 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:56:54.163064 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:56:54.164485 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:56:54.172953 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4489/9016] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:56:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:57:04.168786 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:03.173046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:03.174412 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:03.195761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b375-f68a-4d63-ab7f-65b2611b0039 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:03.202079 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:03.202802 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:03.203153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:03.204268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:03.206400 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:03.220166 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:03.221544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:03.232872 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4528/9017] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:03.359907 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:03.370685 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f45e32-06dd-4e90-9f8d-8b159937b390 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:03.374467 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc0e87-1280-4c3d-b40d-1353b8c67834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:03.375178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc0e87-1280-4c3d-b40d-1353b8c67834 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:03.375501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc0e87-1280-4c3d-b40d-1353b8c67834 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:03.375895 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc0e87-1280-4c3d-b40d-1353b8c67834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:03.377724 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-febc0e87-1280-4c3d-b40d-1353b8c67834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:03.392091 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc0e87-1280-4c3d-b40d-1353b8c67834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:03.393525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc0e87-1280-4c3d-b40d-1353b8c67834 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:03.404649 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4490/9018] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:04.612362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:04.621769 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a728d3-5497-4108-b90b-82c37740f520 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:04.625610 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:04.626245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:04.626532 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:04.626813 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:04.628533 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:04.641926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:04.643351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:04.654230 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4529/9019] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:08.170151 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:08.170655 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:08.170769 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:08.191255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] MFA Rules not processed for user `e8f20ab4946f4ad6ac5637b10b9b7e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:58:08.192640 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:08.200565 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4491/9020] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:58:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:58:11.649239 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:11.658459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1490d-4eff-4b93-bdf7-51ee08abd170 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:11.662191 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:11.662790 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:11.663069 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:11.663342 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:11.739894 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:11.741345 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:11.752778 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4530/9021] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:13.292982 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:13.294634 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:13.335582 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a567235-f77f-4e61-9045-6e290749da18 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:13.344322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:13.345269 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:13.345599 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:13.345991 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:13.347990 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:13.362054 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:13.366955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:13.375351 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4492/9022] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:16.238144 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:16.252339 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41b0a3-0e7e-4c2d-b356-aa938e56cf73 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:16.257553 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:16.258200 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:16.258489 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:16.258831 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:16.260704 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:16.274213 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:16.275852 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:16.290323 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4531/9023] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:16.308069 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:16.319789 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b18eb-505a-4a5e-8e87-0e47a163d254 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:16.328577 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:16.329248 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:16.329535 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:16.329811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] PATH_INFO: `/v3/users/e8f20ab4946f4ad6ac5637b10b9b7e2a` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:16.331127 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f20ab4946f4ad6ac5637b10b9b7e2a)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:16.333085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:16.382671 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f20ab4946f4ad6ac5637b10b9b7e2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.389809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f20ab4946f4ad6ac5637b10b9b7e2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.391085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f20ab4946f4ad6ac5637b10b9b7e2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.399052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f20ab4946f4ad6ac5637b10b9b7e2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.399805 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f20ab4946f4ad6ac5637b10b9b7e2a'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.418903 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4493/9024] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:58:16 2022] DELETE /identity/v3/users/e8f20ab4946f4ad6ac5637b10b9b7e2a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:58:16.592164 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:16.603383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83676641-56de-4bd0-8f7f-2bd721a9ab1d service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:16.667296 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:16.667931 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:16.668217 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:16.668493 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] PATH_INFO: `/v3/projects/f6fed94219e94a079adb19e226519d93` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:16.669603 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fed94219e94a079adb19e226519d93)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:16.671596 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:16.717544 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:16.719292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fed94219e94a079adb19e226519d93', 'request_id': 'req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.726099 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fed94219e94a079adb19e226519d93', 'request_id': 'req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.732286 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fed94219e94a079adb19e226519d93', 'request_id': 'req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.758192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] The token cache is being invalidate because project f6fed94219e94a079adb19e226519d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:58:16.758589 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.759214 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.760725 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4532/9025] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:58:16 2022] DELETE /identity/v3/projects/f6fed94219e94a079adb19e226519d93 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:58:16.764744 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:16.767007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:16.780473 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03b587-45e7-44de-a95b-61400d57dcb7 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:16.836776 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:16.837410 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:16.837692 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:16.838001 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] PATH_INFO: `/v3/users/4a35ba9a479d43a9b20cae84d5d9bad3` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:16.839116 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a35ba9a479d43a9b20cae84d5d9bad3)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:16.841027 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:16.874341 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:16.889566 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a35ba9a479d43a9b20cae84d5d9bad3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.896617 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a35ba9a479d43a9b20cae84d5d9bad3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.897273 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a35ba9a479d43a9b20cae84d5d9bad3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.904239 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a35ba9a479d43a9b20cae84d5d9bad3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.904958 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-b0a31353-819c-4338-af9f-6eed573e988e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a35ba9a479d43a9b20cae84d5d9bad3'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:16.923316 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4494/9026] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 21:58:16 2022] DELETE /identity/v3/users/4a35ba9a479d43a9b20cae84d5d9bad3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:58:17.096317 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.097877 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:17.113469 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e6ac7-7bc0-43ee-a116-65b2a4954dee admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.172806 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.173438 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.173725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.174033 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] PATH_INFO: `/v3/projects/9370ba9f489e40e0ae6b23ba7f14f579` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.175168 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9370ba9f489e40e0ae6b23ba7f14f579)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.177083 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.223604 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9370ba9f489e40e0ae6b23ba7f14f579', 'request_id': 'req-417c4459-bcdb-4b93-9cb4-dd79f539c981'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:17.229004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9370ba9f489e40e0ae6b23ba7f14f579', 'request_id': 'req-417c4459-bcdb-4b93-9cb4-dd79f539c981'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:17.236046 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9370ba9f489e40e0ae6b23ba7f14f579', 'request_id': 'req-417c4459-bcdb-4b93-9cb4-dd79f539c981'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:17.248919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] The token cache is being invalidate because project 9370ba9f489e40e0ae6b23ba7f14f579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 21:58:17.249555 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:17.249888 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-417c4459-bcdb-4b93-9cb4-dd79f539c981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 21:58:17.251544 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4533/9027] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 21:58:17 2022] DELETE /identity/v3/projects/9370ba9f489e40e0ae6b23ba7f14f579 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:58:17.256738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85eb1-1a96-4fd1-9751-ce412e45cb85 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.257025 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85eb1-1a96-4fd1-9751-ce412e45cb85 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.257323 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85eb1-1a96-4fd1-9751-ce412e45cb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.286580 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-13d85eb1-1a96-4fd1-9751-ce412e45cb85 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:58:17.287765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-13d85eb1-1a96-4fd1-9751-ce412e45cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:17.347805 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4495/9028] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:58:17.355902 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69cc73-3be9-4246-9cb2-01cdacf11eea None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.356192 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69cc73-3be9-4246-9cb2-01cdacf11eea None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.356473 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69cc73-3be9-4246-9cb2-01cdacf11eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.377235 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-bd69cc73-3be9-4246-9cb2-01cdacf11eea None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:58:17.378503 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-bd69cc73-3be9-4246-9cb2-01cdacf11eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:17.386353 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4534/9029] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:58:17.390708 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85eb1-1a96-4fd1-9751-ce412e45cb85 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.400861 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85eb1-1a96-4fd1-9751-ce412e45cb85 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.405433 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.406050 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.406870 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.407147 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.407903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93528) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 21:58:17.408206 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.409846 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.421274 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4496/9030] 10.222.0.33 () {58 vars in 1153 bytes} [Sat Sep 10 21:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 21:58:17.426176 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69cc73-3be9-4246-9cb2-01cdacf11eea None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.435730 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69cc73-3be9-4246-9cb2-01cdacf11eea None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.440612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.441215 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.441695 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.442298 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.442816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.444720 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.477437 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4535/9031] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:58:17 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:58:17.478539 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.490036 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624d6a4-4a6c-4899-adba-38982c782bc0 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.552441 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.553052 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.553338 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.553620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.554299 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.556223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.576718 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4497/9032] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:58:17 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 21:58:17.581650 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.592034 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afdf1-e3fa-49fc-8a09-f7bbe6e22091 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.597495 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.598131 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.598416 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.598691 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.599335 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.600926 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.610976 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4536/9033] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:58:17.615366 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.626107 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732665-21c6-41cc-baf5-477160475e6f admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.630989 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6562e3-9083-48d1-946e-3f6f06018b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.631620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6562e3-9083-48d1-946e-3f6f06018b49 admin admin] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.631903 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6562e3-9083-48d1-946e-3f6f06018b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.632177 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6562e3-9083-48d1-946e-3f6f06018b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.632808 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6562e3-9083-48d1-946e-3f6f06018b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.634470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6562e3-9083-48d1-946e-3f6f06018b49 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.646654 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4498/9034] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:58:17.650788 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.661630 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154948b-31c4-4842-889e-ede60101d430 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.666230 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.666867 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.667148 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.667424 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] PATH_INFO: `/v3/projects/c19c892be2ee4051a0a59def52561c8d/users/86134247af5940d6b3011c490cdf6d13/roles/9d599f0800724bd4b78e84df540784c8` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.679248 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:17.682743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19c892be2ee4051a0a59def52561c8d, user_id=86134247af5940d6b3011c490cdf6d13, role_id=9d599f0800724bd4b78e84df540784c8)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.686408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.699927 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4537/9035] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:58:17 2022] PUT /identity/v3/projects/c19c892be2ee4051a0a59def52561c8d/users/86134247af5940d6b3011c490cdf6d13/roles/9d599f0800724bd4b78e84df540784c8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:58:17.705734 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9ba49-71c5-4078-8509-110973fe4684 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.706049 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9ba49-71c5-4078-8509-110973fe4684 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.706322 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9ba49-71c5-4078-8509-110973fe4684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.719986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:17.726363 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-f8c9ba49-71c5-4078-8509-110973fe4684 None None] MFA Rules not processed for user `cd34ffbc794a4c04a8a5c4f195b6d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:58:17.728187 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-f8c9ba49-71c5-4078-8509-110973fe4684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:17.787317 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4499/9036] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:58:17.797096 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:17.799124 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:17.868774 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf97989-5a5d-465f-b904-aec76cc884f2 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:17.872636 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:17.873245 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:17.873530 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:17.873887 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:17.883881 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:17.885318 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:17.896467 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4538/9037] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:22.019929 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e4ee3-6f92-43aa-8c55-c648e17cb4a3 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:22.020452 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e4ee3-6f92-43aa-8c55-c648e17cb4a3 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:22.020590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e4ee3-6f92-43aa-8c55-c648e17cb4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:22.059053 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-427e4ee3-6f92-43aa-8c55-c648e17cb4a3 None None] MFA Rules not processed for user `86134247af5940d6b3011c490cdf6d13`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:58:22.060459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-427e4ee3-6f92-43aa-8c55-c648e17cb4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:22.133320 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4500/9038] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:58:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:58:22.138076 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:22.148700 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70194d-338b-4c73-9bf8-701a763ebd94 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:22.153725 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:22.154476 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:22.154822 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:22.155159 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:22.155866 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:22.158781 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:22.189330 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4539/9039] 10.222.0.33 () {60 vars in 1153 bytes} [Sat Sep 10 21:58:22 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:58:22.193482 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e4ee3-6f92-43aa-8c55-c648e17cb4a3 None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:22.206295 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e4ee3-6f92-43aa-8c55-c648e17cb4a3 None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:22.269313 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:22.269970 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:22.270258 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:22.270534 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] PATH_INFO: `/v3/users` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:22.271176 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:22.276584 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:22.301304 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4501/9040] 10.222.0.33 () {60 vars in 1144 bytes} [Sat Sep 10 21:58:22 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 21:58:22.302560 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:22.314490 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35249e1-0111-4472-9d51-43a5c0ca31a7 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:22.320291 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:22.321011 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:22.321365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:22.321709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:22.322459 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:22.324146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:22.339332 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4540/9041] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 21:58:22.340486 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:22.352184 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980aa2a-b0dc-4981-a668-5106a0840793 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:22.358253 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:22.358922 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:22.359207 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:22.360016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] PATH_INFO: `/v3/projects/87191a10e3164ac19f42e4d5436e5ff5/users/43f3212ffbaa45e6a462d5dfb5b3080f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:22.376765 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87191a10e3164ac19f42e4d5436e5ff5, user_id=43f3212ffbaa45e6a462d5dfb5b3080f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:22.379516 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:22.394762 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4502/9042] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:58:22 2022] PUT /identity/v3/projects/87191a10e3164ac19f42e4d5436e5ff5/users/43f3212ffbaa45e6a462d5dfb5b3080f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 21:58:22.398859 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:22.409812 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bd7b2-c654-40e6-93b7-5b236f3eafe9 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:22.478308 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1c5b1-b71f-4381-b093-e575f81c91d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:22.478935 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1c5b1-b71f-4381-b093-e575f81c91d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:22.479222 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1c5b1-b71f-4381-b093-e575f81c91d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:22.479506 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1c5b1-b71f-4381-b093-e575f81c91d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:22.480180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1c5b1-b71f-4381-b093-e575f81c91d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:22.482146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1c5b1-b71f-4381-b093-e575f81c91d7 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:22.497260 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4541/9043] 10.222.0.33 () {58 vars in 1122 bytes} [Sat Sep 10 21:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 21:58:22.498572 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:22.505950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:22.512476 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c505b-3039-449c-95c0-e387a8d09839 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:22.520513 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:22.521172 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] REQUEST_METHOD: `PUT` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:22.521459 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:22.521756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] PATH_INFO: `/v3/projects/87191a10e3164ac19f42e4d5436e5ff5/users/43f3212ffbaa45e6a462d5dfb5b3080f/roles/d819f0a222ec4024ac762c673f345cf5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:22.523797 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=87191a10e3164ac19f42e4d5436e5ff5, user_id=43f3212ffbaa45e6a462d5dfb5b3080f, role_id=d819f0a222ec4024ac762c673f345cf5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:22.526592 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:22.537154 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4503/9044] 10.222.0.33 () {60 vars in 1483 bytes} [Sat Sep 10 21:58:22 2022] PUT /identity/v3/projects/87191a10e3164ac19f42e4d5436e5ff5/users/43f3212ffbaa45e6a462d5dfb5b3080f/roles/d819f0a222ec4024ac762c673f345cf5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 21:58:27.192333 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82039f-f0d6-40f0-a636-35fded78ba94 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:27.192854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82039f-f0d6-40f0-a636-35fded78ba94 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:27.192854 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82039f-f0d6-40f0-a636-35fded78ba94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:27.231794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-ab82039f-f0d6-40f0-a636-35fded78ba94 None None] MFA Rules not processed for user `43f3212ffbaa45e6a462d5dfb5b3080f`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:58:27.234045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-ab82039f-f0d6-40f0-a636-35fded78ba94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:27.302933 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:27.341989 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4542/9045] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:58:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:58:27.364136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:27.366774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:27.448581 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c998-8836-4740-84f6-d4a08e8011bc admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:27.456191 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:27.456710 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:27.457071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:27.457413 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:27.535562 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:27.537186 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:27.547040 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4504/9046] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:28.630097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3899fc-c781-4d8a-a4a5-eeff56cf24a0 None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:28.630612 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3899fc-c781-4d8a-a4a5-eeff56cf24a0 None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:28.630718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3899fc-c781-4d8a-a4a5-eeff56cf24a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:28.650012 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1a3899fc-c781-4d8a-a4a5-eeff56cf24a0 None None] MFA Rules not processed for user `86134247af5940d6b3011c490cdf6d13`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 21:58:28.651236 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1a3899fc-c781-4d8a-a4a5-eeff56cf24a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:28.658284 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4543/9047] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 21:58:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 21:58:29.331396 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:29.333478 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:29.413815 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe73739-e127-42e0-90e4-4e0f567bee23 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:29.418864 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:29.419469 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:29.419747 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:29.420018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:29.421756 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:29.433398 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:29.435018 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:29.446255 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4505/9048] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:31.380597 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3899fc-c781-4d8a-a4a5-eeff56cf24a0 None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:31.448863 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3899fc-c781-4d8a-a4a5-eeff56cf24a0 None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:31.452786 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:31.453330 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:31.453696 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:31.453991 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:31.455870 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:31.466919 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:31.468305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:31.479499 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4544/9049] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 21:58:37.445979 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:37.462300 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579faad-f284-41f4-b9b9-f86ef7cff65c service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:37.468291 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:37.469638 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:37.469971 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:37.470255 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:37.472096 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:37.485274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:37.486914 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:37.498024 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4506/9050] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:39.156665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:39.158738 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:39.169198 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c96896-4c82-4a16-b89e-4c5a9504db95 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:39.172820 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:39.174365 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:39.174657 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:39.174930 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:39.183826 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:39.185241 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:39.195038 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4545/9051] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:41.049812 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:58:41.055876 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:41.063136 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cce7b0-3658-4869-8997-d86e64cb67b6 service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:58:41.069800 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:58:41.070456 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:58:41.070738 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:58:41.071012 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:58:41.072878 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:58:41.087483 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:58:41.089259 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:58:41.102498 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4507/9052] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:58:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:58:49.190851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=43 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:58:51.096881 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=5 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:59:18.763189 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:59:18.764571 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:59:18.778974 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7654-26ed-4076-b110-925f3e1aa790 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:59:18.783917 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6606d9-667d-41b2-8d1e-93d63a361d6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:59:18.785197 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6606d9-667d-41b2-8d1e-93d63a361d6e service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:59:18.785474 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6606d9-667d-41b2-8d1e-93d63a361d6e service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:59:18.785743 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6606d9-667d-41b2-8d1e-93d63a361d6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:59:18.787497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-3d6606d9-667d-41b2-8d1e-93d63a361d6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:59:18.798582 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6606d9-667d-41b2-8d1e-93d63a361d6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:59:18.800897 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6606d9-667d-41b2-8d1e-93d63a361d6e service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:59:18.811149 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4546/9053] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:59:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 21:59:28.806998 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 21:59:53.889504 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 21:59:53.901703 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580489bc-1763-4ce3-8fc9-7d9b9034e3af service nova] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 21:59:53.906687 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 21:59:53.907274 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 21:59:53.907551 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 21:59:53.907823 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 21:59:53.909574 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 21:59:53.921883 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 21:59:53.923470 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 21:59:53.933431 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4508/9054] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 21:59:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:00:03.929339 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:00:42.651268 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d345e95-5c27-4579-8bb6-059f62a06abb None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:00:42.651779 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d345e95-5c27-4579-8bb6-059f62a06abb None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:00:42.651891 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d345e95-5c27-4579-8bb6-059f62a06abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:00:42.671447 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-1d345e95-5c27-4579-8bb6-059f62a06abb None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 22:00:42.672654 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-1d345e95-5c27-4579-8bb6-059f62a06abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:00:42.729485 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4547/9055] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 22:00:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 22:00:42.780444 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:00:42.781940 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:00:42.852016 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a54368-a16a-4a12-b224-61eeade09690 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:00:42.857041 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0a2b8-6986-43d9-af0f-f0ffdaf43787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:00:42.857636 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a2b8-6986-43d9-af0f-f0ffdaf43787 service swift] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:00:42.857948 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a2b8-6986-43d9-af0f-f0ffdaf43787 service swift] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:00:42.858222 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0a2b8-6986-43d9-af0f-f0ffdaf43787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:00:42.867227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0a2b8-6986-43d9-af0f-f0ffdaf43787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:00:42.868809 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0a2b8-6986-43d9-af0f-f0ffdaf43787 service swift] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:00:42.877313 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4509/9056] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 22:00:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 22:00:52.288595 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d1685-2814-4065-83d6-b2acd1b854cd None None] REQUEST_METHOD: `POST` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:00:52.289158 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d1685-2814-4065-83d6-b2acd1b854cd None None] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:00:52.289292 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d1685-2814-4065-83d6-b2acd1b854cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:00:52.310089 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.auth.core [None req-a91d1685-2814-4065-83d6-b2acd1b854cd None None] MFA Rules not processed for user `86134247af5940d6b3011c490cdf6d13`. Rule list: `[]` (Enabled: `True`). {{(pid=93527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 22:00:52.311297 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-a91d1685-2814-4065-83d6-b2acd1b854cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:00:52.318227 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4548/9057] 10.222.0.33 () {58 vars in 958 bytes} [Sat Sep 10 22:00:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 22:00:52.873784 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=32 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:00:58.355419 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6fc6e-ec67-4d28-b85a-12a79c8a151a None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:00:58.356004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6fc6e-ec67-4d28-b85a-12a79c8a151a None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:00:58.356004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6fc6e-ec67-4d28-b85a-12a79c8a151a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:00:58.384221 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-50a6fc6e-ec67-4d28-b85a-12a79c8a151a None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 22:00:58.385950 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-50a6fc6e-ec67-4d28-b85a-12a79c8a151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:00:58.396344 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4510/9058] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 22:00:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 22:00:58.409705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d1685-2814-4065-83d6-b2acd1b854cd None None] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:00:58.422513 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d1685-2814-4065-83d6-b2acd1b854cd None None] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:00:58.430063 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:00:58.431015 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:00:58.431377 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:00:58.432701 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:00:58.442310 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:00:58.444303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:00:58.464564 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4549/9059] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 22:00:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 22:01:08.375847 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=3 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:01:08.449948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=34 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:01:59.490959 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6fc6e-ec67-4d28-b85a-12a79c8a151a None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:01:59.509705 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6fc6e-ec67-4d28-b85a-12a79c8a151a None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:01:59.513955 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:01:59.514557 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:01:59.514836 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:01:59.515109 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:01:59.516880 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:01:59.527217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:01:59.529282 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:01:59.537654 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4511/9060] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:01:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:02:09.534007 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=10 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:03:31.648364 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:31.677851 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177c7ac-7a00-436a-8a4e-ce65b0c8ba66 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:31.683462 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:31.684064 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:31.684350 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:31.684627 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:31.692684 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:31.694305 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:31.704153 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4550/9061] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:03:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:03:31.770384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:31.771834 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:03:31.828558 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9b12a-ebe3-40f2-b62f-a79052dafe92 service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:31.832134 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed48d-0d82-4441-9942-8ee12b78966f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cdff5f5500040439f4acafa6c9ad948', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:31.832732 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed48d-0d82-4441-9942-8ee12b78966f service glance] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:31.833017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed48d-0d82-4441-9942-8ee12b78966f service glance] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:31.833289 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed48d-0d82-4441-9942-8ee12b78966f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:31.835004 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-cdaed48d-0d82-4441-9942-8ee12b78966f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:03:31.899529 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaed48d-0d82-4441-9942-8ee12b78966f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:31.900879 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaed48d-0d82-4441-9942-8ee12b78966f service glance] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:31.912032 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4512/9062] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:03:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:03:38.842180 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:38.844552 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:03:38.891216 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9199f-24be-4ce5-84ab-5508877565a7 service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:38.897626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:38.897626 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:38.897948 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:38.898225 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:38.911893 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:38.913465 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:38.924603 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4551/9063] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:03:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:03:39.793196 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc335dc-28e1-4344-92b8-ef814324d303 None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:39.793682 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc335dc-28e1-4344-92b8-ef814324d303 None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:39.793804 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc335dc-28e1-4344-92b8-ef814324d303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:39.814123 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-0dc335dc-28e1-4344-92b8-ef814324d303 None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 22:03:39.815330 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-0dc335dc-28e1-4344-92b8-ef814324d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:03:39.822626 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4513/9064] 10.222.0.33 () {58 vars in 1009 bytes} [Sat Sep 10 22:03:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 22:03:39.864499 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:39.876475 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6671961-7cef-4fef-b81e-5353d9ad7b7e service nova] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:39.881502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:39.882240 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:39.882520 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:39.882794 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:39.890723 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:39.892304 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:39.900945 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4552/9065] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 22:03:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 22:03:49.613517 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57b9b-7454-4b97-bd90-58b4db5cdfae None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:49.614085 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57b9b-7454-4b97-bd90-58b4db5cdfae None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:49.614197 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57b9b-7454-4b97-bd90-58b4db5cdfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:49.634591 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e8a57b9b-7454-4b97-bd90-58b4db5cdfae None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 22:03:49.635831 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e8a57b9b-7454-4b97-bd90-58b4db5cdfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:03:49.643381 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4514/9066] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 22:03:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 22:03:49.655408 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:49.665994 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcbaf5-7a81-456e-a891-2dade001242c service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:49.671003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:49.672203 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:49.672482 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:49.672755 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:49.680647 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:49.682253 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:49.690925 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4553/9067] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 22:03:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 22:03:51.139466 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a57b9b-7454-4b97-bd90-58b4db5cdfae None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:51.159974 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a57b9b-7454-4b97-bd90-58b4db5cdfae None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:51.167620 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7d19d-7072-4392-b20f-9bde928d3229 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6307d15dca3043b987a28fc41515acc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:51.168420 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7d19d-7072-4392-b20f-9bde928d3229 service nova] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:51.168810 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7d19d-7072-4392-b20f-9bde928d3229 service nova] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:51.169192 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7d19d-7072-4392-b20f-9bde928d3229 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:51.179993 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7d19d-7072-4392-b20f-9bde928d3229 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:51.182071 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7d19d-7072-4392-b20f-9bde928d3229 service nova] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:51.195240 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4515/9068] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:03:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:03:51.373761 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:51.384351 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e615d-7e01-417e-a81e-8d051ed3a02e service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:51.388142 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:51.388605 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:51.388884 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:51.389153 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:51.390924 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:03:51.404363 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:51.406362 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:51.417480 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4554/9069] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:03:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:03:55.007084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d8f6-3d34-4090-a3a7-85bd2fe107fe None None] REQUEST_METHOD: `POST` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:55.007524 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d8f6-3d34-4090-a3a7-85bd2fe107fe None None] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:55.007659 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423d8f6-3d34-4090-a3a7-85bd2fe107fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:55.027178 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.auth.core [None req-e423d8f6-3d34-4090-a3a7-85bd2fe107fe None None] MFA Rules not processed for user `dcb3cf5d0ccd4ee2a1f71963dbcdfb18`. Rule list: `[]` (Enabled: `True`). {{(pid=93528) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 22:03:55.028372 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-e423d8f6-3d34-4090-a3a7-85bd2fe107fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:03:55.036340 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4516/9070] 10.222.0.33 () {58 vars in 967 bytes} [Sat Sep 10 22:03:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 22:03:55.048405 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:55.059497 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2e160-363a-4f13-b5da-7b81f5b97c60 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:55.064787 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f1fd57a99e4c0b9e50e997c5ee74f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:55.065387 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:55.065673 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:55.066003 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:55.074146 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:55.075718 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:55.084465 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4555/9071] 10.222.0.33 () {60 vars in 1384 bytes} [Sat Sep 10 22:03:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 22:03:56.378622 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423d8f6-3d34-4090-a3a7-85bd2fe107fe None None] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:03:56.389084 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423d8f6-3d34-4090-a3a7-85bd2fe107fe None None] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:03:56.394321 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:03:56.395368 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:03:56.395873 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:03:56.396318 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:03:56.405533 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:03:56.407244 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:03:56.416731 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4517/9072] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:03:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:04:00.015860 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:00.020220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=58 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:04:00.025808 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb533a6d-3ef0-4392-8c2c-71e8bc2e0d94 service swift] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:00.029705 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed817faea9e84f49bb4bfb065fe4adbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:00.030383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:00.030665 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:00.030943 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:00.110726 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:00.112259 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:00.137022 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4556/9073] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:04:03.327217 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:03.333962 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=70 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:04:03.340787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56071-c318-4daa-9769-5fbbbba0614d service neutron] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:03.346179 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:03.346862 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:03.347215 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:03.347570 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:03.349384 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:04:03.362249 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:03.364087 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:03.376457 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4518/9074] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:04:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:04:06.628249 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:06.638525 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46e588-e74a-451f-9ced-d55a67121b80 service neutron] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:06.643409 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:06.644045 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:06.644328 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:06.644607 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] PATH_INFO: `/v3/users/86134247af5940d6b3011c490cdf6d13` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:06.646020 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86134247af5940d6b3011c490cdf6d13)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:06.647955 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:06.695816 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86134247af5940d6b3011c490cdf6d13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:06.710373 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86134247af5940d6b3011c490cdf6d13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:06.717505 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86134247af5940d6b3011c490cdf6d13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:06.725035 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86134247af5940d6b3011c490cdf6d13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:06.725757 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86134247af5940d6b3011c490cdf6d13'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:06.731862 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4557/9075] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 22:04:06 2022] DELETE /identity/v3/users/86134247af5940d6b3011c490cdf6d13 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 22:04:06.906851 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:06.920578 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f494b2-3180-45e7-9e9e-19b6dd359d9d service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:06.979031 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:06.979697 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:06.979980 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:06.980281 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] PATH_INFO: `/v3/projects/c19c892be2ee4051a0a59def52561c8d` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:06.981421 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19c892be2ee4051a0a59def52561c8d)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:06.983986 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:07.030139 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19c892be2ee4051a0a59def52561c8d', 'request_id': 'req-531d4d51-b62a-48f2-b54b-36ee9c31f76c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.036223 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19c892be2ee4051a0a59def52561c8d', 'request_id': 'req-531d4d51-b62a-48f2-b54b-36ee9c31f76c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.043389 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19c892be2ee4051a0a59def52561c8d', 'request_id': 'req-531d4d51-b62a-48f2-b54b-36ee9c31f76c'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.054561 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] The token cache is being invalidate because project c19c892be2ee4051a0a59def52561c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 22:04:07.054857 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.055525 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.057038 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4519/9076] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 22:04:06 2022] DELETE /identity/v3/projects/c19c892be2ee4051a0a59def52561c8d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 22:04:07.061032 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:07.063031 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:04:07.076051 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b0c6-0307-4ff5-966c-36e5b70c27d4 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:07.134209 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:07.134836 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:07.135114 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:07.135397 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] PATH_INFO: `/v3/users/43f3212ffbaa45e6a462d5dfb5b3080f` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:07.136508 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43f3212ffbaa45e6a462d5dfb5b3080f)` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:07.138449 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:07.139892 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:04:07.185538 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43f3212ffbaa45e6a462d5dfb5b3080f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.198445 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43f3212ffbaa45e6a462d5dfb5b3080f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.205944 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43f3212ffbaa45e6a462d5dfb5b3080f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.213383 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43f3212ffbaa45e6a462d5dfb5b3080f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.214052 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.notifications [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43f3212ffbaa45e6a462d5dfb5b3080f'} {{(pid=93527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.220770 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4558/9077] 10.222.0.33 () {58 vars in 1224 bytes} [Sat Sep 10 22:04:07 2022] DELETE /identity/v3/users/43f3212ffbaa45e6a462d5dfb5b3080f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 22:04:07.399033 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:07.400464 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:04:07.413614 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d4d51-b62a-48f2-b54b-36ee9c31f76c admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:07.475595 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd34ffbc794a4c04a8a5c4f195b6d3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff6cfd9ba910450d96d6133cd5225f2f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:07.476225 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:07.476510 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:07.476785 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] PATH_INFO: `/v3/projects/87191a10e3164ac19f42e4d5436e5ff5` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:07.477917 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87191a10e3164ac19f42e4d5436e5ff5)` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:07.479844 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:07.503468 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:04:07.528404 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87191a10e3164ac19f42e4d5436e5ff5', 'request_id': 'req-c54b0316-100b-4693-8bda-c65d36e8f935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.534493 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87191a10e3164ac19f42e4d5436e5ff5', 'request_id': 'req-c54b0316-100b-4693-8bda-c65d36e8f935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.541590 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87191a10e3164ac19f42e4d5436e5ff5', 'request_id': 'req-c54b0316-100b-4693-8bda-c65d36e8f935'} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.551926 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] The token cache is being invalidate because project 87191a10e3164ac19f42e4d5436e5ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93528) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 22:04:07.552213 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.552838 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.notifications [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93528) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 22:04:07.554397 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4520/9078] 10.222.0.33 () {58 vars in 1233 bytes} [Sat Sep 10 22:04:07 2022] DELETE /identity/v3/projects/87191a10e3164ac19f42e4d5436e5ff5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 22:04:17.216461 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:04:17.548928 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:04:24.737220 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:24.738745 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:04:24.818832 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f92288-b8b7-4491-b1c0-b64d55252d82 admin admin] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:24.880145 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:24.880748 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:24.881030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:24.881303 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:24.883030 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.fernet_utils [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:04:24.894004 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:24.895569 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:24.905420 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4559/9079] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:04:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:04:34.901094 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=62 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:04:56.844328 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:04:56.845811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:04:56.861787 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b0316-100b-4693-8bda-c65d36e8f935 admin admin] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:04:56.867126 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:04:56.867728 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:04:56.868011 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:04:56.868284 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:04:56.870017 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.fernet_utils [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93528) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 22:04:56.881203 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:04:56.882811 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:04:56.892514 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4521/9080] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:04:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:05:06.888411 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:09:30.754754 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] Authenticating user token {{(pid=93527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:09:30.765267 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ff69a-3a28-4747-9723-7b5875984eec service placement] Validating token access rules against request {{(pid=93527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:09:30.770502 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5db71-2b43-41c8-a147-6416c3b990b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:09:30.771097 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5db71-2b43-41c8-a147-6416c3b990b6 service placement] REQUEST_METHOD: `GET` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:09:30.771431 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5db71-2b43-41c8-a147-6416c3b990b6 service placement] SCRIPT_NAME: `/identity` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:09:30.771709 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5db71-2b43-41c8-a147-6416c3b990b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:09:30.780059 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5db71-2b43-41c8-a147-6416c3b990b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:09:30.781631 ncdv-master-1 devstack@keystone.service[93527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5db71-2b43-41c8-a147-6416c3b990b6 service placement] RBAC: Authorization granted {{(pid=93527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:09:30.791497 ncdv-master-1 devstack@keystone.service[93527]: [pid: 93527|app: 0|req: 4560/9081] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:09:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 22:09:40.787398 ncdv-master-1 devstack@keystone.service[93527]: DEBUG dbcounter [-] [93527] Writing DB stats keystone:SELECT=6 {{(pid=93527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 22:10:01.814774 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] Authenticating user token {{(pid=93528) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 22:10:01.825242 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f2441-5543-45b8-b8ee-e7b708e2ec39 service placement] Validating token access rules against request {{(pid=93528) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 22:10:01.830351 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459dc53-fcca-41f4-b71a-08dcba2d3917 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ea3c7b376b40cfa12b65cd62046345', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a01cb3a7943240f6984664abc52ce14e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93528) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 22:10:01.830942 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459dc53-fcca-41f4-b71a-08dcba2d3917 service placement] REQUEST_METHOD: `GET` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 22:10:01.831227 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459dc53-fcca-41f4-b71a-08dcba2d3917 service placement] SCRIPT_NAME: `/identity` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 22:10:01.831501 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459dc53-fcca-41f4-b71a-08dcba2d3917 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93528) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 22:10:01.839848 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8459dc53-fcca-41f4-b71a-08dcba2d3917 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 22:10:01.841422 ncdv-master-1 devstack@keystone.service[93528]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8459dc53-fcca-41f4-b71a-08dcba2d3917 service placement] RBAC: Authorization granted {{(pid=93528) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 22:10:01.851909 ncdv-master-1 devstack@keystone.service[93528]: [pid: 93528|app: 0|req: 4522/9082] 10.222.0.33 () {60 vars in 1365 bytes} [Sat Sep 10 22:10:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 22:10:11.847733 ncdv-master-1 devstack@keystone.service[93528]: DEBUG dbcounter [-] [93528] Writing DB stats keystone:SELECT=6 {{(pid=93528) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}